Re: [DISCUSS] Pub/Sub Streamer Implementation

[Emmanouil Gkatziouras <gk...@gmail.com>]

Hi all!

This is an update. The integration is tested with mocks based on the
original Ignite repo [1].
I am currently waiting for the first commits on the new integration project
to take place in order to create my pull request accordingly to the initial
commits.
In order to simulate the Pub/Sub server in a test I used a test server
implementation [2]. Unfortunately they don't have this util uploaded in a
maven repo, thus it needs to be hosted on a maven repo (Apache License
2.0). Is there a repo for utilities that Ignite uses for the test/build?
I have also created a ticket [3]

Kind regards

[1] https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
[2] https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
[3] https://issues.apache.org/jira/browse/IGNITE-12262

*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura


On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <sa...@gmail.com> wrote:

> Hello Ilya,
>
> Thank you for your email. Yes, before we remove any integration we will
> move them to another repository and release.
>
> We are thinking we will start with Pub/Sub Streamer implementation and
> continue to move other integration as mentioned in the following doc.
>
>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>
> Regards,
> Saikat
>
>
> On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
> ilya.kasnacheev@gmail.com>
> wrote:
>
> > Hello!
> >
> > My take on this: we want to offload some of integrations, but please
> commit
> > and release them to another project first, and only then we will decide
> to
> > remove them from our repository.
> >
> > Please also make sure they are available with same artifact name and API.
> > If this has to be changed, let's wait for AI 3.0
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <sa...@gmail.com>:
> >
> > > Hello Emmanouil,
> > >
> > > As discussed earlier, I discussed with Apache Bahir community and they
> > are
> > > interested to have Apache Ignite extensions as part of Apache Bahir
> > > project.
> > >
> > > Can you please share Pub/Sub streamer implementation details with
> Apache
> > > Bahir community and request for feedback.
> > >
> > > https://bahir.apache.org/
> > >
> > > I have also requested for contributor access in Jira for Apache Bahir
> > > project so that I can create issues and assign to myself.
> > >
> > > I can help with code reviews as well.
> > >
> > > Here is the email thread for reference
> > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > >
> > > Regards,
> > > Saikat
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <saikat.maitra@gmail.com
> >
> > > wrote:
> > >
> > > > Hello,
> > > >
> > > > As discussed I have created following discussion threads on our
> > migration
> > > > proposals for Apache Ignite extensions:
> > > >
> > > >
> > > >
> > >
> >
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> > > >
> > > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > > >
> > > > I will share update as I hear more information.
> > > >
> > > > Regards,
> > > > Saikat
> > > >
> > > >
> > > >
> > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra <
> saikat.maitra@gmail.com
> > >
> > > > wrote:
> > > >
> > > >> Hello Denis,
> > > >>
> > > >> Sure, sounds good. I will create a separate discussion thread to get
> > > >> community feedback on whether we
> > > >> should join the Bahir or kick-off "Ignite Extensions" project.
> > > >>
> > > >> Regards,
> > > >> Saikat
> > > >>
> > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda <dm...@apache.org>
> > wrote:
> > > >>
> > > >>> Folks,
> > > >>>
> > > >>> The concept of Apache Bahir is precisely what we need! Saikat,
> thanks
> > > for
> > > >>> doing the research. Why I believe the * idea* fits us well:
> > > >>>
> > > >>>    - All integrations can be stored in separate Github repositories
> > and
> > > >>>    have their dev lifecycles. We've not obliged to couple all the
> > > >>> integrations
> > > >>>    in a single repo. For instance, Spark can be located in a
> > dedicated
> > > >>> repo
> > > >>>    while streaming integrations might be in a single one. It's up
> to
> > > us.
> > > >>>    - All the repositories and integrations belong to ASF. We're not
> > > >>> dumping
> > > >>>    them on Github but rather keep supporting and developing in
> > > >>> accordance with
> > > >>>    ASF vision and practices.
> > > >>>    - There is a way to reward contributors via committership and
> PMC
> > > >>>    membership. You won't get this if a project is just one of the
> > > >>> millions of
> > > >>>    Github projects.
> > > >>>
> > > >>> Saikat, as Ignite PMC member, could you please kick-off a separate
> > > >>> dev-list
> > > >>> discussion to involve more community members and referring to this
> > > >>> thread?
> > > >>> I think the primary question the community needs to answer whether
> we
> > > >>> should join the Bahir or kick-off "Ignite Extensions" project?
> > > >>>
> > > >>> -
> > > >>> Denis
> > > >>>
> > > >>>
> > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev <
> > > zaleslaw.sin@gmail.com>
> > > >>> wrote:
> > > >>>
> > > >>> > Maybe we could move all our Streaming Integrations there, but
> what
> > is
> > > >>> about
> > > >>> > maintaining and committer permissions to the new repositories?
> > > >>> > I see the list of the committers and PMC members there
> > > >>> > https://bahir.apache.org/community-members/
> > > >>> >
> > > >>> > Could somebody from Ignite community be added to this list as a
> > > >>> > PMC/committer to maintain Ignite integrations?
> > > >>> > If yes, I'd happy to join this project and collaborate with these
> > > guys
> > > >>> > together
> > > >>> > If no, and we should start from the zero level with external PRs
> -
> > > >>> hmmm,
> > > >>> > it's better to have our own external repository with ApacheBahirr
> > > >>> ideology.
> > > >>> >
> > > >>> > Also, I agree, that all connectors could be moved there (in
> > > >>> ApacheBahirr
> > > >>> > like repository), but not all modules from the page
> > > >>> >
> > > >>> >
> > > >>>
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > >>> > because
> > > >>> > they use different parts of core modules and could be developed
> > with
> > > >>> > different velocity.
> > > >>> >
> > > >>> > In reality, before creation of new repositories we need wide
> > > discussion
> > > >>> > based on the document mentioned above.
> > > >>> >
> > > >>> > P.S. Of course, we could experiment in the next release 2.8 with
> > one
> > > >>> or two
> > > >>> > integrations like twitter or ZeroMQ
> > > >>> > Also, I'd like Denis Magda idea for the separate repositories in
> > > Apache
> > > >>> > Github to maintain them by anybody.
> > > >>> >
> > > >>> >
> > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra <
> > saikat.maitra@gmail.com
> > > >:
> > > >>> >
> > > >>> > > Hi Denis, Emmanouil
> > > >>> > >
> > > >>> > > Thank you for your email. I think creating a separate
> integration
> > > >>> project
> > > >>> > > in github and also proposing it as an apache incubator project
> > will
> > > >>> be a
> > > >>> > > good move. The other separate integration modules
> > > >>> > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > >>> > > can
> > > >>> > > be moved to this new apache incubator project.
> > > >>> > >
> > > >>> > > There are similar integration available for Flink and Spark in
> > > Apache
> > > >>> > Bahir
> > > >>> > > https://bahir.apache.org/
> > > >>> > >
> > > >>> > > Do you think we should reach out to Apache Bahir community
> with a
> > > >>> > proposal
> > > >>> > > or we should plan to create a separate Apache Incubator
> project?
> > > >>> > >
> > > >>> > > Regards,
> > > >>> > > Saikat
> > > >>> > >
> > > >>> > >
> > > >>> > >
> > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil Gkatziouras <
> > > >>> > > gkatzioura@gmail.com>
> > > >>> > > wrote:
> > > >>> > >
> > > >>> > > > Hi all!
> > > >>> > > >
> > > >>> > > > Based on the discussions most probably this is going to be on
> > > >>> another
> > > >>> > > > GitHub repo. Therefore I will prepare a standalone project
> with
> > > the
> > > >>> > > Pub/Sub
> > > >>> > > > module and once the repository is created a pull request
> shall
> > be
> > > >>> > issued.
> > > >>> > > > If there is anything else I can do in order to facilitate
> this
> > > >>> please
> > > >>> > let
> > > >>> > > > me know.
> > > >>> > > >
> > > >>> > > > If a ticket is created for this issue, I shall be able to
> > create
> > > a
> > > >>> pull
> > > >>> > > > request and thus a build will take place on ignite's CI.
> > > >>> > > > If everyone is aligned with adding a Pub/Sub implementation
> > may I
> > > >>> > create
> > > >>> > > a
> > > >>> > > > ticket on JIRA?
> > > >>> > > >
> > > >>> > > > My next question has to do on how Ignite configures the build
> > > jobs
> > > >>> in
> > > >>> > > Team
> > > >>> > > > City. The unit tests for my implementation need a local
> Pub/Sub
> > > >>> server
> > > >>> > up
> > > >>> > > > and running (they provide one for testing purposes). Will
> this
> > > take
> > > >>> > > effect
> > > >>> > > > in a form of a build script inside my implementation or is it
> > > >>> something
> > > >>> > > > that should be configured on Team City?
> > > >>> > > >
> > > >>> > > > Kind regards,
> > > >>> > > > Emmanouil
> > > >>> > > >
> > > >>> > > >
> > > >>> > > > *Emmanouil Gkatziouras*
> > > >>> > > > https://egkatzioura.com/ |
> > > >>> > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >>> > > > https://github.com/gkatzioura
> > > >>> > > >
> > > >>> > > >
> > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis Magda <dmagda@apache.org
> >
> > > >>> wrote:
> > > >>> > > >
> > > >>> > > > > Emmanouil, Saikat,
> > > >>> > > > >
> > > >>> > > > > After contemplating on this, let me propose another way
> with
> > > this
> > > >>> > > > > particular integration that is aligned with our
> > modularization
> > > >>> > > endeavor.
> > > >>> > > > > The modularization [1] defines the Ignite core, Ignite
> > modules,
> > > >>> and
> > > >>> > > > > independent integrations. The first two are to be located
> in
> > > >>> Ignite
> > > >>> > > > > repositories and to be tested/updated by the community for
> > > every
> > > >>> > > release.
> > > >>> > > > > The modules are a subset of existing integrations that are
> > > >>> adopted
> > > >>> > > widely
> > > >>> > > > > and require community attention.
> > > >>> > > > >
> > > >>> > > > > As for the independent integrations [1], those are supposed
> > to
> > > be
> > > >>> > > located
> > > >>> > > > > in a separate GitHub repo. I would advise us to start
> > > >>> implementing
> > > >>> > this
> > > >>> > > > > concept with the Pub/Sub integration. Even though it will
> be
> > > >>> located
> > > >>> > > in a
> > > >>> > > > > separate Github repo, it will be listed among officially
> > > >>> available
> > > >>> > > > > integrations on the Ignite website featuring Emmanouil as a
> > > >>> > > contributor.
> > > >>> > > > >
> > > >>> > > > > What do you think?
> > > >>> > > > >
> > > >>> > > > >
> > > >>> > > > > [1]
> > > >>> > > > >
> > > >>> > > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > >>> > > > >
> > > >>> > > > >
> > > >>> > > > >
> > > >>> > > > > -
> > > >>> > > > > Denis
> > > >>> > > > >
> > > >>> > > > >
> > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM Saikat Maitra <
> > > >>> > saikat.maitra@gmail.com
> > > >>> > > >
> > > >>> > > > > wrote:
> > > >>> > > > >
> > > >>> > > > > > Hi Emmanouil,
> > > >>> > > > > >
> > > >>> > > > > > The changes looks good to me. I wanted to check if you
> are
> > > also
> > > >>> > able
> > > >>> > > to
> > > >>> > > > > > validate that the once data is added to Ignite cluster
> you
> > > are
> > > >>> also
> > > >>> > > > able
> > > >>> > > > > to
> > > >>> > > > > > access it using another Ignite client or using rest
> > > endpoints.
> > > >>> > > > > >
> > > >>> > > > > > We use Teamcity for CI process, the details are as
> follows
> > > >>> > > > > >
> > > >>> > > > > > Trigger validation of those test suites that have been
> > > >>> affected by
> > > >>> > > your
> > > >>> > > > > > changes on TeamCity <http://ci.ignite.apache.org/>: <
> > > >>> > > > > http://204.14.53.153/
> > > >>> > > > > > >
> > > >>> > > > > >
> > > >>> > > > > >    - Locate a test suite you have to check, press button
> > > named
> > > >>> > "..."
> > > >>> > > > that
> > > >>> > > > > >    is located on the left of "Run" button. "Run custom
> > build"
> > > >>> > dialog
> > > >>> > > > will
> > > >>> > > > > >    appear;
> > > >>> > > > > >    - Go to "Changes" tab and choose
> > > >>> > "pull/<pull-request-number>/head"
> > > >>> > > > in
> > > >>> > > > > >    "Build branch" dropdown list;
> > > >>> > > > > >    - Press "Run build" button and monitor tests results.
> > > >>> > > > > >
> > > >>> > > > > >
> > > >>> > > > > >
> > > >>> > > > > > @Roman rshtykh@yahoo.com - can you also please take a
> > look?
> > > >>> > > > > >
> > > >>> > > > > > Regards,
> > > >>> > > > > > Saikat
> > > >>> > > > > >
> > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20 PM Emmanouil Gkatziouras <
> > > >>> > > > > > gkatzioura@gmail.com>
> > > >>> > > > > > wrote:
> > > >>> > > > > >
> > > >>> > > > > > > Hi @Saikat Maitra
> > > >>> > > > > > >
> > > >>> > > > > > > I have my implementation on an Ignite fork on my
> personal
> > > >>> GitHub
> > > >>> > > > > account
> > > >>> > > > > > > [1]
> > > >>> > > > > > > It is based on the Kafka implementation which is of a
> > > polling
> > > >>> > > nature.
> > > >>> > > > > > > GCP does not provide a service like Kafka. Instead it
> > > >>> provides us
> > > >>> > > > with
> > > >>> > > > > > > Pub/Sub which is just like Kafka.
> > > >>> > > > > > > Therefore using the Kafka streamer as a guideline made
> > more
> > > >>> sense
> > > >>> > > to
> > > >>> > > > > me.
> > > >>> > > > > > >
> > > >>> > > > > > > Apart from the implementation I have also added a unit
> > > test.
> > > >>> > > > > > > It definitely needs some polishing and a double check
> on
> > my
> > > >>> side
> > > >>> > to
> > > >>> > > > > make
> > > >>> > > > > > > the review smooth.
> > > >>> > > > > > > For the tests a local Pub/Sub [2] server has to be
> setup
> > > >>> however
> > > >>> > I
> > > >>> > > am
> > > >>> > > > > not
> > > >>> > > > > > > sure which is the best way to do so with regards to
> your
> > > >>> CI/CD.
> > > >>> > > > > > >
> > > >>> > > > > > > [1]
> > > >>> > >
> https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > > >>> > > > > > > [2]
> > > >>> https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
> > > >>> > > > > > >
> > > >>> > > > > > > *Emmanouil Gkatziouras*
> > > >>> > > > > > > https://egkatzioura.com/ |
> > > >>> > > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >>> > > > > > > https://github.com/gkatzioura
> > > >>> > > > > > >
> > > >>> > > > > > >
> > > >>> > > > > > > On Sat, 12 Oct 2019 at 02:11, Saikat Maitra <
> > > >>> > > saikat.maitra@gmail.com
> > > >>> > > > >
> > > >>> > > > > > > wrote:
> > > >>> > > > > > >
> > > >>> > > > > > > > Hello Emmanouil,
> > > >>> > > > > > > >
> > > >>> > > > > > > > Can you please share more info about the
> > implementation?
> > > >>> > > > > > > >
> > > >>> > > > > > > > We have similar implementation for data streamers
> with
> > > >>> Kafka
> > > >>> > and
> > > >>> > > > > > RocketMQ
> > > >>> > > > > > > >
> > > >>> > > > > > > >
> https://apacheignite-mix.readme.io/docs/kafka-streamer
> > > >>> > > > > > > >
> > > >>> > > > > > > >
> > > https://apacheignite-mix.readme.io/docs/rocketmq-streamer
> > > >>> > > > > > > >
> > > >>> > > > > > > > Please review and share if you have any questions.
> > > >>> > > > > > > >
> > > >>> > > > > > > > Regards,
> > > >>> > > > > > > > Saikat
> > > >>> > > > > > > >
> > > >>> > > > > > > >
> > > >>> > > > > > > >
> > > >>> > > > > > > > On Fri, Oct 11, 2019 at 4:56 PM Emmanouil
> Gkatziouras <
> > > >>> > > > > > > > gkatzioura@gmail.com>
> > > >>> > > > > > > > wrote:
> > > >>> > > > > > > >
> > > >>> > > > > > > > > Hi all,
> > > >>> > > > > > > > >
> > > >>> > > > > > > > > By mentioning Pub/Sub I mean Google Cloud's Pub/Sub
> > [1]
> > > >>> > > > > > > > > Yes it is indeed cloud specific. It is the
> messaging
> > > >>> service
> > > >>> > > > > provided
> > > >>> > > > > > > by
> > > >>> > > > > > > > > GCP.
> > > >>> > > > > > > > > I have made the integration with it, but I haven't
> > > >>> published
> > > >>> > it
> > > >>> > > > yet
> > > >>> > > > > > (I
> > > >>> > > > > > > am
> > > >>> > > > > > > > > waiting for the discussion/ticket and proceed with
> > the
> > > >>> > > > > contribution).
> > > >>> > > > > > > > >
> > > >>> > > > > > > > > Kind regards,
> > > >>> > > > > > > > > Emmanouil
> > > >>> > > > > > > > >
> > > >>> > > > > > > > > [1] https://cloud.google.com/pubsub/docs/overview
> > > >>> > > > > > > > >
> > > >>> > > > > > > > > *Emmanouil Gkatziouras*
> > > >>> > > > > > > > > https://egkatzioura.com/ |
> > > >>> > > > > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >>> > > > > > > > > https://github.com/gkatzioura
> > > >>> > > > > > > > >
> > > >>> > > > > > > > >
> > > >>> > > > > > > > > On Fri, 11 Oct 2019 at 21:45, Denis Magda <
> > > >>> dmagda@apache.org
> > > >>> > >
> > > >>> > > > > wrote:
> > > >>> > > > > > > > >
> > > >>> > > > > > > > > > Hello Emmanouil,
> > > >>> > > > > > > > > >
> > > >>> > > > > > > > > > Are you referring to any specific implementation
> > > >>> tailored
> > > >>> > > for a
> > > >>> > > > > > > cloud?
> > > >>> > > > > > > > If
> > > >>> > > > > > > > > > I'm not mistaken you were mentioning AWS/GCE. Is
> > > there
> > > >>> any
> > > >>> > > > Github
> > > >>> > > > > > > > source
> > > >>> > > > > > > > > > code?
> > > >>> > > > > > > > > >
> > > >>> > > > > > > > > > Here is a list of existing integrations with
> > > streaming
> > > >>> and
> > > >>> > > > > pub/sub
> > > >>> > > > > > > > > > frameworks:
> > > >>> > > > > > > > > > https://apacheignite-mix.readme.io/docs/overview
> > > >>> > > > > > > > > >
> > > >>> > > > > > > > > > with the source code in Ignite master:
> > > >>> > > > > > > > > >
> > https://github.com/apache/ignite/tree/master/modules
> > > >>> > > > > > > > > >
> > > >>> > > > > > > > > > Check out those resources to see what are the
> most
> > > >>> suitable
> > > >>> > > > > Ignite
> > > >>> > > > > > > APIs
> > > >>> > > > > > > > > for
> > > >>> > > > > > > > > > such integrations.
> > > >>> > > > > > > > > >
> > > >>> > > > > > > > > > @Saikat Maitra <sa...@gmail.com> ,
> @Roman
> > <
> > > >>> > > > > > rshtykh@yahoo.com
> > > >>> > > > > > > >
> > > >>> > > > > > > > ,
> > > >>> > > > > > > > > as
> > > >>> > > > > > > > > > streaming technologies contributors/maintainers,
> > > could
> > > >>> you
> > > >>> > > > please
> > > >>> > > > > > > join
> > > >>> > > > > > > > > this
> > > >>> > > > > > > > > > thread?
> > > >>> > > > > > > > > >
> > > >>> > > > > > > > > > -
> > > >>> > > > > > > > > > Denis
> > > >>> > > > > > > > > >
> > > >>> > > > > > > > > >
> > > >>> > > > > > > > > > On Thu, Oct 10, 2019 at 7:58 AM Emmanouil
> > > Gkatziouras <
> > > >>> > > > > > > > > > gkatzioura@gmail.com>
> > > >>> > > > > > > > > > wrote:
> > > >>> > > > > > > > > >
> > > >>> > > > > > > > > > > Hello everyone,
> > > >>> > > > > > > > > > >
> > > >>> > > > > > > > > > > I started using Ignite lately. Part of my work
> > > >>> involves
> > > >>> > > > > Pub/Sub,
> > > >>> > > > > > > > thus I
> > > >>> > > > > > > > > > > created a pub/sub streamer.
> > > >>> > > > > > > > > > > Pub/Sub is pretty close to Kafka.
> > > >>> > > > > > > > > > > Here's how the streamer works. Every node
> creates
> > > an
> > > >>> > > instance
> > > >>> > > > > of
> > > >>> > > > > > a
> > > >>> > > > > > > > > > Pub/Sub
> > > >>> > > > > > > > > > > client.
> > > >>> > > > > > > > > > > The clients subscribe to the same topic using
> the
> > > >>> same
> > > >>> > > > > > subscription
> > > >>> > > > > > > > > name.
> > > >>> > > > > > > > > > > A message is being sent and one of the nodes
> will
> > > >>> receive
> > > >>> > > the
> > > >>> > > > > > > > message,
> > > >>> > > > > > > > > > > process it and then forward it to Ignite using
> > the
> > > >>> > > extractor
> > > >>> > > > > > > > provided.
> > > >>> > > > > > > > > > >
> > > >>> > > > > > > > > > > Any thoughts?
> > > >>> > > > > > > > > > >
> > > >>> > > > > > > > > > > *Emmanouil Gkatziouras*
> > > >>> > > > > > > > > > > https://egkatzioura.com/ |
> > > >>> > > > > > > > > > >
> > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >>> > > > > > > > > > > https://github.com/gkatzioura
> > > >>> > > > > > > > > > >
> > > >>> > > > > > > > > >
> > > >>> > > > > > > > >
> > > >>> > > > > > > >
> > > >>> > > > > > >
> > > >>> > > > > >
> > > >>> > > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > > >>
> > >
> >
>

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra <sa...@gmail.com>]

Hi Ivan,

Yes, we will need to remove flink module from main ignite repo.

I can do it as part of this issue.

IGNITE-12356 Migrate Flink module to ignite-extensions

Regards,
Saikat

On Mon, Dec 30, 2019 at 12:54 AM Ivan Pavlukhin <vo...@gmail.com> wrote:

> Folks,
>
> Do not we need to remove flink integration sources from main ignite
> repository [1]? Is this work in progress?
>
> [1] https://github.com/apache/ignite
>
> вс, 29 дек. 2019 г. в 22:19, Emmanouil Gkatziouras <gk...@gmail.com>:
> >
> > Hi Saikat!
> >
> > Thank you for your time and assistance.
> >
> > Kind regards,
> > Emmanouil
> >
> > *Emmanouil Gkatziouras*
> > https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> > https://github.com/gkatzioura
> >
> >
> > On Sun, 29 Dec 2019 at 18:35, Saikat Maitra <sa...@gmail.com>
> wrote:
> >
> > > Hi Emmanouil,
> > >
> > > Thank you for your contribution. I have merged your changes in master.
> > >
> > > Regards,
> > > Saikat
> > >
> > >
> > > On Sun, Dec 29, 2019 at 12:20 PM Saikat Maitra <
> saikat.maitra@gmail.com>
> > > wrote:
> > >
> > > > Thank you, Emmanouil
> > > >
> > > > I am reviewing the changes.
> > > >
> > > > Regards,
> > > > Saikat
> > > >
> > > > On Sat, Dec 28, 2019 at 3:48 PM Emmanouil Gkatziouras <
> > > > gkatzioura@gmail.com> wrote:
> > > >
> > > >> Hi Saikat!
> > > >>
> > > >> The dependencies have been updated, and the merge conflicts
> resolved.
> > > >> Kind regards
> > > >>
> > > >> On Sat, 28 Dec 2019, 23:27 Saikat Maitra, <sa...@gmail.com>
> > > >> wrote:
> > > >>
> > > >> > Hi Emmanouil,
> > > >> >
> > > >> > Thank you for making the changes. I have merged my PR changes in
> > > master.
> > > >> > Can you please take the latest from master branch and resolve the
> > > >> conflicts
> > > >> > in your PR.
> > > >> >
> > > >> > I have also reviewed the changes in your PR and requested minor
> > > changes
> > > >> > specific to dependencies version and license information.
> > > >> >
> > > >> > Can you please review and share your feedback?
> > > >> >
> > > >> > Regards,
> > > >> > Saikat
> > > >> >
> > > >> >
> > > >> >
> > > >> >
> > > >> > On Sat, Dec 28, 2019 at 4:38 AM Emmanouil Gkatziouras <
> > > >> > gkatzioura@gmail.com>
> > > >> > wrote:
> > > >> >
> > > >> > > Hi Saikat!
> > > >> > >
> > > >> > > Thank you on this one! I tried it and it worked as expected [1]
> > > >> > >
> > > >> > > Kind regards
> > > >> > > Emmanouil
> > > >> > >
> > > >> > > [1]
> > > >> > >
> > > >> > >
> > > >> >
> > > >>
> > >
> https://ci.ignite.apache.org/viewLog.html?buildId=4888357&tab=buildLog&_focus=2046#_state=2046
> > > >> > >
> > > >> > > *Emmanouil Gkatziouras*
> > > >> > > https://egkatzioura.com/ |
> > > >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> > > https://github.com/gkatzioura
> > > >> > >
> > > >> > >
> > > >> > > On Thu, 26 Dec 2019 at 19:50, Saikat Maitra <
> > > saikat.maitra@gmail.com>
> > > >> > > wrote:
> > > >> > >
> > > >> > > > Hi Emmanouil,
> > > >> > > >
> > > >> > > > I have been able to resolve the build failures for missing
> > > modules.
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> https://ci.ignite.apache.org/viewLog.html?buildId=4883394&buildTypeId=IgniteExtensions_Build
> > > >> > > >
> > > >> > > > Will you please take the latest changes from my PR and run the
> > > build
> > > >> > for
> > > >> > > > your PR.
> > > >> > > >
> > > >> > > > https://github.com/apache/ignite-extensions/pull/1/files
> > > >> > > >
> > > >> > > > The new build configurations should let your module build also
> > > pass.
> > > >> > > >
> > > >> > > > The issue that was causing failure was that I had put
> ignite.zip
> > > as
> > > >> > > > artifact dependency and it was downloading the artifacts
> including
> > > >> pom
> > > >> > > file
> > > >> > > > inside the build checkout folder of ignite-extensions and as a
> > > >> result
> > > >> > the
> > > >> > > > pom.xml of ignite-extensions was getting overwritten by
> ignite's
> > > >> > pom.xml.
> > > >> > > > This was resulting in missing flink-ext module error as it
> was not
> > > >> even
> > > >> > > > present in the ignite main pom.xml.
> > > >> > > >
> > > >> > > > To resolve the issue I just had to download and extract
> artifact
> > > of
> > > >> > > > ignite.zip in a separate directory.
> > > >> > > >
> > > >> > > > Regards,
> > > >> > > > Saikat
> > > >> > > >
> > > >> > > >
> > > >> > > > On Sat, Nov 30, 2019 at 5:53 PM Emmanouil Gkatziouras <
> > > >> > > > gkatzioura@gmail.com>
> > > >> > > > wrote:
> > > >> > > >
> > > >> > > > > Hi Saikat!
> > > >> > > > >
> > > >> > > > > Thank you for your assistance on that!
> > > >> > > > >
> > > >> > > > > Kind regards
> > > >> > > > >
> > > >> > > > > *Emmanouil Gkatziouras*
> > > >> > > > > https://egkatzioura.com/ |
> > > >> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> > > > > https://github.com/gkatzioura
> > > >> > > > >
> > > >> > > > >
> > > >> > > > > On Sat, 30 Nov 2019 at 23:51, Saikat Maitra <
> > > >> saikat.maitra@gmail.com
> > > >> > >
> > > >> > > > > wrote:
> > > >> > > > >
> > > >> > > > > > Hi Emmanouil,
> > > >> > > > > >
> > > >> > > > > > I looked into the build logs and I observed that since I
> added
> > > >> > > > artifacts
> > > >> > > > > > dependencies to "Build apache Ignite"[1] it was able to
> pull
> > > the
> > > >> > > > required
> > > >> > > > > > dependencies and was able to run the existing tests. It is
> > > >> however
> > > >> > > not
> > > >> > > > > > identifying new modules like I changed from flink to
> flink-ext
> > > >> or
> > > >> > > > > pub-sub.
> > > >> > > > > >
> > > >> > > > > > [1]
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> https://ci.ignite.apache.org/admin/editDependencies.html?id=buildType:IgniteExtensions_Build
> > > >> > > > > >
> > > >> > > > > > I will look into the issue and debug further.
> > > >> > > > > >
> > > >> > > > > > Regards,
> > > >> > > > > > Saikat
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > > On Sat, Nov 30, 2019 at 5:35 PM Emmanouil Gkatziouras <
> > > >> > > > > > gkatzioura@gmail.com>
> > > >> > > > > > wrote:
> > > >> > > > > >
> > > >> > > > > > > Hi Saikat!
> > > >> > > > > > >
> > > >> > > > > > > From the logs[1] it seems that TC cannot find the
> project.
> > > >> > However
> > > >> > > I
> > > >> > > > > did
> > > >> > > > > > > include the project on the parent pom (modules
> sections) [2]
> > > >> > > > > > > I tried to reproduce locally unfortunately I am not
> aware of
> > > >> the
> > > >> > > full
> > > >> > > > > > > context on TC. It seems as if a different parent pom is
> > > being
> > > >> > > picked
> > > >> > > > > up.
> > > >> > > > > > >
> > > >> > > > > > > Kind regards
> > > >> > > > > > >
> > > >> > > > > > > [1]
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> https://ci.ignite.apache.org/viewLog.html?buildId=4804893&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=1704
> > > >> > > > > > > [2]
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> https://github.com/gkatzioura/ignite-extensions/blob/d6a8beee7feff28f59d9a12be692016305564d25/pom.xml#L46
> > > >> > > > > > >
> > > >> > > > > > > *Emmanouil Gkatziouras*
> > > >> > > > > > > https://egkatzioura.com/ |
> > > >> > > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> > > > > > > https://github.com/gkatzioura
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > > > On Sat, 30 Nov 2019 at 22:20, Saikat Maitra <
> > > >> > > saikat.maitra@gmail.com
> > > >> > > > >
> > > >> > > > > > > wrote:
> > > >> > > > > > >
> > > >> > > > > > > > Hello,
> > > >> > > > > > > >
> > > >> > > > > > > > I have made changes in Ignite Extensions build
> > > >> configurations.
> > > >> > > > > > > >
> > > >> > > > > > > > It is now running the tests as expected.
> > > >> > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> https://ci.ignite.apache.org/viewLog.html?buildId=4804480&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> > > >> > > > > > > >
> > > >> > > > > > > > Regards,
> > > >> > > > > > > > Saikat
> > > >> > > > > > > >
> > > >> > > > > > > > On Thu, Nov 28, 2019 at 1:29 PM Saikat Maitra <
> > > >> > > > > saikat.maitra@gmail.com
> > > >> > > > > > >
> > > >> > > > > > > > wrote:
> > > >> > > > > > > >
> > > >> > > > > > > > > Hi,
> > > >> > > > > > > > >
> > > >> > > > > > > > > We do run build with -DskipTests and then configure
> > > >> selective
> > > >> > > > test
> > > >> > > > > > > suites
> > > >> > > > > > > > > to run.
> > > >> > > > > > > > >
> > > >> > > > > > > > > I will check why these test suites are not getting
> > > >> executed.
> > > >> > > > > > > > >
> > > >> > > > > > > > > Regards
> > > >> > > > > > > > > Saikat
> > > >> > > > > > > > >
> > > >> > > > > > > > > On Wed, 27 Nov 2019 at 3:05 PM, Emmanouil
> Gkatziouras <
> > > >> > > > > > > > > gkatzioura@gmail.com> wrote:
> > > >> > > > > > > > >
> > > >> > > > > > > > >> Hi all,
> > > >> > > > > > > > >>
> > > >> > > > > > > > >> I did add a test suite and run the build with the
> > > >> > > corresponding
> > > >> > > > > > > > arguments,
> > > >> > > > > > > > >> however the tests did not run [1].
> > > >> > > > > > > > >> I checked the Flink's build logs and the same
> message
> > > is
> > > >> > > > displayed
> > > >> > > > > > > there
> > > >> > > > > > > > >> too `No tests to run` [2].
> > > >> > > > > > > > >>
> > > >> > > > > > > > >> [1]
> > > >> > > > > > > > >>
> > > >> > > > > > > > >>
> > > >> > > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> https://ci.ignite.apache.org/viewLog.html?buildId=4798203&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=248
> > > >> > > > > > > > >> [2]
> > > >> > > > > > > > >>
> > > >> > > > > > > > >>
> > > >> > > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4789199&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=1256
> > > >> > > > > > > > >>
> > > >> > > > > > > > >> Kind regards
> > > >> > > > > > > > >> *Emmanouil Gkatziouras*
> > > >> > > > > > > > >> https://egkatzioura.com/ |
> > > >> > > > > > > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> > > > > > > > >> https://github.com/gkatzioura
> > > >> > > > > > > > >>
> > > >> > > > > > > > >>
> > > >> > > > > > > > >> On Wed, 27 Nov 2019 at 18:58, Saikat Maitra <
> > > >> > > > > > saikat.maitra@gmail.com>
> > > >> > > > > > > > >> wrote:
> > > >> > > > > > > > >>
> > > >> > > > > > > > >> > Hi,
> > > >> > > > > > > > >> >
> > > >> > > > > > > > >> > We will need to add test suites for tests to be
> > > >> executed
> > > >> > in
> > > >> > > > > build.
> > > >> > > > > > > We
> > > >> > > > > > > > >> pass
> > > >> > > > > > > > >> > them as TestSuites param. I had added Flink sink
> and
> > > >> > source
> > > >> > > > > > > > testsuites.
> > > >> > > > > > > > >> >
> > > >> > > > > > > > >> > Can you please review and confirm.
> > > >> > > > > > > > >> >
> > > >> > > > > > > > >> > Regards
> > > >> > > > > > > > >> > Saikat
> > > >> > > > > > > > >> >
> > > >> > > > > > > > >> > On Wed, 27 Nov 2019 at 6:00 AM, Emmanouil
> > > Gkatziouras <
> > > >> > > > > > > > >> > gkatzioura@gmail.com>
> > > >> > > > > > > > >> > wrote:
> > > >> > > > > > > > >> >
> > > >> > > > > > > > >> > > Hi all!
> > > >> > > > > > > > >> > >
> > > >> > > > > > > > >> > > I was successful building and running my pull
> > > >> requests.
> > > >> > It
> > > >> > > > > seems
> > > >> > > > > > > > that
> > > >> > > > > > > > >> the
> > > >> > > > > > > > >> > > tests do not run both for Flink and Pub/Sub [1]
> > > >> > > > > > > > >> > > If there is something I cannot do to make them
> > > >> > > discoverable
> > > >> > > > > > please
> > > >> > > > > > > > >> let me
> > > >> > > > > > > > >> > > know.
> > > >> > > > > > > > >> > >
> > > >> > > > > > > > >> > > [1]
> > > >> > > > > > > > >> > >
> > > >> > > > > > > > >> > >
> > > >> > > > > > > > >> >
> > > >> > > > > > > > >>
> > > >> > > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4796722&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=283
> > > >> > > > > > > > >> > >
> > > >> > > > > > > > >> > > *Emmanouil Gkatziouras*
> > > >> > > > > > > > >> > > https://egkatzioura.com/ |
> > > >> > > > > > > > >> > >
> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> > > > > > > > >> > > https://github.com/gkatzioura
> > > >> > > > > > > > >> > >
> > > >> > > > > > > > >> > >
> > > >> > > > > > > > >> > > On Wed, 27 Nov 2019 at 04:57, Saikat Maitra <
> > > >> > > > > > > > saikat.maitra@gmail.com>
> > > >> > > > > > > > >> > > wrote:
> > > >> > > > > > > > >> > >
> > > >> > > > > > > > >> > > > Hi Emmanouil,
> > > >> > > > > > > > >> > > >
> > > >> > > > > > > > >> > > > I have added you as contributor in Ignite
> > > >> Extensions
> > > >> > > > > project.
> > > >> > > > > > > Can
> > > >> > > > > > > > >> you
> > > >> > > > > > > > >> > > > please check and confirm if you are able to
> see
> > > the
> > > >> > > > project
> > > >> > > > > > and
> > > >> > > > > > > > >> execute
> > > >> > > > > > > > >> > > > build on your pull request.
> > > >> > > > > > > > >> > > >
> > > >> > > > > > > > >> > > > Regards,
> > > >> > > > > > > > >> > > > Saikat
> > > >> > > > > > > > >> > > >
> > > >> > > > > > > > >> > > > On Tue, Nov 26, 2019 at 12:49 PM Emmanouil
> > > >> > Gkatziouras <
> > > >> > > > > > > > >> > > > gkatzioura@gmail.com>
> > > >> > > > > > > > >> > > > wrote:
> > > >> > > > > > > > >> > > >
> > > >> > > > > > > > >> > > > > Hi all!
> > > >> > > > > > > > >> > > > >
> > > >> > > > > > > > >> > > > > If someone could give me read access to
> the Job
> > > >> of
> > > >> > > > Ignite
> > > >> > > > > > > > >> Extensions
> > > >> > > > > > > > >> > on
> > > >> > > > > > > > >> > > > > Team City will greatly help me (username
> > > >> > gkatzioura).
> > > >> > > > > > > > >> > > > > I suppose the builds get triggered
> > > automatically
> > > >> on
> > > >> > a
> > > >> > > > pull
> > > >> > > > > > > > >> request.
> > > >> > > > > > > > >> > > > >
> > > >> > > > > > > > >> > > > > Kind regards,
> > > >> > > > > > > > >> > > > > Emmanouil
> > > >> > > > > > > > >> > > > >
> > > >> > > > > > > > >> > > > > *Emmanouil Gkatziouras*
> > > >> > > > > > > > >> > > > > https://egkatzioura.com/ |
> > > >> > > > > > > > >> > > > >
> > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> > > > > > > > >> > > > > https://github.com/gkatzioura
> > > >> > > > > > > > >> > > > >
> > > >> > > > > > > > >> > > > >
> > > >> > > > > > > > >> > > > > On Sun, 24 Nov 2019 at 21:11, Emmanouil
> > > >> Gkatziouras
> > > >> > <
> > > >> > > > > > > > >> > > > gkatzioura@gmail.com>
> > > >> > > > > > > > >> > > > > wrote:
> > > >> > > > > > > > >> > > > >
> > > >> > > > > > > > >> > > > > > Hi Saikat!
> > > >> > > > > > > > >> > > > > >
> > > >> > > > > > > > >> > > > > > I just rebased with the flink branch.
> > > >> > Unfortunately
> > > >> > > I
> > > >> > > > do
> > > >> > > > > > not
> > > >> > > > > > > > >> have
> > > >> > > > > > > > >> > > read
> > > >> > > > > > > > >> > > > > > access to the team city link you
> provided and
> > > >> thus
> > > >> > > > > > evaluate
> > > >> > > > > > > > the
> > > >> > > > > > > > >> > tests
> > > >> > > > > > > > >> > > > for
> > > >> > > > > > > > >> > > > > > my pull request.
> > > >> > > > > > > > >> > > > > > I guess it has to do with ignite
> extensions
> > > >> being
> > > >> > > new.
> > > >> > > > > > > > >> > > > > > My username is gkatzioura and email the
> one I
> > > >> am
> > > >> > > using
> > > >> > > > > on
> > > >> > > > > > > this
> > > >> > > > > > > > >> > email
> > > >> > > > > > > > >> > > > > >
> > > >> > > > > > > > >> > > > > > Thank you for your time.
> > > >> > > > > > > > >> > > > > >
> > > >> > > > > > > > >> > > > > >
> > > >> > > > > > > > >> > > > > > *Emmanouil Gkatziouras*
> > > >> > > > > > > > >> > > > > > https://egkatzioura.com/ |
> > > >> > > > > > > > >> > > > > >
> > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> > > > > > > > >> > > > > > https://github.com/gkatzioura
> > > >> > > > > > > > >> > > > > >
> > > >> > > > > > > > >> > > > > >
> > > >> > > > > > > > >> > > > > > On Sun, 24 Nov 2019 at 18:03, Saikat
> Maitra <
> > > >> > > > > > > > >> > saikat.maitra@gmail.com
> > > >> > > > > > > > >> > > >
> > > >> > > > > > > > >> > > > > > wrote:
> > > >> > > > > > > > >> > > > > >
> > > >> > > > > > > > >> > > > > >> Hi Emmanouil,
> > > >> > > > > > > > >> > > > > >>
> > > >> > > > > > > > >> > > > > >> The latest build on teamcity has passed
> on
> > > >> Flink
> > > >> > > pull
> > > >> > > > > > > > request.
> > > >> > > > > > > > >> > > > > >>
> > > >> > > > > > > > >> > > > > >>
> > > >> > > > > > > > >> > > > > >>
> > > >> > > > > > > > >> > > > >
> > > >> > > > > > > > >> > > >
> > > >> > > > > > > > >> > >
> > > >> > > > > > > > >> >
> > > >> > > > > > > > >>
> > > >> > > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> https://ci.ignite.apache.org/viewLog.html?buildId=4788928&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> > > >> > > > > > > > >> > > > > >>
> > > >> > > > > > > > >> > > > > >> you should be able to take the changes I
> > > made
> > > >> in
> > > >> > my
> > > >> > > > PR
> > > >> > > > > > and
> > > >> > > > > > > > run
> > > >> > > > > > > > >> > build
> > > >> > > > > > > > >> > > > on
> > > >> > > > > > > > >> > > > > >> your pull request.
> > > >> > > > > > > > >> > > > > >>
> > > >> > > > > > > > >> > > > > >> Please let me know if you have any
> > > questions.
> > > >> > > > > > > > >> > > > > >>
> > > >> > > > > > > > >> > > > > >> Regards,
> > > >> > > > > > > > >> > > > > >> Saikat
> > > >> > > > > > > > >> > > > > >>
> > > >> > > > > > > > >> > > > > >> On Sun, Nov 24, 2019 at 10:44 AM Saikat
> > > >> Maitra <
> > > >> > > > > > > > >> > > > saikat.maitra@gmail.com
> > > >> > > > > > > > >> > > > > >
> > > >> > > > > > > > >> > > > > >> wrote:
> > > >> > > > > > > > >> > > > > >>
> > > >> > > > > > > > >> > > > > >> > Hi Emmanouil,
> > > >> > > > > > > > >> > > > > >> >
> > > >> > > > > > > > >> > > > > >> > I have fixed the Flink module
> testsuites
> > > and
> > > >> > have
> > > >> > > > > setup
> > > >> > > > > > > new
> > > >> > > > > > > > >> > > project
> > > >> > > > > > > > >> > > > in
> > > >> > > > > > > > >> > > > > >> > teamcity.
> > > >> > > > > > > > >> > > > > >> >
> > > >> > > > > > > > >> > > > > >> >
> > > >> > > > > > > > >> > > > > >> >
> > > >> > > > > > > > >> > > > > >>
> > > >> > > > > > > > >> > > > >
> > > >> > > > > > > > >> > > >
> > > >> > > > > > > > >> > >
> > > >> > > > > > > > >> >
> > > >> > > > > > > > >>
> > > >> > > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview
> > > >> > > > > > > > >> > > > > >> >
> > > >> > > > > > > > >> > > > > >> > I am looking into setting up the
> build.
> > > >> > > > > > > > >> > > > > >> >
> > > >> > > > > > > > >> > > > > >> > Regards,
> > > >> > > > > > > > >> > > > > >> > Saikat
> > > >> > > > > > > > >> > > > > >> >
> > > >> > > > > > > > >> > > > > >> > On Sat, Nov 23, 2019 at 9:52 AM Saikat
> > > >> Maitra <
> > > >> > > > > > > > >> > > > > saikat.maitra@gmail.com>
> > > >> > > > > > > > >> > > > > >> > wrote:
> > > >> > > > > > > > >> > > > > >> >
> > > >> > > > > > > > >> > > > > >> >> Hi Emmanouil,
> > > >> > > > > > > > >> > > > > >> >>
> > > >> > > > > > > > >> > > > > >> >> Thank you for your email. Yes, the
> plan
> > > is
> > > >> > once
> > > >> > > > the
> > > >> > > > > > > Flink
> > > >> > > > > > > > >> PR is
> > > >> > > > > > > > >> > > > > merged
> > > >> > > > > > > > >> > > > > >> it
> > > >> > > > > > > > >> > > > > >> >> will provide the Licence, parent
> POMs etc
> > > >> and
> > > >> > > you
> > > >> > > > > can
> > > >> > > > > > > > rebase
> > > >> > > > > > > > >> > from
> > > >> > > > > > > > >> > > > > >> master
> > > >> > > > > > > > >> > > > > >> >> branch and apply the changes on top
> of
> > > it.
> > > >> > > > > > > > >> > > > > >> >>
> > > >> > > > > > > > >> > > > > >> >> I am getting some issues with test
> > > >> failures in
> > > >> > > > local
> > > >> > > > > > > with
> > > >> > > > > > > > >> > > > > >> >> GridTestProperties as the
> test.properties
> > > >> is
> > > >> > not
> > > >> > > > > > present
> > > >> > > > > > > > in
> > > >> > > > > > > > >> > this
> > > >> > > > > > > > >> > > > new
> > > >> > > > > > > > >> > > > > >> >> project but should be available from
> > > >> > > dependencies.
> > > >> > > > > > > > >> > > > > >> >>
> > > >> > > > > > > > >> > > > > >> >> Once I address this issue, I will go
> > > ahead
> > > >> and
> > > >> > > > merge
> > > >> > > > > > the
> > > >> > > > > > > > >> > changes
> > > >> > > > > > > > >> > > > and
> > > >> > > > > > > > >> > > > > >> then
> > > >> > > > > > > > >> > > > > >> >> we can take it from there.
> > > >> > > > > > > > >> > > > > >> >>
> > > >> > > > > > > > >> > > > > >> >> Regards,
> > > >> > > > > > > > >> > > > > >> >> Saikat
> > > >> > > > > > > > >> > > > > >> >>
> > > >> > > > > > > > >> > > > > >> >> On Fri, Nov 22, 2019 at 5:23 PM
> Emmanouil
> > > >> > > > > Gkatziouras
> > > >> > > > > > <
> > > >> > > > > > > > >> > > > > >> >> gkatzioura@gmail.com> wrote:
> > > >> > > > > > > > >> > > > > >> >>
> > > >> > > > > > > > >> > > > > >> >>> Hi all,
> > > >> > > > > > > > >> > > > > >> >>>
> > > >> > > > > > > > >> > > > > >> >>> I made my first pull request [1].
> Since
> > > >> this
> > > >> > > > > project
> > > >> > > > > > is
> > > >> > > > > > > > >> brand
> > > >> > > > > > > > >> > > new
> > > >> > > > > > > > >> > > > > (no
> > > >> > > > > > > > >> > > > > >> >>> parent poms, licensing), I reckoned
> it
> > > was
> > > >> > best
> > > >> > > > to
> > > >> > > > > > use
> > > >> > > > > > > > >> > Saikat's
> > > >> > > > > > > > >> > > > > >> branch on
> > > >> > > > > > > > >> > > > > >> >>> Flink.
> > > >> > > > > > > > >> > > > > >> >>> I suppose the Flink branch will be
> > > merged
> > > >> > > first.
> > > >> > > > If
> > > >> > > > > > not
> > > >> > > > > > > > >> please
> > > >> > > > > > > > >> > > > give
> > > >> > > > > > > > >> > > > > me
> > > >> > > > > > > > >> > > > > >> >>> guidelines on how I should proceed
> next.
> > > >> > > > > > > > >> > > > > >> >>>
> > > >> > > > > > > > >> > > > > >> >>> Kind regards
> > > >> > > > > > > > >> > > > > >> >>> Emmanouil
> > > >> > > > > > > > >> > > > > >> >>>
> > > >> > > > > > > > >> > > > > >> >>> [1]
> > > >> > > > > > https://github.com/apache/ignite-extensions/pull/2
> > > >> > > > > > > > >> > > > > >> >>>
> > > >> > > > > > > > >> > > > > >> >>> *Emmanouil Gkatziouras*
> > > >> > > > > > > > >> > > > > >> >>> https://egkatzioura.com/ |
> > > >> > > > > > > > >> > > > > >> >>>
> > > >> > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> > > > > > > > >> > > > > >> >>> https://github.com/gkatzioura
> > > >> > > > > > > > >> > > > > >> >>>
> > > >> > > > > > > > >> > > > > >> >>>
> > > >> > > > > > > > >> > > > > >> >>> On Fri, 22 Nov 2019 at 20:55, Denis
> > > Magda
> > > >> <
> > > >> > > > > > > > >> dmagda@apache.org>
> > > >> > > > > > > > >> > > > > wrote:
> > > >> > > > > > > > >> > > > > >> >>>
> > > >> > > > > > > > >> > > > > >> >>> > Awesome, ping us whenever you're
> > > ready!
> > > >> > > > > > > > >> > > > > >> >>> >
> > > >> > > > > > > > >> > > > > >> >>> > -
> > > >> > > > > > > > >> > > > > >> >>> > Denis
> > > >> > > > > > > > >> > > > > >> >>> >
> > > >> > > > > > > > >> > > > > >> >>> >
> > > >> > > > > > > > >> > > > > >> >>> > On Fri, Nov 22, 2019 at 12:52 PM
> > > >> Emmanouil
> > > >> > > > > > > Gkatziouras
> > > >> > > > > > > > <
> > > >> > > > > > > > >> > > > > >> >>> > gkatzioura@gmail.com>
> > > >> > > > > > > > >> > > > > >> >>> > wrote:
> > > >> > > > > > > > >> > > > > >> >>> >
> > > >> > > > > > > > >> > > > > >> >>> > > Hi all!
> > > >> > > > > > > > >> > > > > >> >>> > >
> > > >> > > > > > > > >> > > > > >> >>> > > I am sorry for being late on
> that. I
> > > >> was
> > > >> > > > trying
> > > >> > > > > > to
> > > >> > > > > > > > >> > refactor
> > > >> > > > > > > > >> > > > the
> > > >> > > > > > > > >> > > > > >> test
> > > >> > > > > > > > >> > > > > >> >>> in
> > > >> > > > > > > > >> > > > > >> >>> > > order not to be in need of any
> > > >> external
> > > >> > > tools
> > > >> > > > > or
> > > >> > > > > > > > >> spinning
> > > >> > > > > > > > >> > > up a
> > > >> > > > > > > > >> > > > > >> >>> server.
> > > >> > > > > > > > >> > > > > >> >>> > > I did forked the new repo and
> indeed
> > > >> my
> > > >> > > > changes
> > > >> > > > > > > > there,
> > > >> > > > > > > > >> so
> > > >> > > > > > > > >> > a
> > > >> > > > > > > > >> > > > pull
> > > >> > > > > > > > >> > > > > >> >>> request
> > > >> > > > > > > > >> > > > > >> >>> > is
> > > >> > > > > > > > >> > > > > >> >>> > > a matter of time!
> > > >> > > > > > > > >> > > > > >> >>> > >
> > > >> > > > > > > > >> > > > > >> >>> > > Kind regards
> > > >> > > > > > > > >> > > > > >> >>> > >
> > > >> > > > > > > > >> > > > > >> >>> > > *Emmanouil Gkatziouras*
> > > >> > > > > > > > >> > > > > >> >>> > > https://egkatzioura.com/ |
> > > >> > > > > > > > >> > > > > >> >>> > >
> > > >> > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> > > > > > > > >> > > > > >> >>> > > https://github.com/gkatzioura
> > > >> > > > > > > > >> > > > > >> >>> > >
> > > >> > > > > > > > >> > > > > >> >>> > >
> > > >> > > > > > > > >> > > > > >> >>> > > On Fri, 22 Nov 2019 at 20:45,
> Denis
> > > >> > Magda <
> > > >> > > > > > > > >> > > dmagda@apache.org>
> > > >> > > > > > > > >> > > > > >> wrote:
> > > >> > > > > > > > >> > > > > >> >>> > >
> > > >> > > > > > > > >> > > > > >> >>> > > > Hi Emmanouil,
> > > >> > > > > > > > >> > > > > >> >>> > > >
> > > >> > > > > > > > >> > > > > >> >>> > > > Do you have any questions or
> need
> > > >> any
> > > >> > > > support
> > > >> > > > > > > from
> > > >> > > > > > > > >> the
> > > >> > > > > > > > >> > > > > >> community?
> > > >> > > > > > > > >> > > > > >> >>> > > >
> > > >> > > > > > > > >> > > > > >> >>> > > > -
> > > >> > > > > > > > >> > > > > >> >>> > > > Denis
> > > >> > > > > > > > >> > > > > >> >>> > > >
> > > >> > > > > > > > >> > > > > >> >>> > > >
> > > >> > > > > > > > >> > > > > >> >>> > > > On Sun, Nov 10, 2019 at 3:07
> PM
> > > >> Saikat
> > > >> > > > > Maitra <
> > > >> > > > > > > > >> > > > > >> >>> saikat.maitra@gmail.com
> > > >> > > > > > > > >> > > > > >> >>> > >
> > > >> > > > > > > > >> > > > > >> >>> > > > wrote:
> > > >> > > > > > > > >> > > > > >> >>> > > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> Hi Emmanouil,
> > > >> > > > > > > > >> > > > > >> >>> > > >>
> > > >> > > > > > > > >> > > > > >> >>> > > >> Can you please take a looks
> at
> > > dev
> > > >> > > utils,
> > > >> > > > if
> > > >> > > > > > > this
> > > >> > > > > > > > is
> > > >> > > > > > > > >> > > > > something
> > > >> > > > > > > > >> > > > > >> >>> you are
> > > >> > > > > > > > >> > > > > >> >>> > > >> looking for
> > > >> > > > > > > > >> > > > > >> >>> > > >>
> > > >> > > > > > > > >> > > > >
> > > >> > > > > > >
> > > >> https://github.com/apache/ignite/tree/master/modules/dev-utils
> > > >> > > > > > > > >> > > > > >> ?
> > > >> > > > > > > > >> > > > > >> >>> > > >>
> > > >> > > > > > > > >> > > > > >> >>> > > >> IMO, if you can release
> Pub/Sub
> > > >> server
> > > >> > > in
> > > >> > > > > > maven
> > > >> > > > > > > > and
> > > >> > > > > > > > >> > then
> > > >> > > > > > > > >> > > > use
> > > >> > > > > > > > >> > > > > >> it as
> > > >> > > > > > > > >> > > > > >> >>> > > >> dependency, that would be
> great.
> > > >> > > > > > > > >> > > > > >> >>> > > >>
> > > >> > > > > > > > >> > > > > >> >>> > > >> Regards,
> > > >> > > > > > > > >> > > > > >> >>> > > >> Saikat
> > > >> > > > > > > > >> > > > > >> >>> > > >>
> > > >> > > > > > > > >> > > > > >> >>> > > >> On Sun, Nov 10, 2019 at 5:00
> PM
> > > >> Saikat
> > > >> > > > > Maitra
> > > >> > > > > > <
> > > >> > > > > > > > >> > > > > >> >>> > saikat.maitra@gmail.com>
> > > >> > > > > > > > >> > > > > >> >>> > > >> wrote:
> > > >> > > > > > > > >> > > > > >> >>> > > >>
> > > >> > > > > > > > >> > > > > >> >>> > > >> > Hi Emmanouil,
> > > >> > > > > > > > >> > > > > >> >>> > > >> >
> > > >> > > > > > > > >> > > > > >> >>> > > >> > The master branch has been
> > > >> > > initialized.
> > > >> > > > I
> > > >> > > > > > have
> > > >> > > > > > > > >> > raised a
> > > >> > > > > > > > >> > > > PR
> > > >> > > > > > > > >> > > > > to
> > > >> > > > > > > > >> > > > > >> >>> > migrate
> > > >> > > > > > > > >> > > > > >> >>> > > >> > Flink module to
> > > >> ignite-extensions.
> > > >> > > > > > > > >> > > > > >> >>> > > >> >
> > > >> > > > > > > > >> > > > > >> >>> > > >> > PR
> > > >> > > > > > > > >> >
> https://github.com/apache/ignite-extensions/pull/1
> > > >> > > > > > > > >> > > > > >> >>> > > >> > Jira :
> > > >> > > > > > > > >> > > >
> > > https://issues.apache.org/jira/browse/IGNITE-12356
> > > >> > > > > > > > >> > > > > >> >>> > > >> >
> > > >> > > > > > > > >> > > > > >> >>> > > >> > Regards,
> > > >> > > > > > > > >> > > > > >> >>> > > >> > Saikat
> > > >> > > > > > > > >> > > > > >> >>> > > >> >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >
> > > >> > > > > > > > >> > > > > >> >>> > > >> > On Sun, Nov 10, 2019 at
> 3:39 PM
> > > >> > > > Emmanouil
> > > >> > > > > > > > >> > Gkatziouras <
> > > >> > > > > > > > >> > > > > >> >>> > > >> > gkatzioura@gmail.com>
> wrote:
> > > >> > > > > > > > >> > > > > >> >>> > > >> >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> Hi all!
> > > >> > > > > > > > >> > > > > >> >>> > > >> >>
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> This is an update. The
> > > >> integration
> > > >> > is
> > > >> > > > > > tested
> > > >> > > > > > > > with
> > > >> > > > > > > > >> > > mocks
> > > >> > > > > > > > >> > > > > >> based
> > > >> > > > > > > > >> > > > > >> >>> on
> > > >> > > > > > > > >> > > > > >> >>> > the
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> original Ignite repo [1].
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> I am currently waiting
> for the
> > > >> > first
> > > >> > > > > > commits
> > > >> > > > > > > on
> > > >> > > > > > > > >> the
> > > >> > > > > > > > >> > > new
> > > >> > > > > > > > >> > > > > >> >>> integration
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> project
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> to take place in order to
> > > >> create my
> > > >> > > > pull
> > > >> > > > > > > > request
> > > >> > > > > > > > >> > > > > accordingly
> > > >> > > > > > > > >> > > > > >> >>> to the
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> initial
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> commits.
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> In order to simulate the
> > > Pub/Sub
> > > >> > > server
> > > >> > > > > in
> > > >> > > > > > a
> > > >> > > > > > > > >> test I
> > > >> > > > > > > > >> > > > used a
> > > >> > > > > > > > >> > > > > >> test
> > > >> > > > > > > > >> > > > > >> >>> > > server
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> implementation [2].
> > > >> Unfortunately
> > > >> > > they
> > > >> > > > > > don't
> > > >> > > > > > > > have
> > > >> > > > > > > > >> > this
> > > >> > > > > > > > >> > > > > util
> > > >> > > > > > > > >> > > > > >> >>> > uploaded
> > > >> > > > > > > > >> > > > > >> >>> > > >> in a
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> maven repo, thus it needs
> to
> > > be
> > > >> > > hosted
> > > >> > > > > on a
> > > >> > > > > > > > maven
> > > >> > > > > > > > >> > repo
> > > >> > > > > > > > >> > > > > >> (Apache
> > > >> > > > > > > > >> > > > > >> >>> > > License
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> 2.0). Is there a repo for
> > > >> utilities
> > > >> > > > that
> > > >> > > > > > > Ignite
> > > >> > > > > > > > >> uses
> > > >> > > > > > > > >> > > for
> > > >> > > > > > > > >> > > > > the
> > > >> > > > > > > > >> > > > > >> >>> > > >> test/build?
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> I have also created a
> ticket
> > > [3]
> > > >> > > > > > > > >> > > > > >> >>> > > >> >>
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> Kind regards
> > > >> > > > > > > > >> > > > > >> >>> > > >> >>
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> [1]
> > > >> > > > > > > > >> > > > > >> >>> >
> > > >> > > > > > > > >> > >
> > > >> > > > >
> https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> [2]
> > > >> > > > > > > > >> > > > > >> >>>
> > > >> > > > > > > > >>
> > > >> > https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> [3]
> > > >> > > > > > > > >> > >
> https://issues.apache.org/jira/browse/IGNITE-12262
> > > >> > > > > > > > >> > > > > >> >>> > > >> >>
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> *Emmanouil Gkatziouras*
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> https://egkatzioura.com/
> |
> > > >> > > > > > > > >> > > > > >> >>> > > >> >>
> > > >> > > > > > > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> > > > > > > > >> > > > > >> >>> > > >> >>
> https://github.com/gkatzioura
> > > >> > > > > > > > >> > > > > >> >>> > > >> >>
> > > >> > > > > > > > >> > > > > >> >>> > > >> >>
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> On Fri, 25 Oct 2019 at
> 03:35,
> > > >> > Saikat
> > > >> > > > > > Maitra <
> > > >> > > > > > > > >> > > > > >> >>> > saikat.maitra@gmail.com
> > > >> > > > > > > > >> > > > > >> >>> > > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> wrote:
> > > >> > > > > > > > >> > > > > >> >>> > > >> >>
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > Hello Ilya,
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > Thank you for your
> email.
> > > Yes,
> > > >> > > before
> > > >> > > > > we
> > > >> > > > > > > > remove
> > > >> > > > > > > > >> > any
> > > >> > > > > > > > >> > > > > >> >>> integration
> > > >> > > > > > > > >> > > > > >> >>> > we
> > > >> > > > > > > > >> > > > > >> >>> > > >> will
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > move them to another
> > > >> repository
> > > >> > and
> > > >> > > > > > > release.
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > We are thinking we will
> > > start
> > > >> > with
> > > >> > > > > > Pub/Sub
> > > >> > > > > > > > >> > Streamer
> > > >> > > > > > > > >> > > > > >> >>> > implementation
> > > >> > > > > > > > >> > > > > >> >>> > > >> and
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > continue to move other
> > > >> > integration
> > > >> > > as
> > > >> > > > > > > > >> mentioned in
> > > >> > > > > > > > >> > > the
> > > >> > > > > > > > >> > > > > >> >>> following
> > > >> > > > > > > > >> > > > > >> >>> > > doc.
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >>
> > > >> > > > > > > > >> > > > > >> >>> > > >>
> > > >> > > > > > > > >> > > > > >> >>> > >
> > > >> > > > > > > > >> > > > > >> >>> >
> > > >> > > > > > > > >> > > > > >> >>>
> > > >> > > > > > > > >> > > > > >>
> > > >> > > > > > > > >> > > > >
> > > >> > > > > > > > >> > > >
> > > >> > > > > > > > >> > >
> > > >> > > > > > > > >> >
> > > >> > > > > > > > >>
> > > >> > > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > Regards,
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > Saikat
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > On Thu, Oct 24, 2019 at
> > > 11:00
> > > >> AM
> > > >> > > Ilya
> > > >> > > > > > > > >> Kasnacheev <
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> >
> ilya.kasnacheev@gmail.com>
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > wrote:
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > Hello!
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > My take on this: we
> want
> > > to
> > > >> > > offload
> > > >> > > > > > some
> > > >> > > > > > > of
> > > >> > > > > > > > >> > > > > >> integrations,
> > > >> > > > > > > > >> > > > > >> >>> but
> > > >> > > > > > > > >> > > > > >> >>> > > >> please
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > commit
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > and release them to
> > > another
> > > >> > > project
> > > >> > > > > > > first,
> > > >> > > > > > > > >> and
> > > >> > > > > > > > >> > > only
> > > >> > > > > > > > >> > > > > >> then we
> > > >> > > > > > > > >> > > > > >> >>> > will
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> decide
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > to
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > remove them from our
> > > >> > repository.
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > Please also make sure
> they
> > > >> are
> > > >> > > > > > available
> > > >> > > > > > > > with
> > > >> > > > > > > > >> > same
> > > >> > > > > > > > >> > > > > >> artifact
> > > >> > > > > > > > >> > > > > >> >>> > name
> > > >> > > > > > > > >> > > > > >> >>> > > >> and
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> API.
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > If this has to be
> changed,
> > > >> > let's
> > > >> > > > wait
> > > >> > > > > > for
> > > >> > > > > > > > AI
> > > >> > > > > > > > >> 3.0
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > Regards,
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > --
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > Ilya Kasnacheev
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > ср, 23 окт. 2019 г. в
> > > 03:23,
> > > >> > > Saikat
> > > >> > > > > > > Maitra
> > > >> > > > > > > > <
> > > >> > > > > > > > >> > > > > >> >>> > > >> saikat.maitra@gmail.com>:
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > Hello Emmanouil,
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > As discussed
> earlier, I
> > > >> > > discussed
> > > >> > > > > > with
> > > >> > > > > > > > >> Apache
> > > >> > > > > > > > >> > > > Bahir
> > > >> > > > > > > > >> > > > > >> >>> community
> > > >> > > > > > > > >> > > > > >> >>> > > and
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> they
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > are
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > interested to have
> > > Apache
> > > >> > > Ignite
> > > >> > > > > > > > >> extensions as
> > > >> > > > > > > > >> > > > part
> > > >> > > > > > > > >> > > > > of
> > > >> > > > > > > > >> > > > > >> >>> Apache
> > > >> > > > > > > > >> > > > > >> >>> > > >> Bahir
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > project.
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > Can you please share
> > > >> Pub/Sub
> > > >> > > > > streamer
> > > >> > > > > > > > >> > > > implementation
> > > >> > > > > > > > >> > > > > >> >>> details
> > > >> > > > > > > > >> > > > > >> >>> > > with
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > Apache
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > Bahir community and
> > > >> request
> > > >> > for
> > > >> > > > > > > feedback.
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > https://bahir.apache.org/
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > I have also
> requested
> > > for
> > > >> > > > > contributor
> > > >> > > > > > > > >> access
> > > >> > > > > > > > >> > in
> > > >> > > > > > > > >> > > > Jira
> > > >> > > > > > > > >> > > > > >> for
> > > >> > > > > > > > >> > > > > >> >>> > Apache
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> Bahir
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > project so that I
> can
> > > >> create
> > > >> > > > issues
> > > >> > > > > > and
> > > >> > > > > > > > >> assign
> > > >> > > > > > > > >> > > to
> > > >> > > > > > > > >> > > > > >> myself.
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > I can help with code
> > > >> reviews
> > > >> > as
> > > >> > > > > well.
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > Here is the email
> thread
> > > >> for
> > > >> > > > > > reference
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > > > > >> > > > > >> >>> > >
> > > >> > > > > > > > >> > > >
> > > >> > > > > >
> > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > Regards,
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > Saikat
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > On Mon, Oct 21,
> 2019 at
> > > >> 7:54
> > > >> > PM
> > > >> > > > > > Saikat
> > > >> > > > > > > > >> Maitra
> > > >> > > > > > > > >> > <
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> saikat.maitra@gmail.com
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > wrote:
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > > Hello,
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > > As discussed I
> have
> > > >> created
> > > >> > > > > > following
> > > >> > > > > > > > >> > > discussion
> > > >> > > > > > > > >> > > > > >> >>> threads on
> > > >> > > > > > > > >> > > > > >> >>> > > our
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > migration
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > > proposals for
> Apache
> > > >> Ignite
> > > >> > > > > > > extensions:
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >>
> > > >> > > > > > > > >> > > > > >> >>> > > >>
> > > >> > > > > > > > >> > > > > >> >>> > >
> > > >> > > > > > > > >> > > > > >> >>> >
> > > >> > > > > > > > >> > > > > >> >>>
> > > >> > > > > > > > >> > > > > >>
> > > >> > > > > > > > >> > > > >
> > > >> > > > > > > > >> > > >
> > > >> > > > > > > > >> > >
> > > >> > > > > > > > >> >
> > > >> > > > > > > > >>
> > > >> > > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > > > > >> > > > > >> >>> > > >>
> > > >> > > > > > > > >> > > > > >>
> > > >> > > > > > > > >>
> > > >> > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > > I will share
> update
> > > as I
> > > >> > hear
> > > >> > > > > more
> > > >> > > > > > > > >> > > information.
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > > Regards,
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > > Saikat
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > > On Fri, Oct 18,
> 2019
> > > at
> > > >> > 9:03
> > > >> > > PM
> > > >> > > > > > > Saikat
> > > >> > > > > > > > >> > Maitra
> > > >> > > > > > > > >> > > <
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > saikat.maitra@gmail.com
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > > wrote:
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >> Hello Denis,
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >> Sure, sounds
> good. I
> > > >> will
> > > >> > > > > create a
> > > >> > > > > > > > >> separate
> > > >> > > > > > > > >> > > > > >> discussion
> > > >> > > > > > > > >> > > > > >> >>> > > thread
> > > >> > > > > > > > >> > > > > >> >>> > > >> to
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> get
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >> community
> feedback on
> > > >> > > whether
> > > >> > > > we
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >> should join the
> Bahir
> > > >> or
> > > >> > > > > kick-off
> > > >> > > > > > > > >> "Ignite
> > > >> > > > > > > > >> > > > > >> Extensions"
> > > >> > > > > > > > >> > > > > >> >>> > > project.
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >> Regards,
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >> Saikat
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >> On Thu, Oct 17,
> 2019
> > > at
> > > >> > 2:21
> > > >> > > > PM
> > > >> > > > > > > Denis
> > > >> > > > > > > > >> > Magda <
> > > >> > > > > > > > >> > > > > >> >>> > > >> dmagda@apache.org>
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > wrote:
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> Folks,
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> The concept of
> > > Apache
> > > >> > Bahir
> > > >> > > > is
> > > >> > > > > > > > >> precisely
> > > >> > > > > > > > >> > > what
> > > >> > > > > > > > >> > > > we
> > > >> > > > > > > > >> > > > > >> >>> need!
> > > >> > > > > > > > >> > > > > >> >>> > > >> Saikat,
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > thanks
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > for
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> doing the
> research.
> > > >> Why I
> > > >> > > > > believe
> > > >> > > > > > > > the *
> > > >> > > > > > > > >> > > idea*
> > > >> > > > > > > > >> > > > > >> fits us
> > > >> > > > > > > > >> > > > > >> >>> > well:
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    - All
> > > integrations
> > > >> can
> > > >> > > be
> > > >> > > > > > stored
> > > >> > > > > > > > in
> > > >> > > > > > > > >> > > > separate
> > > >> > > > > > > > >> > > > > >> >>> Github
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> repositories
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > and
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    have their
> dev
> > > >> > > lifecycles.
> > > >> > > > > > We've
> > > >> > > > > > > > not
> > > >> > > > > > > > >> > > > obliged
> > > >> > > > > > > > >> > > > > to
> > > >> > > > > > > > >> > > > > >> >>> couple
> > > >> > > > > > > > >> > > > > >> >>> > > all
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> the
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> integrations
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    in a single
> repo.
> > > >> For
> > > >> > > > > > instance,
> > > >> > > > > > > > >> Spark
> > > >> > > > > > > > >> > can
> > > >> > > > > > > > >> > > > be
> > > >> > > > > > > > >> > > > > >> >>> located
> > > >> > > > > > > > >> > > > > >> >>> > in
> > > >> > > > > > > > >> > > > > >> >>> > > a
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > dedicated
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> repo
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    while
> streaming
> > > >> > > > integrations
> > > >> > > > > > > might
> > > >> > > > > > > > >> be
> > > >> > > > > > > > >> > in
> > > >> > > > > > > > >> > > a
> > > >> > > > > > > > >> > > > > >> single
> > > >> > > > > > > > >> > > > > >> >>> one.
> > > >> > > > > > > > >> > > > > >> >>> > > >> It's
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> up
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > to
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > us.
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    - All the
> > > >> repositories
> > > >> > > and
> > > >> > > > > > > > >> integrations
> > > >> > > > > > > > >> > > > > belong
> > > >> > > > > > > > >> > > > > >> to
> > > >> > > > > > > > >> > > > > >> >>> ASF.
> > > >> > > > > > > > >> > > > > >> >>> > > >> We're
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> not
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> dumping
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    them on
> Github
> > > but
> > > >> > > rather
> > > >> > > > > keep
> > > >> > > > > > > > >> > supporting
> > > >> > > > > > > > >> > > > and
> > > >> > > > > > > > >> > > > > >> >>> > developing
> > > >> > > > > > > > >> > > > > >> >>> > > >> in
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> accordance with
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    ASF vision
> and
> > > >> > > practices.
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    - There is a
> way
> > > to
> > > >> > > reward
> > > >> > > > > > > > >> contributors
> > > >> > > > > > > > >> > > via
> > > >> > > > > > > > >> > > > > >> >>> > > committership
> > > >> > > > > > > > >> > > > > >> >>> > > >> and
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > PMC
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    membership.
> You
> > > >> won't
> > > >> > > get
> > > >> > > > > this
> > > >> > > > > > > if
> > > >> > > > > > > > a
> > > >> > > > > > > > >> > > project
> > > >> > > > > > > > >> > > > > is
> > > >> > > > > > > > >> > > > > >> >>> just
> > > >> > > > > > > > >> > > > > >> >>> > one
> > > >> > > > > > > > >> > > > > >> >>> > > of
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> the
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> millions of
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    Github
> projects.
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> Saikat, as
> Ignite
> > > PMC
> > > >> > > member,
> > > >> > > > > > could
> > > >> > > > > > > > you
> > > >> > > > > > > > >> > > please
> > > >> > > > > > > > >> > > > > >> >>> kick-off a
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> separate
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> dev-list
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> discussion to
> > > involve
> > > >> > more
> > > >> > > > > > > community
> > > >> > > > > > > > >> > members
> > > >> > > > > > > > >> > > > and
> > > >> > > > > > > > >> > > > > >> >>> > referring
> > > >> > > > > > > > >> > > > > >> >>> > > to
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> this
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> thread?
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> I think the
> primary
> > > >> > > question
> > > >> > > > > the
> > > >> > > > > > > > >> community
> > > >> > > > > > > > >> > > > needs
> > > >> > > > > > > > >> > > > > >> to
> > > >> > > > > > > > >> > > > > >> >>> > answer
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> whether
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > we
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> should join the
> > > Bahir
> > > >> or
> > > >> > > > > kick-off
> > > >> > > > > > > > >> "Ignite
> > > >> > > > > > > > >> > > > > >> Extensions"
> > > >> > > > > > > > >> > > > > >> >>> > > >> project?
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> -
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> Denis
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> On Thu, Oct 17,
> 2019
> > > >> at
> > > >> > > 2:54
> > > >> > > > AM
> > > >> > > > > > > > Alexey
> > > >> > > > > > > > >> > > > Zinoviev
> > > >> > > > > > > > >> > > > > <
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> zaleslaw.sin@gmail.com>
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> wrote:
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > Maybe we could
> > > move
> > > >> all
> > > >> > > our
> > > >> > > > > > > > Streaming
> > > >> > > > > > > > >> > > > > >> Integrations
> > > >> > > > > > > > >> > > > > >> >>> > there,
> > > >> > > > > > > > >> > > > > >> >>> > > >> but
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > what
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > is
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> about
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > maintaining
> and
> > > >> > committer
> > > >> > > > > > > > >> permissions to
> > > >> > > > > > > > >> > > the
> > > >> > > > > > > > >> > > > > new
> > > >> > > > > > > > >> > > > > >> >>> > > >> repositories?
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > I see the
> list of
> > > >> the
> > > >> > > > > > committers
> > > >> > > > > > > > and
> > > >> > > > > > > > >> PMC
> > > >> > > > > > > > >> > > > > members
> > > >> > > > > > > > >> > > > > >> >>> there
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > >> > > > > > > > >> > > https://bahir.apache.org/community-members/
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > Could somebody
> > > from
> > > >> > > Ignite
> > > >> > > > > > > > community
> > > >> > > > > > > > >> be
> > > >> > > > > > > > >> > > > added
> > > >> > > > > > > > >> > > > > to
> > > >> > > > > > > > >> > > > > >> >>> this
> > > >> > > > > > > > >> > > > > >> >>> > > list
> > > >> > > > > > > > >> > > > > >> >>> > > >> as
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> a
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > PMC/committer
> to
> > > >> > maintain
> > > >> > > > > > Ignite
> > > >> > > > > > > > >> > > > integrations?
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > If yes, I'd
> happy
> > > to
> > > >> > join
> > > >> > > > > this
> > > >> > > > > > > > >> project
> > > >> > > > > > > > >> > and
> > > >> > > > > > > > >> > > > > >> >>> collaborate
> > > >> > > > > > > > >> > > > > >> >>> > > with
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> these
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > guys
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > together
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > If no, and we
> > > should
> > > >> > > start
> > > >> > > > > from
> > > >> > > > > > > the
> > > >> > > > > > > > >> zero
> > > >> > > > > > > > >> > > > level
> > > >> > > > > > > > >> > > > > >> with
> > > >> > > > > > > > >> > > > > >> >>> > > >> external
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> PRs
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > -
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> hmmm,
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > it's better to
> > > have
> > > >> our
> > > >> > > own
> > > >> > > > > > > > external
> > > >> > > > > > > > >> > > > > repository
> > > >> > > > > > > > >> > > > > >> >>> with
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> ApacheBahirr
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> ideology.
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > Also, I agree,
> > > that
> > > >> all
> > > >> > > > > > > connectors
> > > >> > > > > > > > >> could
> > > >> > > > > > > > >> > > be
> > > >> > > > > > > > >> > > > > >> moved
> > > >> > > > > > > > >> > > > > >> >>> there
> > > >> > > > > > > > >> > > > > >> >>> > > (in
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> ApacheBahirr
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > like
> repository),
> > > >> but
> > > >> > not
> > > >> > > > all
> > > >> > > > > > > > modules
> > > >> > > > > > > > >> > from
> > > >> > > > > > > > >> > > > the
> > > >> > > > > > > > >> > > > > >> page
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> > >
> > > >> > > > > > > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > > > > >> > >
> > > >>
> > > >
> > >
>
>
>
> --
> Best regards,
> Ivan Pavlukhin
>

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Ivan Pavlukhin <vo...@gmail.com>]

Folks,

Do not we need to remove flink integration sources from main ignite
repository [1]? Is this work in progress?

[1] https://github.com/apache/ignite

вс, 29 дек. 2019 г. в 22:19, Emmanouil Gkatziouras <gk...@gmail.com>:
>
> Hi Saikat!
>
> Thank you for your time and assistance.
>
> Kind regards,
> Emmanouil
>
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>
>
> On Sun, 29 Dec 2019 at 18:35, Saikat Maitra <sa...@gmail.com> wrote:
>
> > Hi Emmanouil,
> >
> > Thank you for your contribution. I have merged your changes in master.
> >
> > Regards,
> > Saikat
> >
> >
> > On Sun, Dec 29, 2019 at 12:20 PM Saikat Maitra <sa...@gmail.com>
> > wrote:
> >
> > > Thank you, Emmanouil
> > >
> > > I am reviewing the changes.
> > >
> > > Regards,
> > > Saikat
> > >
> > > On Sat, Dec 28, 2019 at 3:48 PM Emmanouil Gkatziouras <
> > > gkatzioura@gmail.com> wrote:
> > >
> > >> Hi Saikat!
> > >>
> > >> The dependencies have been updated, and the merge conflicts resolved.
> > >> Kind regards
> > >>
> > >> On Sat, 28 Dec 2019, 23:27 Saikat Maitra, <sa...@gmail.com>
> > >> wrote:
> > >>
> > >> > Hi Emmanouil,
> > >> >
> > >> > Thank you for making the changes. I have merged my PR changes in
> > master.
> > >> > Can you please take the latest from master branch and resolve the
> > >> conflicts
> > >> > in your PR.
> > >> >
> > >> > I have also reviewed the changes in your PR and requested minor
> > changes
> > >> > specific to dependencies version and license information.
> > >> >
> > >> > Can you please review and share your feedback?
> > >> >
> > >> > Regards,
> > >> > Saikat
> > >> >
> > >> >
> > >> >
> > >> >
> > >> > On Sat, Dec 28, 2019 at 4:38 AM Emmanouil Gkatziouras <
> > >> > gkatzioura@gmail.com>
> > >> > wrote:
> > >> >
> > >> > > Hi Saikat!
> > >> > >
> > >> > > Thank you on this one! I tried it and it worked as expected [1]
> > >> > >
> > >> > > Kind regards
> > >> > > Emmanouil
> > >> > >
> > >> > > [1]
> > >> > >
> > >> > >
> > >> >
> > >>
> > https://ci.ignite.apache.org/viewLog.html?buildId=4888357&tab=buildLog&_focus=2046#_state=2046
> > >> > >
> > >> > > *Emmanouil Gkatziouras*
> > >> > > https://egkatzioura.com/ |
> > >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> > > https://github.com/gkatzioura
> > >> > >
> > >> > >
> > >> > > On Thu, 26 Dec 2019 at 19:50, Saikat Maitra <
> > saikat.maitra@gmail.com>
> > >> > > wrote:
> > >> > >
> > >> > > > Hi Emmanouil,
> > >> > > >
> > >> > > > I have been able to resolve the build failures for missing
> > modules.
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > https://ci.ignite.apache.org/viewLog.html?buildId=4883394&buildTypeId=IgniteExtensions_Build
> > >> > > >
> > >> > > > Will you please take the latest changes from my PR and run the
> > build
> > >> > for
> > >> > > > your PR.
> > >> > > >
> > >> > > > https://github.com/apache/ignite-extensions/pull/1/files
> > >> > > >
> > >> > > > The new build configurations should let your module build also
> > pass.
> > >> > > >
> > >> > > > The issue that was causing failure was that I had put ignite.zip
> > as
> > >> > > > artifact dependency and it was downloading the artifacts including
> > >> pom
> > >> > > file
> > >> > > > inside the build checkout folder of ignite-extensions and as a
> > >> result
> > >> > the
> > >> > > > pom.xml of ignite-extensions was getting overwritten by ignite's
> > >> > pom.xml.
> > >> > > > This was resulting in missing flink-ext module error as it was not
> > >> even
> > >> > > > present in the ignite main pom.xml.
> > >> > > >
> > >> > > > To resolve the issue I just had to download and extract artifact
> > of
> > >> > > > ignite.zip in a separate directory.
> > >> > > >
> > >> > > > Regards,
> > >> > > > Saikat
> > >> > > >
> > >> > > >
> > >> > > > On Sat, Nov 30, 2019 at 5:53 PM Emmanouil Gkatziouras <
> > >> > > > gkatzioura@gmail.com>
> > >> > > > wrote:
> > >> > > >
> > >> > > > > Hi Saikat!
> > >> > > > >
> > >> > > > > Thank you for your assistance on that!
> > >> > > > >
> > >> > > > > Kind regards
> > >> > > > >
> > >> > > > > *Emmanouil Gkatziouras*
> > >> > > > > https://egkatzioura.com/ |
> > >> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> > > > > https://github.com/gkatzioura
> > >> > > > >
> > >> > > > >
> > >> > > > > On Sat, 30 Nov 2019 at 23:51, Saikat Maitra <
> > >> saikat.maitra@gmail.com
> > >> > >
> > >> > > > > wrote:
> > >> > > > >
> > >> > > > > > Hi Emmanouil,
> > >> > > > > >
> > >> > > > > > I looked into the build logs and I observed that since I added
> > >> > > > artifacts
> > >> > > > > > dependencies to "Build apache Ignite"[1] it was able to pull
> > the
> > >> > > > required
> > >> > > > > > dependencies and was able to run the existing tests. It is
> > >> however
> > >> > > not
> > >> > > > > > identifying new modules like I changed from flink to flink-ext
> > >> or
> > >> > > > > pub-sub.
> > >> > > > > >
> > >> > > > > > [1]
> > >> > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > https://ci.ignite.apache.org/admin/editDependencies.html?id=buildType:IgniteExtensions_Build
> > >> > > > > >
> > >> > > > > > I will look into the issue and debug further.
> > >> > > > > >
> > >> > > > > > Regards,
> > >> > > > > > Saikat
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > On Sat, Nov 30, 2019 at 5:35 PM Emmanouil Gkatziouras <
> > >> > > > > > gkatzioura@gmail.com>
> > >> > > > > > wrote:
> > >> > > > > >
> > >> > > > > > > Hi Saikat!
> > >> > > > > > >
> > >> > > > > > > From the logs[1] it seems that TC cannot find the project.
> > >> > However
> > >> > > I
> > >> > > > > did
> > >> > > > > > > include the project on the parent pom (modules sections) [2]
> > >> > > > > > > I tried to reproduce locally unfortunately I am not aware of
> > >> the
> > >> > > full
> > >> > > > > > > context on TC. It seems as if a different parent pom is
> > being
> > >> > > picked
> > >> > > > > up.
> > >> > > > > > >
> > >> > > > > > > Kind regards
> > >> > > > > > >
> > >> > > > > > > [1]
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > https://ci.ignite.apache.org/viewLog.html?buildId=4804893&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=1704
> > >> > > > > > > [2]
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > https://github.com/gkatzioura/ignite-extensions/blob/d6a8beee7feff28f59d9a12be692016305564d25/pom.xml#L46
> > >> > > > > > >
> > >> > > > > > > *Emmanouil Gkatziouras*
> > >> > > > > > > https://egkatzioura.com/ |
> > >> > > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> > > > > > > https://github.com/gkatzioura
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > > > On Sat, 30 Nov 2019 at 22:20, Saikat Maitra <
> > >> > > saikat.maitra@gmail.com
> > >> > > > >
> > >> > > > > > > wrote:
> > >> > > > > > >
> > >> > > > > > > > Hello,
> > >> > > > > > > >
> > >> > > > > > > > I have made changes in Ignite Extensions build
> > >> configurations.
> > >> > > > > > > >
> > >> > > > > > > > It is now running the tests as expected.
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > https://ci.ignite.apache.org/viewLog.html?buildId=4804480&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> > >> > > > > > > >
> > >> > > > > > > > Regards,
> > >> > > > > > > > Saikat
> > >> > > > > > > >
> > >> > > > > > > > On Thu, Nov 28, 2019 at 1:29 PM Saikat Maitra <
> > >> > > > > saikat.maitra@gmail.com
> > >> > > > > > >
> > >> > > > > > > > wrote:
> > >> > > > > > > >
> > >> > > > > > > > > Hi,
> > >> > > > > > > > >
> > >> > > > > > > > > We do run build with -DskipTests and then configure
> > >> selective
> > >> > > > test
> > >> > > > > > > suites
> > >> > > > > > > > > to run.
> > >> > > > > > > > >
> > >> > > > > > > > > I will check why these test suites are not getting
> > >> executed.
> > >> > > > > > > > >
> > >> > > > > > > > > Regards
> > >> > > > > > > > > Saikat
> > >> > > > > > > > >
> > >> > > > > > > > > On Wed, 27 Nov 2019 at 3:05 PM, Emmanouil Gkatziouras <
> > >> > > > > > > > > gkatzioura@gmail.com> wrote:
> > >> > > > > > > > >
> > >> > > > > > > > >> Hi all,
> > >> > > > > > > > >>
> > >> > > > > > > > >> I did add a test suite and run the build with the
> > >> > > corresponding
> > >> > > > > > > > arguments,
> > >> > > > > > > > >> however the tests did not run [1].
> > >> > > > > > > > >> I checked the Flink's build logs and the same message
> > is
> > >> > > > displayed
> > >> > > > > > > there
> > >> > > > > > > > >> too `No tests to run` [2].
> > >> > > > > > > > >>
> > >> > > > > > > > >> [1]
> > >> > > > > > > > >>
> > >> > > > > > > > >>
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > https://ci.ignite.apache.org/viewLog.html?buildId=4798203&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=248
> > >> > > > > > > > >> [2]
> > >> > > > > > > > >>
> > >> > > > > > > > >>
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4789199&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=1256
> > >> > > > > > > > >>
> > >> > > > > > > > >> Kind regards
> > >> > > > > > > > >> *Emmanouil Gkatziouras*
> > >> > > > > > > > >> https://egkatzioura.com/ |
> > >> > > > > > > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> > > > > > > > >> https://github.com/gkatzioura
> > >> > > > > > > > >>
> > >> > > > > > > > >>
> > >> > > > > > > > >> On Wed, 27 Nov 2019 at 18:58, Saikat Maitra <
> > >> > > > > > saikat.maitra@gmail.com>
> > >> > > > > > > > >> wrote:
> > >> > > > > > > > >>
> > >> > > > > > > > >> > Hi,
> > >> > > > > > > > >> >
> > >> > > > > > > > >> > We will need to add test suites for tests to be
> > >> executed
> > >> > in
> > >> > > > > build.
> > >> > > > > > > We
> > >> > > > > > > > >> pass
> > >> > > > > > > > >> > them as TestSuites param. I had added Flink sink and
> > >> > source
> > >> > > > > > > > testsuites.
> > >> > > > > > > > >> >
> > >> > > > > > > > >> > Can you please review and confirm.
> > >> > > > > > > > >> >
> > >> > > > > > > > >> > Regards
> > >> > > > > > > > >> > Saikat
> > >> > > > > > > > >> >
> > >> > > > > > > > >> > On Wed, 27 Nov 2019 at 6:00 AM, Emmanouil
> > Gkatziouras <
> > >> > > > > > > > >> > gkatzioura@gmail.com>
> > >> > > > > > > > >> > wrote:
> > >> > > > > > > > >> >
> > >> > > > > > > > >> > > Hi all!
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > I was successful building and running my pull
> > >> requests.
> > >> > It
> > >> > > > > seems
> > >> > > > > > > > that
> > >> > > > > > > > >> the
> > >> > > > > > > > >> > > tests do not run both for Flink and Pub/Sub [1]
> > >> > > > > > > > >> > > If there is something I cannot do to make them
> > >> > > discoverable
> > >> > > > > > please
> > >> > > > > > > > >> let me
> > >> > > > > > > > >> > > know.
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > [1]
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> >
> > >> > > > > > > > >>
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4796722&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=283
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > *Emmanouil Gkatziouras*
> > >> > > > > > > > >> > > https://egkatzioura.com/ |
> > >> > > > > > > > >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> > > > > > > > >> > > https://github.com/gkatzioura
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > On Wed, 27 Nov 2019 at 04:57, Saikat Maitra <
> > >> > > > > > > > saikat.maitra@gmail.com>
> > >> > > > > > > > >> > > wrote:
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> > > > Hi Emmanouil,
> > >> > > > > > > > >> > > >
> > >> > > > > > > > >> > > > I have added you as contributor in Ignite
> > >> Extensions
> > >> > > > > project.
> > >> > > > > > > Can
> > >> > > > > > > > >> you
> > >> > > > > > > > >> > > > please check and confirm if you are able to see
> > the
> > >> > > > project
> > >> > > > > > and
> > >> > > > > > > > >> execute
> > >> > > > > > > > >> > > > build on your pull request.
> > >> > > > > > > > >> > > >
> > >> > > > > > > > >> > > > Regards,
> > >> > > > > > > > >> > > > Saikat
> > >> > > > > > > > >> > > >
> > >> > > > > > > > >> > > > On Tue, Nov 26, 2019 at 12:49 PM Emmanouil
> > >> > Gkatziouras <
> > >> > > > > > > > >> > > > gkatzioura@gmail.com>
> > >> > > > > > > > >> > > > wrote:
> > >> > > > > > > > >> > > >
> > >> > > > > > > > >> > > > > Hi all!
> > >> > > > > > > > >> > > > >
> > >> > > > > > > > >> > > > > If someone could give me read access to the Job
> > >> of
> > >> > > > Ignite
> > >> > > > > > > > >> Extensions
> > >> > > > > > > > >> > on
> > >> > > > > > > > >> > > > > Team City will greatly help me (username
> > >> > gkatzioura).
> > >> > > > > > > > >> > > > > I suppose the builds get triggered
> > automatically
> > >> on
> > >> > a
> > >> > > > pull
> > >> > > > > > > > >> request.
> > >> > > > > > > > >> > > > >
> > >> > > > > > > > >> > > > > Kind regards,
> > >> > > > > > > > >> > > > > Emmanouil
> > >> > > > > > > > >> > > > >
> > >> > > > > > > > >> > > > > *Emmanouil Gkatziouras*
> > >> > > > > > > > >> > > > > https://egkatzioura.com/ |
> > >> > > > > > > > >> > > > >
> > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> > > > > > > > >> > > > > https://github.com/gkatzioura
> > >> > > > > > > > >> > > > >
> > >> > > > > > > > >> > > > >
> > >> > > > > > > > >> > > > > On Sun, 24 Nov 2019 at 21:11, Emmanouil
> > >> Gkatziouras
> > >> > <
> > >> > > > > > > > >> > > > gkatzioura@gmail.com>
> > >> > > > > > > > >> > > > > wrote:
> > >> > > > > > > > >> > > > >
> > >> > > > > > > > >> > > > > > Hi Saikat!
> > >> > > > > > > > >> > > > > >
> > >> > > > > > > > >> > > > > > I just rebased with the flink branch.
> > >> > Unfortunately
> > >> > > I
> > >> > > > do
> > >> > > > > > not
> > >> > > > > > > > >> have
> > >> > > > > > > > >> > > read
> > >> > > > > > > > >> > > > > > access to the team city link you provided and
> > >> thus
> > >> > > > > > evaluate
> > >> > > > > > > > the
> > >> > > > > > > > >> > tests
> > >> > > > > > > > >> > > > for
> > >> > > > > > > > >> > > > > > my pull request.
> > >> > > > > > > > >> > > > > > I guess it has to do with ignite extensions
> > >> being
> > >> > > new.
> > >> > > > > > > > >> > > > > > My username is gkatzioura and email the one I
> > >> am
> > >> > > using
> > >> > > > > on
> > >> > > > > > > this
> > >> > > > > > > > >> > email
> > >> > > > > > > > >> > > > > >
> > >> > > > > > > > >> > > > > > Thank you for your time.
> > >> > > > > > > > >> > > > > >
> > >> > > > > > > > >> > > > > >
> > >> > > > > > > > >> > > > > > *Emmanouil Gkatziouras*
> > >> > > > > > > > >> > > > > > https://egkatzioura.com/ |
> > >> > > > > > > > >> > > > > >
> > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> > > > > > > > >> > > > > > https://github.com/gkatzioura
> > >> > > > > > > > >> > > > > >
> > >> > > > > > > > >> > > > > >
> > >> > > > > > > > >> > > > > > On Sun, 24 Nov 2019 at 18:03, Saikat Maitra <
> > >> > > > > > > > >> > saikat.maitra@gmail.com
> > >> > > > > > > > >> > > >
> > >> > > > > > > > >> > > > > > wrote:
> > >> > > > > > > > >> > > > > >
> > >> > > > > > > > >> > > > > >> Hi Emmanouil,
> > >> > > > > > > > >> > > > > >>
> > >> > > > > > > > >> > > > > >> The latest build on teamcity has passed on
> > >> Flink
> > >> > > pull
> > >> > > > > > > > request.
> > >> > > > > > > > >> > > > > >>
> > >> > > > > > > > >> > > > > >>
> > >> > > > > > > > >> > > > > >>
> > >> > > > > > > > >> > > > >
> > >> > > > > > > > >> > > >
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> >
> > >> > > > > > > > >>
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > https://ci.ignite.apache.org/viewLog.html?buildId=4788928&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> > >> > > > > > > > >> > > > > >>
> > >> > > > > > > > >> > > > > >> you should be able to take the changes I
> > made
> > >> in
> > >> > my
> > >> > > > PR
> > >> > > > > > and
> > >> > > > > > > > run
> > >> > > > > > > > >> > build
> > >> > > > > > > > >> > > > on
> > >> > > > > > > > >> > > > > >> your pull request.
> > >> > > > > > > > >> > > > > >>
> > >> > > > > > > > >> > > > > >> Please let me know if you have any
> > questions.
> > >> > > > > > > > >> > > > > >>
> > >> > > > > > > > >> > > > > >> Regards,
> > >> > > > > > > > >> > > > > >> Saikat
> > >> > > > > > > > >> > > > > >>
> > >> > > > > > > > >> > > > > >> On Sun, Nov 24, 2019 at 10:44 AM Saikat
> > >> Maitra <
> > >> > > > > > > > >> > > > saikat.maitra@gmail.com
> > >> > > > > > > > >> > > > > >
> > >> > > > > > > > >> > > > > >> wrote:
> > >> > > > > > > > >> > > > > >>
> > >> > > > > > > > >> > > > > >> > Hi Emmanouil,
> > >> > > > > > > > >> > > > > >> >
> > >> > > > > > > > >> > > > > >> > I have fixed the Flink module testsuites
> > and
> > >> > have
> > >> > > > > setup
> > >> > > > > > > new
> > >> > > > > > > > >> > > project
> > >> > > > > > > > >> > > > in
> > >> > > > > > > > >> > > > > >> > teamcity.
> > >> > > > > > > > >> > > > > >> >
> > >> > > > > > > > >> > > > > >> >
> > >> > > > > > > > >> > > > > >> >
> > >> > > > > > > > >> > > > > >>
> > >> > > > > > > > >> > > > >
> > >> > > > > > > > >> > > >
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> >
> > >> > > > > > > > >>
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview
> > >> > > > > > > > >> > > > > >> >
> > >> > > > > > > > >> > > > > >> > I am looking into setting up the build.
> > >> > > > > > > > >> > > > > >> >
> > >> > > > > > > > >> > > > > >> > Regards,
> > >> > > > > > > > >> > > > > >> > Saikat
> > >> > > > > > > > >> > > > > >> >
> > >> > > > > > > > >> > > > > >> > On Sat, Nov 23, 2019 at 9:52 AM Saikat
> > >> Maitra <
> > >> > > > > > > > >> > > > > saikat.maitra@gmail.com>
> > >> > > > > > > > >> > > > > >> > wrote:
> > >> > > > > > > > >> > > > > >> >
> > >> > > > > > > > >> > > > > >> >> Hi Emmanouil,
> > >> > > > > > > > >> > > > > >> >>
> > >> > > > > > > > >> > > > > >> >> Thank you for your email. Yes, the plan
> > is
> > >> > once
> > >> > > > the
> > >> > > > > > > Flink
> > >> > > > > > > > >> PR is
> > >> > > > > > > > >> > > > > merged
> > >> > > > > > > > >> > > > > >> it
> > >> > > > > > > > >> > > > > >> >> will provide the Licence, parent POMs etc
> > >> and
> > >> > > you
> > >> > > > > can
> > >> > > > > > > > rebase
> > >> > > > > > > > >> > from
> > >> > > > > > > > >> > > > > >> master
> > >> > > > > > > > >> > > > > >> >> branch and apply the changes on top of
> > it.
> > >> > > > > > > > >> > > > > >> >>
> > >> > > > > > > > >> > > > > >> >> I am getting some issues with test
> > >> failures in
> > >> > > > local
> > >> > > > > > > with
> > >> > > > > > > > >> > > > > >> >> GridTestProperties as the test.properties
> > >> is
> > >> > not
> > >> > > > > > present
> > >> > > > > > > > in
> > >> > > > > > > > >> > this
> > >> > > > > > > > >> > > > new
> > >> > > > > > > > >> > > > > >> >> project but should be available from
> > >> > > dependencies.
> > >> > > > > > > > >> > > > > >> >>
> > >> > > > > > > > >> > > > > >> >> Once I address this issue, I will go
> > ahead
> > >> and
> > >> > > > merge
> > >> > > > > > the
> > >> > > > > > > > >> > changes
> > >> > > > > > > > >> > > > and
> > >> > > > > > > > >> > > > > >> then
> > >> > > > > > > > >> > > > > >> >> we can take it from there.
> > >> > > > > > > > >> > > > > >> >>
> > >> > > > > > > > >> > > > > >> >> Regards,
> > >> > > > > > > > >> > > > > >> >> Saikat
> > >> > > > > > > > >> > > > > >> >>
> > >> > > > > > > > >> > > > > >> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil
> > >> > > > > Gkatziouras
> > >> > > > > > <
> > >> > > > > > > > >> > > > > >> >> gkatzioura@gmail.com> wrote:
> > >> > > > > > > > >> > > > > >> >>
> > >> > > > > > > > >> > > > > >> >>> Hi all,
> > >> > > > > > > > >> > > > > >> >>>
> > >> > > > > > > > >> > > > > >> >>> I made my first pull request [1]. Since
> > >> this
> > >> > > > > project
> > >> > > > > > is
> > >> > > > > > > > >> brand
> > >> > > > > > > > >> > > new
> > >> > > > > > > > >> > > > > (no
> > >> > > > > > > > >> > > > > >> >>> parent poms, licensing), I reckoned it
> > was
> > >> > best
> > >> > > > to
> > >> > > > > > use
> > >> > > > > > > > >> > Saikat's
> > >> > > > > > > > >> > > > > >> branch on
> > >> > > > > > > > >> > > > > >> >>> Flink.
> > >> > > > > > > > >> > > > > >> >>> I suppose the Flink branch will be
> > merged
> > >> > > first.
> > >> > > > If
> > >> > > > > > not
> > >> > > > > > > > >> please
> > >> > > > > > > > >> > > > give
> > >> > > > > > > > >> > > > > me
> > >> > > > > > > > >> > > > > >> >>> guidelines on how I should proceed next.
> > >> > > > > > > > >> > > > > >> >>>
> > >> > > > > > > > >> > > > > >> >>> Kind regards
> > >> > > > > > > > >> > > > > >> >>> Emmanouil
> > >> > > > > > > > >> > > > > >> >>>
> > >> > > > > > > > >> > > > > >> >>> [1]
> > >> > > > > > https://github.com/apache/ignite-extensions/pull/2
> > >> > > > > > > > >> > > > > >> >>>
> > >> > > > > > > > >> > > > > >> >>> *Emmanouil Gkatziouras*
> > >> > > > > > > > >> > > > > >> >>> https://egkatzioura.com/ |
> > >> > > > > > > > >> > > > > >> >>>
> > >> > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> > > > > > > > >> > > > > >> >>> https://github.com/gkatzioura
> > >> > > > > > > > >> > > > > >> >>>
> > >> > > > > > > > >> > > > > >> >>>
> > >> > > > > > > > >> > > > > >> >>> On Fri, 22 Nov 2019 at 20:55, Denis
> > Magda
> > >> <
> > >> > > > > > > > >> dmagda@apache.org>
> > >> > > > > > > > >> > > > > wrote:
> > >> > > > > > > > >> > > > > >> >>>
> > >> > > > > > > > >> > > > > >> >>> > Awesome, ping us whenever you're
> > ready!
> > >> > > > > > > > >> > > > > >> >>> >
> > >> > > > > > > > >> > > > > >> >>> > -
> > >> > > > > > > > >> > > > > >> >>> > Denis
> > >> > > > > > > > >> > > > > >> >>> >
> > >> > > > > > > > >> > > > > >> >>> >
> > >> > > > > > > > >> > > > > >> >>> > On Fri, Nov 22, 2019 at 12:52 PM
> > >> Emmanouil
> > >> > > > > > > Gkatziouras
> > >> > > > > > > > <
> > >> > > > > > > > >> > > > > >> >>> > gkatzioura@gmail.com>
> > >> > > > > > > > >> > > > > >> >>> > wrote:
> > >> > > > > > > > >> > > > > >> >>> >
> > >> > > > > > > > >> > > > > >> >>> > > Hi all!
> > >> > > > > > > > >> > > > > >> >>> > >
> > >> > > > > > > > >> > > > > >> >>> > > I am sorry for being late on that. I
> > >> was
> > >> > > > trying
> > >> > > > > > to
> > >> > > > > > > > >> > refactor
> > >> > > > > > > > >> > > > the
> > >> > > > > > > > >> > > > > >> test
> > >> > > > > > > > >> > > > > >> >>> in
> > >> > > > > > > > >> > > > > >> >>> > > order not to be in need of any
> > >> external
> > >> > > tools
> > >> > > > > or
> > >> > > > > > > > >> spinning
> > >> > > > > > > > >> > > up a
> > >> > > > > > > > >> > > > > >> >>> server.
> > >> > > > > > > > >> > > > > >> >>> > > I did forked the new repo and indeed
> > >> my
> > >> > > > changes
> > >> > > > > > > > there,
> > >> > > > > > > > >> so
> > >> > > > > > > > >> > a
> > >> > > > > > > > >> > > > pull
> > >> > > > > > > > >> > > > > >> >>> request
> > >> > > > > > > > >> > > > > >> >>> > is
> > >> > > > > > > > >> > > > > >> >>> > > a matter of time!
> > >> > > > > > > > >> > > > > >> >>> > >
> > >> > > > > > > > >> > > > > >> >>> > > Kind regards
> > >> > > > > > > > >> > > > > >> >>> > >
> > >> > > > > > > > >> > > > > >> >>> > > *Emmanouil Gkatziouras*
> > >> > > > > > > > >> > > > > >> >>> > > https://egkatzioura.com/ |
> > >> > > > > > > > >> > > > > >> >>> > >
> > >> > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> > > > > > > > >> > > > > >> >>> > > https://github.com/gkatzioura
> > >> > > > > > > > >> > > > > >> >>> > >
> > >> > > > > > > > >> > > > > >> >>> > >
> > >> > > > > > > > >> > > > > >> >>> > > On Fri, 22 Nov 2019 at 20:45, Denis
> > >> > Magda <
> > >> > > > > > > > >> > > dmagda@apache.org>
> > >> > > > > > > > >> > > > > >> wrote:
> > >> > > > > > > > >> > > > > >> >>> > >
> > >> > > > > > > > >> > > > > >> >>> > > > Hi Emmanouil,
> > >> > > > > > > > >> > > > > >> >>> > > >
> > >> > > > > > > > >> > > > > >> >>> > > > Do you have any questions or need
> > >> any
> > >> > > > support
> > >> > > > > > > from
> > >> > > > > > > > >> the
> > >> > > > > > > > >> > > > > >> community?
> > >> > > > > > > > >> > > > > >> >>> > > >
> > >> > > > > > > > >> > > > > >> >>> > > > -
> > >> > > > > > > > >> > > > > >> >>> > > > Denis
> > >> > > > > > > > >> > > > > >> >>> > > >
> > >> > > > > > > > >> > > > > >> >>> > > >
> > >> > > > > > > > >> > > > > >> >>> > > > On Sun, Nov 10, 2019 at 3:07 PM
> > >> Saikat
> > >> > > > > Maitra <
> > >> > > > > > > > >> > > > > >> >>> saikat.maitra@gmail.com
> > >> > > > > > > > >> > > > > >> >>> > >
> > >> > > > > > > > >> > > > > >> >>> > > > wrote:
> > >> > > > > > > > >> > > > > >> >>> > > >
> > >> > > > > > > > >> > > > > >> >>> > > >> Hi Emmanouil,
> > >> > > > > > > > >> > > > > >> >>> > > >>
> > >> > > > > > > > >> > > > > >> >>> > > >> Can you please take a looks at
> > dev
> > >> > > utils,
> > >> > > > if
> > >> > > > > > > this
> > >> > > > > > > > is
> > >> > > > > > > > >> > > > > something
> > >> > > > > > > > >> > > > > >> >>> you are
> > >> > > > > > > > >> > > > > >> >>> > > >> looking for
> > >> > > > > > > > >> > > > > >> >>> > > >>
> > >> > > > > > > > >> > > > >
> > >> > > > > > >
> > >> https://github.com/apache/ignite/tree/master/modules/dev-utils
> > >> > > > > > > > >> > > > > >> ?
> > >> > > > > > > > >> > > > > >> >>> > > >>
> > >> > > > > > > > >> > > > > >> >>> > > >> IMO, if you can release Pub/Sub
> > >> server
> > >> > > in
> > >> > > > > > maven
> > >> > > > > > > > and
> > >> > > > > > > > >> > then
> > >> > > > > > > > >> > > > use
> > >> > > > > > > > >> > > > > >> it as
> > >> > > > > > > > >> > > > > >> >>> > > >> dependency, that would be great.
> > >> > > > > > > > >> > > > > >> >>> > > >>
> > >> > > > > > > > >> > > > > >> >>> > > >> Regards,
> > >> > > > > > > > >> > > > > >> >>> > > >> Saikat
> > >> > > > > > > > >> > > > > >> >>> > > >>
> > >> > > > > > > > >> > > > > >> >>> > > >> On Sun, Nov 10, 2019 at 5:00 PM
> > >> Saikat
> > >> > > > > Maitra
> > >> > > > > > <
> > >> > > > > > > > >> > > > > >> >>> > saikat.maitra@gmail.com>
> > >> > > > > > > > >> > > > > >> >>> > > >> wrote:
> > >> > > > > > > > >> > > > > >> >>> > > >>
> > >> > > > > > > > >> > > > > >> >>> > > >> > Hi Emmanouil,
> > >> > > > > > > > >> > > > > >> >>> > > >> >
> > >> > > > > > > > >> > > > > >> >>> > > >> > The master branch has been
> > >> > > initialized.
> > >> > > > I
> > >> > > > > > have
> > >> > > > > > > > >> > raised a
> > >> > > > > > > > >> > > > PR
> > >> > > > > > > > >> > > > > to
> > >> > > > > > > > >> > > > > >> >>> > migrate
> > >> > > > > > > > >> > > > > >> >>> > > >> > Flink module to
> > >> ignite-extensions.
> > >> > > > > > > > >> > > > > >> >>> > > >> >
> > >> > > > > > > > >> > > > > >> >>> > > >> > PR
> > >> > > > > > > > >> > https://github.com/apache/ignite-extensions/pull/1
> > >> > > > > > > > >> > > > > >> >>> > > >> > Jira :
> > >> > > > > > > > >> > > >
> > https://issues.apache.org/jira/browse/IGNITE-12356
> > >> > > > > > > > >> > > > > >> >>> > > >> >
> > >> > > > > > > > >> > > > > >> >>> > > >> > Regards,
> > >> > > > > > > > >> > > > > >> >>> > > >> > Saikat
> > >> > > > > > > > >> > > > > >> >>> > > >> >
> > >> > > > > > > > >> > > > > >> >>> > > >> >
> > >> > > > > > > > >> > > > > >> >>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM
> > >> > > > Emmanouil
> > >> > > > > > > > >> > Gkatziouras <
> > >> > > > > > > > >> > > > > >> >>> > > >> > gkatzioura@gmail.com> wrote:
> > >> > > > > > > > >> > > > > >> >>> > > >> >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> Hi all!
> > >> > > > > > > > >> > > > > >> >>> > > >> >>
> > >> > > > > > > > >> > > > > >> >>> > > >> >> This is an update. The
> > >> integration
> > >> > is
> > >> > > > > > tested
> > >> > > > > > > > with
> > >> > > > > > > > >> > > mocks
> > >> > > > > > > > >> > > > > >> based
> > >> > > > > > > > >> > > > > >> >>> on
> > >> > > > > > > > >> > > > > >> >>> > the
> > >> > > > > > > > >> > > > > >> >>> > > >> >> original Ignite repo [1].
> > >> > > > > > > > >> > > > > >> >>> > > >> >> I am currently waiting for the
> > >> > first
> > >> > > > > > commits
> > >> > > > > > > on
> > >> > > > > > > > >> the
> > >> > > > > > > > >> > > new
> > >> > > > > > > > >> > > > > >> >>> integration
> > >> > > > > > > > >> > > > > >> >>> > > >> >> project
> > >> > > > > > > > >> > > > > >> >>> > > >> >> to take place in order to
> > >> create my
> > >> > > > pull
> > >> > > > > > > > request
> > >> > > > > > > > >> > > > > accordingly
> > >> > > > > > > > >> > > > > >> >>> to the
> > >> > > > > > > > >> > > > > >> >>> > > >> >> initial
> > >> > > > > > > > >> > > > > >> >>> > > >> >> commits.
> > >> > > > > > > > >> > > > > >> >>> > > >> >> In order to simulate the
> > Pub/Sub
> > >> > > server
> > >> > > > > in
> > >> > > > > > a
> > >> > > > > > > > >> test I
> > >> > > > > > > > >> > > > used a
> > >> > > > > > > > >> > > > > >> test
> > >> > > > > > > > >> > > > > >> >>> > > server
> > >> > > > > > > > >> > > > > >> >>> > > >> >> implementation [2].
> > >> Unfortunately
> > >> > > they
> > >> > > > > > don't
> > >> > > > > > > > have
> > >> > > > > > > > >> > this
> > >> > > > > > > > >> > > > > util
> > >> > > > > > > > >> > > > > >> >>> > uploaded
> > >> > > > > > > > >> > > > > >> >>> > > >> in a
> > >> > > > > > > > >> > > > > >> >>> > > >> >> maven repo, thus it needs to
> > be
> > >> > > hosted
> > >> > > > > on a
> > >> > > > > > > > maven
> > >> > > > > > > > >> > repo
> > >> > > > > > > > >> > > > > >> (Apache
> > >> > > > > > > > >> > > > > >> >>> > > License
> > >> > > > > > > > >> > > > > >> >>> > > >> >> 2.0). Is there a repo for
> > >> utilities
> > >> > > > that
> > >> > > > > > > Ignite
> > >> > > > > > > > >> uses
> > >> > > > > > > > >> > > for
> > >> > > > > > > > >> > > > > the
> > >> > > > > > > > >> > > > > >> >>> > > >> test/build?
> > >> > > > > > > > >> > > > > >> >>> > > >> >> I have also created a ticket
> > [3]
> > >> > > > > > > > >> > > > > >> >>> > > >> >>
> > >> > > > > > > > >> > > > > >> >>> > > >> >> Kind regards
> > >> > > > > > > > >> > > > > >> >>> > > >> >>
> > >> > > > > > > > >> > > > > >> >>> > > >> >> [1]
> > >> > > > > > > > >> > > > > >> >>> >
> > >> > > > > > > > >> > >
> > >> > > > > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > >> > > > > > > > >> > > > > >> >>> > > >> >> [2]
> > >> > > > > > > > >> > > > > >> >>>
> > >> > > > > > > > >>
> > >> > https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> > >> > > > > > > > >> > > > > >> >>> > > >> >> [3]
> > >> > > > > > > > >> > > https://issues.apache.org/jira/browse/IGNITE-12262
> > >> > > > > > > > >> > > > > >> >>> > > >> >>
> > >> > > > > > > > >> > > > > >> >>> > > >> >> *Emmanouil Gkatziouras*
> > >> > > > > > > > >> > > > > >> >>> > > >> >> https://egkatzioura.com/ |
> > >> > > > > > > > >> > > > > >> >>> > > >> >>
> > >> > > > > > > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> > > > > > > > >> > > > > >> >>> > > >> >> https://github.com/gkatzioura
> > >> > > > > > > > >> > > > > >> >>> > > >> >>
> > >> > > > > > > > >> > > > > >> >>> > > >> >>
> > >> > > > > > > > >> > > > > >> >>> > > >> >> On Fri, 25 Oct 2019 at 03:35,
> > >> > Saikat
> > >> > > > > > Maitra <
> > >> > > > > > > > >> > > > > >> >>> > saikat.maitra@gmail.com
> > >> > > > > > > > >> > > > > >> >>> > > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> wrote:
> > >> > > > > > > > >> > > > > >> >>> > > >> >>
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > Hello Ilya,
> > >> > > > > > > > >> > > > > >> >>> > > >> >> >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > Thank you for your email.
> > Yes,
> > >> > > before
> > >> > > > > we
> > >> > > > > > > > remove
> > >> > > > > > > > >> > any
> > >> > > > > > > > >> > > > > >> >>> integration
> > >> > > > > > > > >> > > > > >> >>> > we
> > >> > > > > > > > >> > > > > >> >>> > > >> will
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > move them to another
> > >> repository
> > >> > and
> > >> > > > > > > release.
> > >> > > > > > > > >> > > > > >> >>> > > >> >> >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > We are thinking we will
> > start
> > >> > with
> > >> > > > > > Pub/Sub
> > >> > > > > > > > >> > Streamer
> > >> > > > > > > > >> > > > > >> >>> > implementation
> > >> > > > > > > > >> > > > > >> >>> > > >> and
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > continue to move other
> > >> > integration
> > >> > > as
> > >> > > > > > > > >> mentioned in
> > >> > > > > > > > >> > > the
> > >> > > > > > > > >> > > > > >> >>> following
> > >> > > > > > > > >> > > > > >> >>> > > doc.
> > >> > > > > > > > >> > > > > >> >>> > > >> >> >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> >
> > >> > > > > > > > >> > > > > >> >>> > > >> >>
> > >> > > > > > > > >> > > > > >> >>> > > >>
> > >> > > > > > > > >> > > > > >> >>> > >
> > >> > > > > > > > >> > > > > >> >>> >
> > >> > > > > > > > >> > > > > >> >>>
> > >> > > > > > > > >> > > > > >>
> > >> > > > > > > > >> > > > >
> > >> > > > > > > > >> > > >
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> >
> > >> > > > > > > > >>
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > >> > > > > > > > >> > > > > >> >>> > > >> >> >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > Regards,
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > Saikat
> > >> > > > > > > > >> > > > > >> >>> > > >> >> >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > On Thu, Oct 24, 2019 at
> > 11:00
> > >> AM
> > >> > > Ilya
> > >> > > > > > > > >> Kasnacheev <
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > ilya.kasnacheev@gmail.com>
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > wrote:
> > >> > > > > > > > >> > > > > >> >>> > > >> >> >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > Hello!
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > My take on this: we want
> > to
> > >> > > offload
> > >> > > > > > some
> > >> > > > > > > of
> > >> > > > > > > > >> > > > > >> integrations,
> > >> > > > > > > > >> > > > > >> >>> but
> > >> > > > > > > > >> > > > > >> >>> > > >> please
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > commit
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > and release them to
> > another
> > >> > > project
> > >> > > > > > > first,
> > >> > > > > > > > >> and
> > >> > > > > > > > >> > > only
> > >> > > > > > > > >> > > > > >> then we
> > >> > > > > > > > >> > > > > >> >>> > will
> > >> > > > > > > > >> > > > > >> >>> > > >> >> decide
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > to
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > remove them from our
> > >> > repository.
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > Please also make sure they
> > >> are
> > >> > > > > > available
> > >> > > > > > > > with
> > >> > > > > > > > >> > same
> > >> > > > > > > > >> > > > > >> artifact
> > >> > > > > > > > >> > > > > >> >>> > name
> > >> > > > > > > > >> > > > > >> >>> > > >> and
> > >> > > > > > > > >> > > > > >> >>> > > >> >> API.
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > If this has to be changed,
> > >> > let's
> > >> > > > wait
> > >> > > > > > for
> > >> > > > > > > > AI
> > >> > > > > > > > >> 3.0
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > Regards,
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > --
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > Ilya Kasnacheev
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > ср, 23 окт. 2019 г. в
> > 03:23,
> > >> > > Saikat
> > >> > > > > > > Maitra
> > >> > > > > > > > <
> > >> > > > > > > > >> > > > > >> >>> > > >> saikat.maitra@gmail.com>:
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > Hello Emmanouil,
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > As discussed earlier, I
> > >> > > discussed
> > >> > > > > > with
> > >> > > > > > > > >> Apache
> > >> > > > > > > > >> > > > Bahir
> > >> > > > > > > > >> > > > > >> >>> community
> > >> > > > > > > > >> > > > > >> >>> > > and
> > >> > > > > > > > >> > > > > >> >>> > > >> >> they
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > are
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > interested to have
> > Apache
> > >> > > Ignite
> > >> > > > > > > > >> extensions as
> > >> > > > > > > > >> > > > part
> > >> > > > > > > > >> > > > > of
> > >> > > > > > > > >> > > > > >> >>> Apache
> > >> > > > > > > > >> > > > > >> >>> > > >> Bahir
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > project.
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > Can you please share
> > >> Pub/Sub
> > >> > > > > streamer
> > >> > > > > > > > >> > > > implementation
> > >> > > > > > > > >> > > > > >> >>> details
> > >> > > > > > > > >> > > > > >> >>> > > with
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > Apache
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > Bahir community and
> > >> request
> > >> > for
> > >> > > > > > > feedback.
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > https://bahir.apache.org/
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > I have also requested
> > for
> > >> > > > > contributor
> > >> > > > > > > > >> access
> > >> > > > > > > > >> > in
> > >> > > > > > > > >> > > > Jira
> > >> > > > > > > > >> > > > > >> for
> > >> > > > > > > > >> > > > > >> >>> > Apache
> > >> > > > > > > > >> > > > > >> >>> > > >> >> Bahir
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > project so that I can
> > >> create
> > >> > > > issues
> > >> > > > > > and
> > >> > > > > > > > >> assign
> > >> > > > > > > > >> > > to
> > >> > > > > > > > >> > > > > >> myself.
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > I can help with code
> > >> reviews
> > >> > as
> > >> > > > > well.
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > Here is the email thread
> > >> for
> > >> > > > > > reference
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > > > > >> > > > > >> >>> > >
> > >> > > > > > > > >> > > >
> > >> > > > > >
> > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > Regards,
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > Saikat
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > On Mon, Oct 21, 2019 at
> > >> 7:54
> > >> > PM
> > >> > > > > > Saikat
> > >> > > > > > > > >> Maitra
> > >> > > > > > > > >> > <
> > >> > > > > > > > >> > > > > >> >>> > > >> >> saikat.maitra@gmail.com
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > wrote:
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > > Hello,
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > > As discussed I have
> > >> created
> > >> > > > > > following
> > >> > > > > > > > >> > > discussion
> > >> > > > > > > > >> > > > > >> >>> threads on
> > >> > > > > > > > >> > > > > >> >>> > > our
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > migration
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > > proposals for Apache
> > >> Ignite
> > >> > > > > > > extensions:
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> >
> > >> > > > > > > > >> > > > > >> >>> > > >> >>
> > >> > > > > > > > >> > > > > >> >>> > > >>
> > >> > > > > > > > >> > > > > >> >>> > >
> > >> > > > > > > > >> > > > > >> >>> >
> > >> > > > > > > > >> > > > > >> >>>
> > >> > > > > > > > >> > > > > >>
> > >> > > > > > > > >> > > > >
> > >> > > > > > > > >> > > >
> > >> > > > > > > > >> > >
> > >> > > > > > > > >> >
> > >> > > > > > > > >>
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > > > > >> > > > > >> >>> > > >>
> > >> > > > > > > > >> > > > > >>
> > >> > > > > > > > >>
> > >> > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > > I will share update
> > as I
> > >> > hear
> > >> > > > > more
> > >> > > > > > > > >> > > information.
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > > Regards,
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > > Saikat
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > > On Fri, Oct 18, 2019
> > at
> > >> > 9:03
> > >> > > PM
> > >> > > > > > > Saikat
> > >> > > > > > > > >> > Maitra
> > >> > > > > > > > >> > > <
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > saikat.maitra@gmail.com
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > > wrote:
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >> Hello Denis,
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >> Sure, sounds good. I
> > >> will
> > >> > > > > create a
> > >> > > > > > > > >> separate
> > >> > > > > > > > >> > > > > >> discussion
> > >> > > > > > > > >> > > > > >> >>> > > thread
> > >> > > > > > > > >> > > > > >> >>> > > >> to
> > >> > > > > > > > >> > > > > >> >>> > > >> >> get
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >> community feedback on
> > >> > > whether
> > >> > > > we
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >> should join the Bahir
> > >> or
> > >> > > > > kick-off
> > >> > > > > > > > >> "Ignite
> > >> > > > > > > > >> > > > > >> Extensions"
> > >> > > > > > > > >> > > > > >> >>> > > project.
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >> Regards,
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >> Saikat
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >> On Thu, Oct 17, 2019
> > at
> > >> > 2:21
> > >> > > > PM
> > >> > > > > > > Denis
> > >> > > > > > > > >> > Magda <
> > >> > > > > > > > >> > > > > >> >>> > > >> dmagda@apache.org>
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > wrote:
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> Folks,
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> The concept of
> > Apache
> > >> > Bahir
> > >> > > > is
> > >> > > > > > > > >> precisely
> > >> > > > > > > > >> > > what
> > >> > > > > > > > >> > > > we
> > >> > > > > > > > >> > > > > >> >>> need!
> > >> > > > > > > > >> > > > > >> >>> > > >> Saikat,
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > thanks
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > for
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> doing the research.
> > >> Why I
> > >> > > > > believe
> > >> > > > > > > > the *
> > >> > > > > > > > >> > > idea*
> > >> > > > > > > > >> > > > > >> fits us
> > >> > > > > > > > >> > > > > >> >>> > well:
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    - All
> > integrations
> > >> can
> > >> > > be
> > >> > > > > > stored
> > >> > > > > > > > in
> > >> > > > > > > > >> > > > separate
> > >> > > > > > > > >> > > > > >> >>> Github
> > >> > > > > > > > >> > > > > >> >>> > > >> >> repositories
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > and
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    have their dev
> > >> > > lifecycles.
> > >> > > > > > We've
> > >> > > > > > > > not
> > >> > > > > > > > >> > > > obliged
> > >> > > > > > > > >> > > > > to
> > >> > > > > > > > >> > > > > >> >>> couple
> > >> > > > > > > > >> > > > > >> >>> > > all
> > >> > > > > > > > >> > > > > >> >>> > > >> >> the
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> integrations
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    in a single repo.
> > >> For
> > >> > > > > > instance,
> > >> > > > > > > > >> Spark
> > >> > > > > > > > >> > can
> > >> > > > > > > > >> > > > be
> > >> > > > > > > > >> > > > > >> >>> located
> > >> > > > > > > > >> > > > > >> >>> > in
> > >> > > > > > > > >> > > > > >> >>> > > a
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > dedicated
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> repo
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    while streaming
> > >> > > > integrations
> > >> > > > > > > might
> > >> > > > > > > > >> be
> > >> > > > > > > > >> > in
> > >> > > > > > > > >> > > a
> > >> > > > > > > > >> > > > > >> single
> > >> > > > > > > > >> > > > > >> >>> one.
> > >> > > > > > > > >> > > > > >> >>> > > >> It's
> > >> > > > > > > > >> > > > > >> >>> > > >> >> up
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > to
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > us.
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    - All the
> > >> repositories
> > >> > > and
> > >> > > > > > > > >> integrations
> > >> > > > > > > > >> > > > > belong
> > >> > > > > > > > >> > > > > >> to
> > >> > > > > > > > >> > > > > >> >>> ASF.
> > >> > > > > > > > >> > > > > >> >>> > > >> We're
> > >> > > > > > > > >> > > > > >> >>> > > >> >> not
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> dumping
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    them on Github
> > but
> > >> > > rather
> > >> > > > > keep
> > >> > > > > > > > >> > supporting
> > >> > > > > > > > >> > > > and
> > >> > > > > > > > >> > > > > >> >>> > developing
> > >> > > > > > > > >> > > > > >> >>> > > >> in
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> accordance with
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    ASF vision and
> > >> > > practices.
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    - There is a way
> > to
> > >> > > reward
> > >> > > > > > > > >> contributors
> > >> > > > > > > > >> > > via
> > >> > > > > > > > >> > > > > >> >>> > > committership
> > >> > > > > > > > >> > > > > >> >>> > > >> and
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > PMC
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    membership. You
> > >> won't
> > >> > > get
> > >> > > > > this
> > >> > > > > > > if
> > >> > > > > > > > a
> > >> > > > > > > > >> > > project
> > >> > > > > > > > >> > > > > is
> > >> > > > > > > > >> > > > > >> >>> just
> > >> > > > > > > > >> > > > > >> >>> > one
> > >> > > > > > > > >> > > > > >> >>> > > of
> > >> > > > > > > > >> > > > > >> >>> > > >> >> the
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> millions of
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    Github projects.
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> Saikat, as Ignite
> > PMC
> > >> > > member,
> > >> > > > > > could
> > >> > > > > > > > you
> > >> > > > > > > > >> > > please
> > >> > > > > > > > >> > > > > >> >>> kick-off a
> > >> > > > > > > > >> > > > > >> >>> > > >> >> separate
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> dev-list
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> discussion to
> > involve
> > >> > more
> > >> > > > > > > community
> > >> > > > > > > > >> > members
> > >> > > > > > > > >> > > > and
> > >> > > > > > > > >> > > > > >> >>> > referring
> > >> > > > > > > > >> > > > > >> >>> > > to
> > >> > > > > > > > >> > > > > >> >>> > > >> >> this
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> thread?
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> I think the primary
> > >> > > question
> > >> > > > > the
> > >> > > > > > > > >> community
> > >> > > > > > > > >> > > > needs
> > >> > > > > > > > >> > > > > >> to
> > >> > > > > > > > >> > > > > >> >>> > answer
> > >> > > > > > > > >> > > > > >> >>> > > >> >> whether
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > we
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> should join the
> > Bahir
> > >> or
> > >> > > > > kick-off
> > >> > > > > > > > >> "Ignite
> > >> > > > > > > > >> > > > > >> Extensions"
> > >> > > > > > > > >> > > > > >> >>> > > >> project?
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> -
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> Denis
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> On Thu, Oct 17, 2019
> > >> at
> > >> > > 2:54
> > >> > > > AM
> > >> > > > > > > > Alexey
> > >> > > > > > > > >> > > > Zinoviev
> > >> > > > > > > > >> > > > > <
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > zaleslaw.sin@gmail.com>
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> wrote:
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > Maybe we could
> > move
> > >> all
> > >> > > our
> > >> > > > > > > > Streaming
> > >> > > > > > > > >> > > > > >> Integrations
> > >> > > > > > > > >> > > > > >> >>> > there,
> > >> > > > > > > > >> > > > > >> >>> > > >> but
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > what
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > is
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> about
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > maintaining and
> > >> > committer
> > >> > > > > > > > >> permissions to
> > >> > > > > > > > >> > > the
> > >> > > > > > > > >> > > > > new
> > >> > > > > > > > >> > > > > >> >>> > > >> repositories?
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > I see the list of
> > >> the
> > >> > > > > > committers
> > >> > > > > > > > and
> > >> > > > > > > > >> PMC
> > >> > > > > > > > >> > > > > members
> > >> > > > > > > > >> > > > > >> >>> there
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > >> > > > > > > > >> > > https://bahir.apache.org/community-members/
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > Could somebody
> > from
> > >> > > Ignite
> > >> > > > > > > > community
> > >> > > > > > > > >> be
> > >> > > > > > > > >> > > > added
> > >> > > > > > > > >> > > > > to
> > >> > > > > > > > >> > > > > >> >>> this
> > >> > > > > > > > >> > > > > >> >>> > > list
> > >> > > > > > > > >> > > > > >> >>> > > >> as
> > >> > > > > > > > >> > > > > >> >>> > > >> >> a
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > PMC/committer to
> > >> > maintain
> > >> > > > > > Ignite
> > >> > > > > > > > >> > > > integrations?
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > If yes, I'd happy
> > to
> > >> > join
> > >> > > > > this
> > >> > > > > > > > >> project
> > >> > > > > > > > >> > and
> > >> > > > > > > > >> > > > > >> >>> collaborate
> > >> > > > > > > > >> > > > > >> >>> > > with
> > >> > > > > > > > >> > > > > >> >>> > > >> >> these
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > guys
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > together
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > If no, and we
> > should
> > >> > > start
> > >> > > > > from
> > >> > > > > > > the
> > >> > > > > > > > >> zero
> > >> > > > > > > > >> > > > level
> > >> > > > > > > > >> > > > > >> with
> > >> > > > > > > > >> > > > > >> >>> > > >> external
> > >> > > > > > > > >> > > > > >> >>> > > >> >> PRs
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > -
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> hmmm,
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > it's better to
> > have
> > >> our
> > >> > > own
> > >> > > > > > > > external
> > >> > > > > > > > >> > > > > repository
> > >> > > > > > > > >> > > > > >> >>> with
> > >> > > > > > > > >> > > > > >> >>> > > >> >> ApacheBahirr
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> ideology.
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > Also, I agree,
> > that
> > >> all
> > >> > > > > > > connectors
> > >> > > > > > > > >> could
> > >> > > > > > > > >> > > be
> > >> > > > > > > > >> > > > > >> moved
> > >> > > > > > > > >> > > > > >> >>> there
> > >> > > > > > > > >> > > > > >> >>> > > (in
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> ApacheBahirr
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > like repository),
> > >> but
> > >> > not
> > >> > > > all
> > >> > > > > > > > modules
> > >> > > > > > > > >> > from
> > >> > > > > > > > >> > > > the
> > >> > > > > > > > >> > > > > >> page
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> > >
> > >> > > > > > > > >> > > > > >> >>> > > >> >> >
> > >> > > > > > > > >> > >
> > >>
> > >
> >



-- 
Best regards,
Ivan Pavlukhin

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Emmanouil Gkatziouras <gk...@gmail.com>]

Hi Saikat!

Thank you for your time and assistance.

Kind regards,
Emmanouil

*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura


On Sun, 29 Dec 2019 at 18:35, Saikat Maitra <sa...@gmail.com> wrote:

> Hi Emmanouil,
>
> Thank you for your contribution. I have merged your changes in master.
>
> Regards,
> Saikat
>
>
> On Sun, Dec 29, 2019 at 12:20 PM Saikat Maitra <sa...@gmail.com>
> wrote:
>
> > Thank you, Emmanouil
> >
> > I am reviewing the changes.
> >
> > Regards,
> > Saikat
> >
> > On Sat, Dec 28, 2019 at 3:48 PM Emmanouil Gkatziouras <
> > gkatzioura@gmail.com> wrote:
> >
> >> Hi Saikat!
> >>
> >> The dependencies have been updated, and the merge conflicts resolved.
> >> Kind regards
> >>
> >> On Sat, 28 Dec 2019, 23:27 Saikat Maitra, <sa...@gmail.com>
> >> wrote:
> >>
> >> > Hi Emmanouil,
> >> >
> >> > Thank you for making the changes. I have merged my PR changes in
> master.
> >> > Can you please take the latest from master branch and resolve the
> >> conflicts
> >> > in your PR.
> >> >
> >> > I have also reviewed the changes in your PR and requested minor
> changes
> >> > specific to dependencies version and license information.
> >> >
> >> > Can you please review and share your feedback?
> >> >
> >> > Regards,
> >> > Saikat
> >> >
> >> >
> >> >
> >> >
> >> > On Sat, Dec 28, 2019 at 4:38 AM Emmanouil Gkatziouras <
> >> > gkatzioura@gmail.com>
> >> > wrote:
> >> >
> >> > > Hi Saikat!
> >> > >
> >> > > Thank you on this one! I tried it and it worked as expected [1]
> >> > >
> >> > > Kind regards
> >> > > Emmanouil
> >> > >
> >> > > [1]
> >> > >
> >> > >
> >> >
> >>
> https://ci.ignite.apache.org/viewLog.html?buildId=4888357&tab=buildLog&_focus=2046#_state=2046
> >> > >
> >> > > *Emmanouil Gkatziouras*
> >> > > https://egkatzioura.com/ |
> >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > https://github.com/gkatzioura
> >> > >
> >> > >
> >> > > On Thu, 26 Dec 2019 at 19:50, Saikat Maitra <
> saikat.maitra@gmail.com>
> >> > > wrote:
> >> > >
> >> > > > Hi Emmanouil,
> >> > > >
> >> > > > I have been able to resolve the build failures for missing
> modules.
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> https://ci.ignite.apache.org/viewLog.html?buildId=4883394&buildTypeId=IgniteExtensions_Build
> >> > > >
> >> > > > Will you please take the latest changes from my PR and run the
> build
> >> > for
> >> > > > your PR.
> >> > > >
> >> > > > https://github.com/apache/ignite-extensions/pull/1/files
> >> > > >
> >> > > > The new build configurations should let your module build also
> pass.
> >> > > >
> >> > > > The issue that was causing failure was that I had put ignite.zip
> as
> >> > > > artifact dependency and it was downloading the artifacts including
> >> pom
> >> > > file
> >> > > > inside the build checkout folder of ignite-extensions and as a
> >> result
> >> > the
> >> > > > pom.xml of ignite-extensions was getting overwritten by ignite's
> >> > pom.xml.
> >> > > > This was resulting in missing flink-ext module error as it was not
> >> even
> >> > > > present in the ignite main pom.xml.
> >> > > >
> >> > > > To resolve the issue I just had to download and extract artifact
> of
> >> > > > ignite.zip in a separate directory.
> >> > > >
> >> > > > Regards,
> >> > > > Saikat
> >> > > >
> >> > > >
> >> > > > On Sat, Nov 30, 2019 at 5:53 PM Emmanouil Gkatziouras <
> >> > > > gkatzioura@gmail.com>
> >> > > > wrote:
> >> > > >
> >> > > > > Hi Saikat!
> >> > > > >
> >> > > > > Thank you for your assistance on that!
> >> > > > >
> >> > > > > Kind regards
> >> > > > >
> >> > > > > *Emmanouil Gkatziouras*
> >> > > > > https://egkatzioura.com/ |
> >> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > > > https://github.com/gkatzioura
> >> > > > >
> >> > > > >
> >> > > > > On Sat, 30 Nov 2019 at 23:51, Saikat Maitra <
> >> saikat.maitra@gmail.com
> >> > >
> >> > > > > wrote:
> >> > > > >
> >> > > > > > Hi Emmanouil,
> >> > > > > >
> >> > > > > > I looked into the build logs and I observed that since I added
> >> > > > artifacts
> >> > > > > > dependencies to "Build apache Ignite"[1] it was able to pull
> the
> >> > > > required
> >> > > > > > dependencies and was able to run the existing tests. It is
> >> however
> >> > > not
> >> > > > > > identifying new modules like I changed from flink to flink-ext
> >> or
> >> > > > > pub-sub.
> >> > > > > >
> >> > > > > > [1]
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://ci.ignite.apache.org/admin/editDependencies.html?id=buildType:IgniteExtensions_Build
> >> > > > > >
> >> > > > > > I will look into the issue and debug further.
> >> > > > > >
> >> > > > > > Regards,
> >> > > > > > Saikat
> >> > > > > >
> >> > > > > >
> >> > > > > > On Sat, Nov 30, 2019 at 5:35 PM Emmanouil Gkatziouras <
> >> > > > > > gkatzioura@gmail.com>
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > > > Hi Saikat!
> >> > > > > > >
> >> > > > > > > From the logs[1] it seems that TC cannot find the project.
> >> > However
> >> > > I
> >> > > > > did
> >> > > > > > > include the project on the parent pom (modules sections) [2]
> >> > > > > > > I tried to reproduce locally unfortunately I am not aware of
> >> the
> >> > > full
> >> > > > > > > context on TC. It seems as if a different parent pom is
> being
> >> > > picked
> >> > > > > up.
> >> > > > > > >
> >> > > > > > > Kind regards
> >> > > > > > >
> >> > > > > > > [1]
> >> > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://ci.ignite.apache.org/viewLog.html?buildId=4804893&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=1704
> >> > > > > > > [2]
> >> > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://github.com/gkatzioura/ignite-extensions/blob/d6a8beee7feff28f59d9a12be692016305564d25/pom.xml#L46
> >> > > > > > >
> >> > > > > > > *Emmanouil Gkatziouras*
> >> > > > > > > https://egkatzioura.com/ |
> >> > > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > > > > > https://github.com/gkatzioura
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > On Sat, 30 Nov 2019 at 22:20, Saikat Maitra <
> >> > > saikat.maitra@gmail.com
> >> > > > >
> >> > > > > > > wrote:
> >> > > > > > >
> >> > > > > > > > Hello,
> >> > > > > > > >
> >> > > > > > > > I have made changes in Ignite Extensions build
> >> configurations.
> >> > > > > > > >
> >> > > > > > > > It is now running the tests as expected.
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://ci.ignite.apache.org/viewLog.html?buildId=4804480&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> >> > > > > > > >
> >> > > > > > > > Regards,
> >> > > > > > > > Saikat
> >> > > > > > > >
> >> > > > > > > > On Thu, Nov 28, 2019 at 1:29 PM Saikat Maitra <
> >> > > > > saikat.maitra@gmail.com
> >> > > > > > >
> >> > > > > > > > wrote:
> >> > > > > > > >
> >> > > > > > > > > Hi,
> >> > > > > > > > >
> >> > > > > > > > > We do run build with -DskipTests and then configure
> >> selective
> >> > > > test
> >> > > > > > > suites
> >> > > > > > > > > to run.
> >> > > > > > > > >
> >> > > > > > > > > I will check why these test suites are not getting
> >> executed.
> >> > > > > > > > >
> >> > > > > > > > > Regards
> >> > > > > > > > > Saikat
> >> > > > > > > > >
> >> > > > > > > > > On Wed, 27 Nov 2019 at 3:05 PM, Emmanouil Gkatziouras <
> >> > > > > > > > > gkatzioura@gmail.com> wrote:
> >> > > > > > > > >
> >> > > > > > > > >> Hi all,
> >> > > > > > > > >>
> >> > > > > > > > >> I did add a test suite and run the build with the
> >> > > corresponding
> >> > > > > > > > arguments,
> >> > > > > > > > >> however the tests did not run [1].
> >> > > > > > > > >> I checked the Flink's build logs and the same message
> is
> >> > > > displayed
> >> > > > > > > there
> >> > > > > > > > >> too `No tests to run` [2].
> >> > > > > > > > >>
> >> > > > > > > > >> [1]
> >> > > > > > > > >>
> >> > > > > > > > >>
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://ci.ignite.apache.org/viewLog.html?buildId=4798203&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=248
> >> > > > > > > > >> [2]
> >> > > > > > > > >>
> >> > > > > > > > >>
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4789199&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=1256
> >> > > > > > > > >>
> >> > > > > > > > >> Kind regards
> >> > > > > > > > >> *Emmanouil Gkatziouras*
> >> > > > > > > > >> https://egkatzioura.com/ |
> >> > > > > > > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > > > > > > >> https://github.com/gkatzioura
> >> > > > > > > > >>
> >> > > > > > > > >>
> >> > > > > > > > >> On Wed, 27 Nov 2019 at 18:58, Saikat Maitra <
> >> > > > > > saikat.maitra@gmail.com>
> >> > > > > > > > >> wrote:
> >> > > > > > > > >>
> >> > > > > > > > >> > Hi,
> >> > > > > > > > >> >
> >> > > > > > > > >> > We will need to add test suites for tests to be
> >> executed
> >> > in
> >> > > > > build.
> >> > > > > > > We
> >> > > > > > > > >> pass
> >> > > > > > > > >> > them as TestSuites param. I had added Flink sink and
> >> > source
> >> > > > > > > > testsuites.
> >> > > > > > > > >> >
> >> > > > > > > > >> > Can you please review and confirm.
> >> > > > > > > > >> >
> >> > > > > > > > >> > Regards
> >> > > > > > > > >> > Saikat
> >> > > > > > > > >> >
> >> > > > > > > > >> > On Wed, 27 Nov 2019 at 6:00 AM, Emmanouil
> Gkatziouras <
> >> > > > > > > > >> > gkatzioura@gmail.com>
> >> > > > > > > > >> > wrote:
> >> > > > > > > > >> >
> >> > > > > > > > >> > > Hi all!
> >> > > > > > > > >> > >
> >> > > > > > > > >> > > I was successful building and running my pull
> >> requests.
> >> > It
> >> > > > > seems
> >> > > > > > > > that
> >> > > > > > > > >> the
> >> > > > > > > > >> > > tests do not run both for Flink and Pub/Sub [1]
> >> > > > > > > > >> > > If there is something I cannot do to make them
> >> > > discoverable
> >> > > > > > please
> >> > > > > > > > >> let me
> >> > > > > > > > >> > > know.
> >> > > > > > > > >> > >
> >> > > > > > > > >> > > [1]
> >> > > > > > > > >> > >
> >> > > > > > > > >> > >
> >> > > > > > > > >> >
> >> > > > > > > > >>
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4796722&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=283
> >> > > > > > > > >> > >
> >> > > > > > > > >> > > *Emmanouil Gkatziouras*
> >> > > > > > > > >> > > https://egkatzioura.com/ |
> >> > > > > > > > >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > > > > > > >> > > https://github.com/gkatzioura
> >> > > > > > > > >> > >
> >> > > > > > > > >> > >
> >> > > > > > > > >> > > On Wed, 27 Nov 2019 at 04:57, Saikat Maitra <
> >> > > > > > > > saikat.maitra@gmail.com>
> >> > > > > > > > >> > > wrote:
> >> > > > > > > > >> > >
> >> > > > > > > > >> > > > Hi Emmanouil,
> >> > > > > > > > >> > > >
> >> > > > > > > > >> > > > I have added you as contributor in Ignite
> >> Extensions
> >> > > > > project.
> >> > > > > > > Can
> >> > > > > > > > >> you
> >> > > > > > > > >> > > > please check and confirm if you are able to see
> the
> >> > > > project
> >> > > > > > and
> >> > > > > > > > >> execute
> >> > > > > > > > >> > > > build on your pull request.
> >> > > > > > > > >> > > >
> >> > > > > > > > >> > > > Regards,
> >> > > > > > > > >> > > > Saikat
> >> > > > > > > > >> > > >
> >> > > > > > > > >> > > > On Tue, Nov 26, 2019 at 12:49 PM Emmanouil
> >> > Gkatziouras <
> >> > > > > > > > >> > > > gkatzioura@gmail.com>
> >> > > > > > > > >> > > > wrote:
> >> > > > > > > > >> > > >
> >> > > > > > > > >> > > > > Hi all!
> >> > > > > > > > >> > > > >
> >> > > > > > > > >> > > > > If someone could give me read access to the Job
> >> of
> >> > > > Ignite
> >> > > > > > > > >> Extensions
> >> > > > > > > > >> > on
> >> > > > > > > > >> > > > > Team City will greatly help me (username
> >> > gkatzioura).
> >> > > > > > > > >> > > > > I suppose the builds get triggered
> automatically
> >> on
> >> > a
> >> > > > pull
> >> > > > > > > > >> request.
> >> > > > > > > > >> > > > >
> >> > > > > > > > >> > > > > Kind regards,
> >> > > > > > > > >> > > > > Emmanouil
> >> > > > > > > > >> > > > >
> >> > > > > > > > >> > > > > *Emmanouil Gkatziouras*
> >> > > > > > > > >> > > > > https://egkatzioura.com/ |
> >> > > > > > > > >> > > > >
> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > > > > > > >> > > > > https://github.com/gkatzioura
> >> > > > > > > > >> > > > >
> >> > > > > > > > >> > > > >
> >> > > > > > > > >> > > > > On Sun, 24 Nov 2019 at 21:11, Emmanouil
> >> Gkatziouras
> >> > <
> >> > > > > > > > >> > > > gkatzioura@gmail.com>
> >> > > > > > > > >> > > > > wrote:
> >> > > > > > > > >> > > > >
> >> > > > > > > > >> > > > > > Hi Saikat!
> >> > > > > > > > >> > > > > >
> >> > > > > > > > >> > > > > > I just rebased with the flink branch.
> >> > Unfortunately
> >> > > I
> >> > > > do
> >> > > > > > not
> >> > > > > > > > >> have
> >> > > > > > > > >> > > read
> >> > > > > > > > >> > > > > > access to the team city link you provided and
> >> thus
> >> > > > > > evaluate
> >> > > > > > > > the
> >> > > > > > > > >> > tests
> >> > > > > > > > >> > > > for
> >> > > > > > > > >> > > > > > my pull request.
> >> > > > > > > > >> > > > > > I guess it has to do with ignite extensions
> >> being
> >> > > new.
> >> > > > > > > > >> > > > > > My username is gkatzioura and email the one I
> >> am
> >> > > using
> >> > > > > on
> >> > > > > > > this
> >> > > > > > > > >> > email
> >> > > > > > > > >> > > > > >
> >> > > > > > > > >> > > > > > Thank you for your time.
> >> > > > > > > > >> > > > > >
> >> > > > > > > > >> > > > > >
> >> > > > > > > > >> > > > > > *Emmanouil Gkatziouras*
> >> > > > > > > > >> > > > > > https://egkatzioura.com/ |
> >> > > > > > > > >> > > > > >
> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > > > > > > >> > > > > > https://github.com/gkatzioura
> >> > > > > > > > >> > > > > >
> >> > > > > > > > >> > > > > >
> >> > > > > > > > >> > > > > > On Sun, 24 Nov 2019 at 18:03, Saikat Maitra <
> >> > > > > > > > >> > saikat.maitra@gmail.com
> >> > > > > > > > >> > > >
> >> > > > > > > > >> > > > > > wrote:
> >> > > > > > > > >> > > > > >
> >> > > > > > > > >> > > > > >> Hi Emmanouil,
> >> > > > > > > > >> > > > > >>
> >> > > > > > > > >> > > > > >> The latest build on teamcity has passed on
> >> Flink
> >> > > pull
> >> > > > > > > > request.
> >> > > > > > > > >> > > > > >>
> >> > > > > > > > >> > > > > >>
> >> > > > > > > > >> > > > > >>
> >> > > > > > > > >> > > > >
> >> > > > > > > > >> > > >
> >> > > > > > > > >> > >
> >> > > > > > > > >> >
> >> > > > > > > > >>
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://ci.ignite.apache.org/viewLog.html?buildId=4788928&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> >> > > > > > > > >> > > > > >>
> >> > > > > > > > >> > > > > >> you should be able to take the changes I
> made
> >> in
> >> > my
> >> > > > PR
> >> > > > > > and
> >> > > > > > > > run
> >> > > > > > > > >> > build
> >> > > > > > > > >> > > > on
> >> > > > > > > > >> > > > > >> your pull request.
> >> > > > > > > > >> > > > > >>
> >> > > > > > > > >> > > > > >> Please let me know if you have any
> questions.
> >> > > > > > > > >> > > > > >>
> >> > > > > > > > >> > > > > >> Regards,
> >> > > > > > > > >> > > > > >> Saikat
> >> > > > > > > > >> > > > > >>
> >> > > > > > > > >> > > > > >> On Sun, Nov 24, 2019 at 10:44 AM Saikat
> >> Maitra <
> >> > > > > > > > >> > > > saikat.maitra@gmail.com
> >> > > > > > > > >> > > > > >
> >> > > > > > > > >> > > > > >> wrote:
> >> > > > > > > > >> > > > > >>
> >> > > > > > > > >> > > > > >> > Hi Emmanouil,
> >> > > > > > > > >> > > > > >> >
> >> > > > > > > > >> > > > > >> > I have fixed the Flink module testsuites
> and
> >> > have
> >> > > > > setup
> >> > > > > > > new
> >> > > > > > > > >> > > project
> >> > > > > > > > >> > > > in
> >> > > > > > > > >> > > > > >> > teamcity.
> >> > > > > > > > >> > > > > >> >
> >> > > > > > > > >> > > > > >> >
> >> > > > > > > > >> > > > > >> >
> >> > > > > > > > >> > > > > >>
> >> > > > > > > > >> > > > >
> >> > > > > > > > >> > > >
> >> > > > > > > > >> > >
> >> > > > > > > > >> >
> >> > > > > > > > >>
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview
> >> > > > > > > > >> > > > > >> >
> >> > > > > > > > >> > > > > >> > I am looking into setting up the build.
> >> > > > > > > > >> > > > > >> >
> >> > > > > > > > >> > > > > >> > Regards,
> >> > > > > > > > >> > > > > >> > Saikat
> >> > > > > > > > >> > > > > >> >
> >> > > > > > > > >> > > > > >> > On Sat, Nov 23, 2019 at 9:52 AM Saikat
> >> Maitra <
> >> > > > > > > > >> > > > > saikat.maitra@gmail.com>
> >> > > > > > > > >> > > > > >> > wrote:
> >> > > > > > > > >> > > > > >> >
> >> > > > > > > > >> > > > > >> >> Hi Emmanouil,
> >> > > > > > > > >> > > > > >> >>
> >> > > > > > > > >> > > > > >> >> Thank you for your email. Yes, the plan
> is
> >> > once
> >> > > > the
> >> > > > > > > Flink
> >> > > > > > > > >> PR is
> >> > > > > > > > >> > > > > merged
> >> > > > > > > > >> > > > > >> it
> >> > > > > > > > >> > > > > >> >> will provide the Licence, parent POMs etc
> >> and
> >> > > you
> >> > > > > can
> >> > > > > > > > rebase
> >> > > > > > > > >> > from
> >> > > > > > > > >> > > > > >> master
> >> > > > > > > > >> > > > > >> >> branch and apply the changes on top of
> it.
> >> > > > > > > > >> > > > > >> >>
> >> > > > > > > > >> > > > > >> >> I am getting some issues with test
> >> failures in
> >> > > > local
> >> > > > > > > with
> >> > > > > > > > >> > > > > >> >> GridTestProperties as the test.properties
> >> is
> >> > not
> >> > > > > > present
> >> > > > > > > > in
> >> > > > > > > > >> > this
> >> > > > > > > > >> > > > new
> >> > > > > > > > >> > > > > >> >> project but should be available from
> >> > > dependencies.
> >> > > > > > > > >> > > > > >> >>
> >> > > > > > > > >> > > > > >> >> Once I address this issue, I will go
> ahead
> >> and
> >> > > > merge
> >> > > > > > the
> >> > > > > > > > >> > changes
> >> > > > > > > > >> > > > and
> >> > > > > > > > >> > > > > >> then
> >> > > > > > > > >> > > > > >> >> we can take it from there.
> >> > > > > > > > >> > > > > >> >>
> >> > > > > > > > >> > > > > >> >> Regards,
> >> > > > > > > > >> > > > > >> >> Saikat
> >> > > > > > > > >> > > > > >> >>
> >> > > > > > > > >> > > > > >> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil
> >> > > > > Gkatziouras
> >> > > > > > <
> >> > > > > > > > >> > > > > >> >> gkatzioura@gmail.com> wrote:
> >> > > > > > > > >> > > > > >> >>
> >> > > > > > > > >> > > > > >> >>> Hi all,
> >> > > > > > > > >> > > > > >> >>>
> >> > > > > > > > >> > > > > >> >>> I made my first pull request [1]. Since
> >> this
> >> > > > > project
> >> > > > > > is
> >> > > > > > > > >> brand
> >> > > > > > > > >> > > new
> >> > > > > > > > >> > > > > (no
> >> > > > > > > > >> > > > > >> >>> parent poms, licensing), I reckoned it
> was
> >> > best
> >> > > > to
> >> > > > > > use
> >> > > > > > > > >> > Saikat's
> >> > > > > > > > >> > > > > >> branch on
> >> > > > > > > > >> > > > > >> >>> Flink.
> >> > > > > > > > >> > > > > >> >>> I suppose the Flink branch will be
> merged
> >> > > first.
> >> > > > If
> >> > > > > > not
> >> > > > > > > > >> please
> >> > > > > > > > >> > > > give
> >> > > > > > > > >> > > > > me
> >> > > > > > > > >> > > > > >> >>> guidelines on how I should proceed next.
> >> > > > > > > > >> > > > > >> >>>
> >> > > > > > > > >> > > > > >> >>> Kind regards
> >> > > > > > > > >> > > > > >> >>> Emmanouil
> >> > > > > > > > >> > > > > >> >>>
> >> > > > > > > > >> > > > > >> >>> [1]
> >> > > > > > https://github.com/apache/ignite-extensions/pull/2
> >> > > > > > > > >> > > > > >> >>>
> >> > > > > > > > >> > > > > >> >>> *Emmanouil Gkatziouras*
> >> > > > > > > > >> > > > > >> >>> https://egkatzioura.com/ |
> >> > > > > > > > >> > > > > >> >>>
> >> > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > > > > > > >> > > > > >> >>> https://github.com/gkatzioura
> >> > > > > > > > >> > > > > >> >>>
> >> > > > > > > > >> > > > > >> >>>
> >> > > > > > > > >> > > > > >> >>> On Fri, 22 Nov 2019 at 20:55, Denis
> Magda
> >> <
> >> > > > > > > > >> dmagda@apache.org>
> >> > > > > > > > >> > > > > wrote:
> >> > > > > > > > >> > > > > >> >>>
> >> > > > > > > > >> > > > > >> >>> > Awesome, ping us whenever you're
> ready!
> >> > > > > > > > >> > > > > >> >>> >
> >> > > > > > > > >> > > > > >> >>> > -
> >> > > > > > > > >> > > > > >> >>> > Denis
> >> > > > > > > > >> > > > > >> >>> >
> >> > > > > > > > >> > > > > >> >>> >
> >> > > > > > > > >> > > > > >> >>> > On Fri, Nov 22, 2019 at 12:52 PM
> >> Emmanouil
> >> > > > > > > Gkatziouras
> >> > > > > > > > <
> >> > > > > > > > >> > > > > >> >>> > gkatzioura@gmail.com>
> >> > > > > > > > >> > > > > >> >>> > wrote:
> >> > > > > > > > >> > > > > >> >>> >
> >> > > > > > > > >> > > > > >> >>> > > Hi all!
> >> > > > > > > > >> > > > > >> >>> > >
> >> > > > > > > > >> > > > > >> >>> > > I am sorry for being late on that. I
> >> was
> >> > > > trying
> >> > > > > > to
> >> > > > > > > > >> > refactor
> >> > > > > > > > >> > > > the
> >> > > > > > > > >> > > > > >> test
> >> > > > > > > > >> > > > > >> >>> in
> >> > > > > > > > >> > > > > >> >>> > > order not to be in need of any
> >> external
> >> > > tools
> >> > > > > or
> >> > > > > > > > >> spinning
> >> > > > > > > > >> > > up a
> >> > > > > > > > >> > > > > >> >>> server.
> >> > > > > > > > >> > > > > >> >>> > > I did forked the new repo and indeed
> >> my
> >> > > > changes
> >> > > > > > > > there,
> >> > > > > > > > >> so
> >> > > > > > > > >> > a
> >> > > > > > > > >> > > > pull
> >> > > > > > > > >> > > > > >> >>> request
> >> > > > > > > > >> > > > > >> >>> > is
> >> > > > > > > > >> > > > > >> >>> > > a matter of time!
> >> > > > > > > > >> > > > > >> >>> > >
> >> > > > > > > > >> > > > > >> >>> > > Kind regards
> >> > > > > > > > >> > > > > >> >>> > >
> >> > > > > > > > >> > > > > >> >>> > > *Emmanouil Gkatziouras*
> >> > > > > > > > >> > > > > >> >>> > > https://egkatzioura.com/ |
> >> > > > > > > > >> > > > > >> >>> > >
> >> > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > > > > > > >> > > > > >> >>> > > https://github.com/gkatzioura
> >> > > > > > > > >> > > > > >> >>> > >
> >> > > > > > > > >> > > > > >> >>> > >
> >> > > > > > > > >> > > > > >> >>> > > On Fri, 22 Nov 2019 at 20:45, Denis
> >> > Magda <
> >> > > > > > > > >> > > dmagda@apache.org>
> >> > > > > > > > >> > > > > >> wrote:
> >> > > > > > > > >> > > > > >> >>> > >
> >> > > > > > > > >> > > > > >> >>> > > > Hi Emmanouil,
> >> > > > > > > > >> > > > > >> >>> > > >
> >> > > > > > > > >> > > > > >> >>> > > > Do you have any questions or need
> >> any
> >> > > > support
> >> > > > > > > from
> >> > > > > > > > >> the
> >> > > > > > > > >> > > > > >> community?
> >> > > > > > > > >> > > > > >> >>> > > >
> >> > > > > > > > >> > > > > >> >>> > > > -
> >> > > > > > > > >> > > > > >> >>> > > > Denis
> >> > > > > > > > >> > > > > >> >>> > > >
> >> > > > > > > > >> > > > > >> >>> > > >
> >> > > > > > > > >> > > > > >> >>> > > > On Sun, Nov 10, 2019 at 3:07 PM
> >> Saikat
> >> > > > > Maitra <
> >> > > > > > > > >> > > > > >> >>> saikat.maitra@gmail.com
> >> > > > > > > > >> > > > > >> >>> > >
> >> > > > > > > > >> > > > > >> >>> > > > wrote:
> >> > > > > > > > >> > > > > >> >>> > > >
> >> > > > > > > > >> > > > > >> >>> > > >> Hi Emmanouil,
> >> > > > > > > > >> > > > > >> >>> > > >>
> >> > > > > > > > >> > > > > >> >>> > > >> Can you please take a looks at
> dev
> >> > > utils,
> >> > > > if
> >> > > > > > > this
> >> > > > > > > > is
> >> > > > > > > > >> > > > > something
> >> > > > > > > > >> > > > > >> >>> you are
> >> > > > > > > > >> > > > > >> >>> > > >> looking for
> >> > > > > > > > >> > > > > >> >>> > > >>
> >> > > > > > > > >> > > > >
> >> > > > > > >
> >> https://github.com/apache/ignite/tree/master/modules/dev-utils
> >> > > > > > > > >> > > > > >> ?
> >> > > > > > > > >> > > > > >> >>> > > >>
> >> > > > > > > > >> > > > > >> >>> > > >> IMO, if you can release Pub/Sub
> >> server
> >> > > in
> >> > > > > > maven
> >> > > > > > > > and
> >> > > > > > > > >> > then
> >> > > > > > > > >> > > > use
> >> > > > > > > > >> > > > > >> it as
> >> > > > > > > > >> > > > > >> >>> > > >> dependency, that would be great.
> >> > > > > > > > >> > > > > >> >>> > > >>
> >> > > > > > > > >> > > > > >> >>> > > >> Regards,
> >> > > > > > > > >> > > > > >> >>> > > >> Saikat
> >> > > > > > > > >> > > > > >> >>> > > >>
> >> > > > > > > > >> > > > > >> >>> > > >> On Sun, Nov 10, 2019 at 5:00 PM
> >> Saikat
> >> > > > > Maitra
> >> > > > > > <
> >> > > > > > > > >> > > > > >> >>> > saikat.maitra@gmail.com>
> >> > > > > > > > >> > > > > >> >>> > > >> wrote:
> >> > > > > > > > >> > > > > >> >>> > > >>
> >> > > > > > > > >> > > > > >> >>> > > >> > Hi Emmanouil,
> >> > > > > > > > >> > > > > >> >>> > > >> >
> >> > > > > > > > >> > > > > >> >>> > > >> > The master branch has been
> >> > > initialized.
> >> > > > I
> >> > > > > > have
> >> > > > > > > > >> > raised a
> >> > > > > > > > >> > > > PR
> >> > > > > > > > >> > > > > to
> >> > > > > > > > >> > > > > >> >>> > migrate
> >> > > > > > > > >> > > > > >> >>> > > >> > Flink module to
> >> ignite-extensions.
> >> > > > > > > > >> > > > > >> >>> > > >> >
> >> > > > > > > > >> > > > > >> >>> > > >> > PR
> >> > > > > > > > >> > https://github.com/apache/ignite-extensions/pull/1
> >> > > > > > > > >> > > > > >> >>> > > >> > Jira :
> >> > > > > > > > >> > > >
> https://issues.apache.org/jira/browse/IGNITE-12356
> >> > > > > > > > >> > > > > >> >>> > > >> >
> >> > > > > > > > >> > > > > >> >>> > > >> > Regards,
> >> > > > > > > > >> > > > > >> >>> > > >> > Saikat
> >> > > > > > > > >> > > > > >> >>> > > >> >
> >> > > > > > > > >> > > > > >> >>> > > >> >
> >> > > > > > > > >> > > > > >> >>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM
> >> > > > Emmanouil
> >> > > > > > > > >> > Gkatziouras <
> >> > > > > > > > >> > > > > >> >>> > > >> > gkatzioura@gmail.com> wrote:
> >> > > > > > > > >> > > > > >> >>> > > >> >
> >> > > > > > > > >> > > > > >> >>> > > >> >> Hi all!
> >> > > > > > > > >> > > > > >> >>> > > >> >>
> >> > > > > > > > >> > > > > >> >>> > > >> >> This is an update. The
> >> integration
> >> > is
> >> > > > > > tested
> >> > > > > > > > with
> >> > > > > > > > >> > > mocks
> >> > > > > > > > >> > > > > >> based
> >> > > > > > > > >> > > > > >> >>> on
> >> > > > > > > > >> > > > > >> >>> > the
> >> > > > > > > > >> > > > > >> >>> > > >> >> original Ignite repo [1].
> >> > > > > > > > >> > > > > >> >>> > > >> >> I am currently waiting for the
> >> > first
> >> > > > > > commits
> >> > > > > > > on
> >> > > > > > > > >> the
> >> > > > > > > > >> > > new
> >> > > > > > > > >> > > > > >> >>> integration
> >> > > > > > > > >> > > > > >> >>> > > >> >> project
> >> > > > > > > > >> > > > > >> >>> > > >> >> to take place in order to
> >> create my
> >> > > > pull
> >> > > > > > > > request
> >> > > > > > > > >> > > > > accordingly
> >> > > > > > > > >> > > > > >> >>> to the
> >> > > > > > > > >> > > > > >> >>> > > >> >> initial
> >> > > > > > > > >> > > > > >> >>> > > >> >> commits.
> >> > > > > > > > >> > > > > >> >>> > > >> >> In order to simulate the
> Pub/Sub
> >> > > server
> >> > > > > in
> >> > > > > > a
> >> > > > > > > > >> test I
> >> > > > > > > > >> > > > used a
> >> > > > > > > > >> > > > > >> test
> >> > > > > > > > >> > > > > >> >>> > > server
> >> > > > > > > > >> > > > > >> >>> > > >> >> implementation [2].
> >> Unfortunately
> >> > > they
> >> > > > > > don't
> >> > > > > > > > have
> >> > > > > > > > >> > this
> >> > > > > > > > >> > > > > util
> >> > > > > > > > >> > > > > >> >>> > uploaded
> >> > > > > > > > >> > > > > >> >>> > > >> in a
> >> > > > > > > > >> > > > > >> >>> > > >> >> maven repo, thus it needs to
> be
> >> > > hosted
> >> > > > > on a
> >> > > > > > > > maven
> >> > > > > > > > >> > repo
> >> > > > > > > > >> > > > > >> (Apache
> >> > > > > > > > >> > > > > >> >>> > > License
> >> > > > > > > > >> > > > > >> >>> > > >> >> 2.0). Is there a repo for
> >> utilities
> >> > > > that
> >> > > > > > > Ignite
> >> > > > > > > > >> uses
> >> > > > > > > > >> > > for
> >> > > > > > > > >> > > > > the
> >> > > > > > > > >> > > > > >> >>> > > >> test/build?
> >> > > > > > > > >> > > > > >> >>> > > >> >> I have also created a ticket
> [3]
> >> > > > > > > > >> > > > > >> >>> > > >> >>
> >> > > > > > > > >> > > > > >> >>> > > >> >> Kind regards
> >> > > > > > > > >> > > > > >> >>> > > >> >>
> >> > > > > > > > >> > > > > >> >>> > > >> >> [1]
> >> > > > > > > > >> > > > > >> >>> >
> >> > > > > > > > >> > >
> >> > > > > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> >> > > > > > > > >> > > > > >> >>> > > >> >> [2]
> >> > > > > > > > >> > > > > >> >>>
> >> > > > > > > > >>
> >> > https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> >> > > > > > > > >> > > > > >> >>> > > >> >> [3]
> >> > > > > > > > >> > > https://issues.apache.org/jira/browse/IGNITE-12262
> >> > > > > > > > >> > > > > >> >>> > > >> >>
> >> > > > > > > > >> > > > > >> >>> > > >> >> *Emmanouil Gkatziouras*
> >> > > > > > > > >> > > > > >> >>> > > >> >> https://egkatzioura.com/ |
> >> > > > > > > > >> > > > > >> >>> > > >> >>
> >> > > > > > > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > > > > > > >> > > > > >> >>> > > >> >> https://github.com/gkatzioura
> >> > > > > > > > >> > > > > >> >>> > > >> >>
> >> > > > > > > > >> > > > > >> >>> > > >> >>
> >> > > > > > > > >> > > > > >> >>> > > >> >> On Fri, 25 Oct 2019 at 03:35,
> >> > Saikat
> >> > > > > > Maitra <
> >> > > > > > > > >> > > > > >> >>> > saikat.maitra@gmail.com
> >> > > > > > > > >> > > > > >> >>> > > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> wrote:
> >> > > > > > > > >> > > > > >> >>> > > >> >>
> >> > > > > > > > >> > > > > >> >>> > > >> >> > Hello Ilya,
> >> > > > > > > > >> > > > > >> >>> > > >> >> >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > Thank you for your email.
> Yes,
> >> > > before
> >> > > > > we
> >> > > > > > > > remove
> >> > > > > > > > >> > any
> >> > > > > > > > >> > > > > >> >>> integration
> >> > > > > > > > >> > > > > >> >>> > we
> >> > > > > > > > >> > > > > >> >>> > > >> will
> >> > > > > > > > >> > > > > >> >>> > > >> >> > move them to another
> >> repository
> >> > and
> >> > > > > > > release.
> >> > > > > > > > >> > > > > >> >>> > > >> >> >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > We are thinking we will
> start
> >> > with
> >> > > > > > Pub/Sub
> >> > > > > > > > >> > Streamer
> >> > > > > > > > >> > > > > >> >>> > implementation
> >> > > > > > > > >> > > > > >> >>> > > >> and
> >> > > > > > > > >> > > > > >> >>> > > >> >> > continue to move other
> >> > integration
> >> > > as
> >> > > > > > > > >> mentioned in
> >> > > > > > > > >> > > the
> >> > > > > > > > >> > > > > >> >>> following
> >> > > > > > > > >> > > > > >> >>> > > doc.
> >> > > > > > > > >> > > > > >> >>> > > >> >> >
> >> > > > > > > > >> > > > > >> >>> > > >> >> >
> >> > > > > > > > >> > > > > >> >>> > > >> >> >
> >> > > > > > > > >> > > > > >> >>> > > >> >>
> >> > > > > > > > >> > > > > >> >>> > > >>
> >> > > > > > > > >> > > > > >> >>> > >
> >> > > > > > > > >> > > > > >> >>> >
> >> > > > > > > > >> > > > > >> >>>
> >> > > > > > > > >> > > > > >>
> >> > > > > > > > >> > > > >
> >> > > > > > > > >> > > >
> >> > > > > > > > >> > >
> >> > > > > > > > >> >
> >> > > > > > > > >>
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >> > > > > > > > >> > > > > >> >>> > > >> >> >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > Regards,
> >> > > > > > > > >> > > > > >> >>> > > >> >> > Saikat
> >> > > > > > > > >> > > > > >> >>> > > >> >> >
> >> > > > > > > > >> > > > > >> >>> > > >> >> >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > On Thu, Oct 24, 2019 at
> 11:00
> >> AM
> >> > > Ilya
> >> > > > > > > > >> Kasnacheev <
> >> > > > > > > > >> > > > > >> >>> > > >> >> > ilya.kasnacheev@gmail.com>
> >> > > > > > > > >> > > > > >> >>> > > >> >> > wrote:
> >> > > > > > > > >> > > > > >> >>> > > >> >> >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > Hello!
> >> > > > > > > > >> > > > > >> >>> > > >> >> > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > My take on this: we want
> to
> >> > > offload
> >> > > > > > some
> >> > > > > > > of
> >> > > > > > > > >> > > > > >> integrations,
> >> > > > > > > > >> > > > > >> >>> but
> >> > > > > > > > >> > > > > >> >>> > > >> please
> >> > > > > > > > >> > > > > >> >>> > > >> >> > commit
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > and release them to
> another
> >> > > project
> >> > > > > > > first,
> >> > > > > > > > >> and
> >> > > > > > > > >> > > only
> >> > > > > > > > >> > > > > >> then we
> >> > > > > > > > >> > > > > >> >>> > will
> >> > > > > > > > >> > > > > >> >>> > > >> >> decide
> >> > > > > > > > >> > > > > >> >>> > > >> >> > to
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > remove them from our
> >> > repository.
> >> > > > > > > > >> > > > > >> >>> > > >> >> > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > Please also make sure they
> >> are
> >> > > > > > available
> >> > > > > > > > with
> >> > > > > > > > >> > same
> >> > > > > > > > >> > > > > >> artifact
> >> > > > > > > > >> > > > > >> >>> > name
> >> > > > > > > > >> > > > > >> >>> > > >> and
> >> > > > > > > > >> > > > > >> >>> > > >> >> API.
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > If this has to be changed,
> >> > let's
> >> > > > wait
> >> > > > > > for
> >> > > > > > > > AI
> >> > > > > > > > >> 3.0
> >> > > > > > > > >> > > > > >> >>> > > >> >> > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > Regards,
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > --
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > Ilya Kasnacheev
> >> > > > > > > > >> > > > > >> >>> > > >> >> > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > ср, 23 окт. 2019 г. в
> 03:23,
> >> > > Saikat
> >> > > > > > > Maitra
> >> > > > > > > > <
> >> > > > > > > > >> > > > > >> >>> > > >> saikat.maitra@gmail.com>:
> >> > > > > > > > >> > > > > >> >>> > > >> >> > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > Hello Emmanouil,
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > As discussed earlier, I
> >> > > discussed
> >> > > > > > with
> >> > > > > > > > >> Apache
> >> > > > > > > > >> > > > Bahir
> >> > > > > > > > >> > > > > >> >>> community
> >> > > > > > > > >> > > > > >> >>> > > and
> >> > > > > > > > >> > > > > >> >>> > > >> >> they
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > are
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > interested to have
> Apache
> >> > > Ignite
> >> > > > > > > > >> extensions as
> >> > > > > > > > >> > > > part
> >> > > > > > > > >> > > > > of
> >> > > > > > > > >> > > > > >> >>> Apache
> >> > > > > > > > >> > > > > >> >>> > > >> Bahir
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > project.
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > Can you please share
> >> Pub/Sub
> >> > > > > streamer
> >> > > > > > > > >> > > > implementation
> >> > > > > > > > >> > > > > >> >>> details
> >> > > > > > > > >> > > > > >> >>> > > with
> >> > > > > > > > >> > > > > >> >>> > > >> >> > Apache
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > Bahir community and
> >> request
> >> > for
> >> > > > > > > feedback.
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> https://bahir.apache.org/
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > I have also requested
> for
> >> > > > > contributor
> >> > > > > > > > >> access
> >> > > > > > > > >> > in
> >> > > > > > > > >> > > > Jira
> >> > > > > > > > >> > > > > >> for
> >> > > > > > > > >> > > > > >> >>> > Apache
> >> > > > > > > > >> > > > > >> >>> > > >> >> Bahir
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > project so that I can
> >> create
> >> > > > issues
> >> > > > > > and
> >> > > > > > > > >> assign
> >> > > > > > > > >> > > to
> >> > > > > > > > >> > > > > >> myself.
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > I can help with code
> >> reviews
> >> > as
> >> > > > > well.
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > Here is the email thread
> >> for
> >> > > > > > reference
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> >> > > > > > > > >> > > > > >> >>> > >
> >> > > > > > > > >> > > >
> >> > > > > >
> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > Regards,
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > Saikat
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > On Mon, Oct 21, 2019 at
> >> 7:54
> >> > PM
> >> > > > > > Saikat
> >> > > > > > > > >> Maitra
> >> > > > > > > > >> > <
> >> > > > > > > > >> > > > > >> >>> > > >> >> saikat.maitra@gmail.com
> >> > > > > > > > >> > > > > >> >>> > > >> >> > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > wrote:
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > > Hello,
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > > As discussed I have
> >> created
> >> > > > > > following
> >> > > > > > > > >> > > discussion
> >> > > > > > > > >> > > > > >> >>> threads on
> >> > > > > > > > >> > > > > >> >>> > > our
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > migration
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > > proposals for Apache
> >> Ignite
> >> > > > > > > extensions:
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> >
> >> > > > > > > > >> > > > > >> >>> > > >> >>
> >> > > > > > > > >> > > > > >> >>> > > >>
> >> > > > > > > > >> > > > > >> >>> > >
> >> > > > > > > > >> > > > > >> >>> >
> >> > > > > > > > >> > > > > >> >>>
> >> > > > > > > > >> > > > > >>
> >> > > > > > > > >> > > > >
> >> > > > > > > > >> > > >
> >> > > > > > > > >> > >
> >> > > > > > > > >> >
> >> > > > > > > > >>
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> >> > > > > > > > >> > > > > >> >>> > > >>
> >> > > > > > > > >> > > > > >>
> >> > > > > > > > >>
> >> > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > > I will share update
> as I
> >> > hear
> >> > > > > more
> >> > > > > > > > >> > > information.
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > > Regards,
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > > Saikat
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > > On Fri, Oct 18, 2019
> at
> >> > 9:03
> >> > > PM
> >> > > > > > > Saikat
> >> > > > > > > > >> > Maitra
> >> > > > > > > > >> > > <
> >> > > > > > > > >> > > > > >> >>> > > >> >> > saikat.maitra@gmail.com
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > > wrote:
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >> Hello Denis,
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >> Sure, sounds good. I
> >> will
> >> > > > > create a
> >> > > > > > > > >> separate
> >> > > > > > > > >> > > > > >> discussion
> >> > > > > > > > >> > > > > >> >>> > > thread
> >> > > > > > > > >> > > > > >> >>> > > >> to
> >> > > > > > > > >> > > > > >> >>> > > >> >> get
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >> community feedback on
> >> > > whether
> >> > > > we
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >> should join the Bahir
> >> or
> >> > > > > kick-off
> >> > > > > > > > >> "Ignite
> >> > > > > > > > >> > > > > >> Extensions"
> >> > > > > > > > >> > > > > >> >>> > > project.
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >> Regards,
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >> Saikat
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >> On Thu, Oct 17, 2019
> at
> >> > 2:21
> >> > > > PM
> >> > > > > > > Denis
> >> > > > > > > > >> > Magda <
> >> > > > > > > > >> > > > > >> >>> > > >> dmagda@apache.org>
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > wrote:
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> Folks,
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> The concept of
> Apache
> >> > Bahir
> >> > > > is
> >> > > > > > > > >> precisely
> >> > > > > > > > >> > > what
> >> > > > > > > > >> > > > we
> >> > > > > > > > >> > > > > >> >>> need!
> >> > > > > > > > >> > > > > >> >>> > > >> Saikat,
> >> > > > > > > > >> > > > > >> >>> > > >> >> > thanks
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > for
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> doing the research.
> >> Why I
> >> > > > > believe
> >> > > > > > > > the *
> >> > > > > > > > >> > > idea*
> >> > > > > > > > >> > > > > >> fits us
> >> > > > > > > > >> > > > > >> >>> > well:
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    - All
> integrations
> >> can
> >> > > be
> >> > > > > > stored
> >> > > > > > > > in
> >> > > > > > > > >> > > > separate
> >> > > > > > > > >> > > > > >> >>> Github
> >> > > > > > > > >> > > > > >> >>> > > >> >> repositories
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > and
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    have their dev
> >> > > lifecycles.
> >> > > > > > We've
> >> > > > > > > > not
> >> > > > > > > > >> > > > obliged
> >> > > > > > > > >> > > > > to
> >> > > > > > > > >> > > > > >> >>> couple
> >> > > > > > > > >> > > > > >> >>> > > all
> >> > > > > > > > >> > > > > >> >>> > > >> >> the
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> integrations
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    in a single repo.
> >> For
> >> > > > > > instance,
> >> > > > > > > > >> Spark
> >> > > > > > > > >> > can
> >> > > > > > > > >> > > > be
> >> > > > > > > > >> > > > > >> >>> located
> >> > > > > > > > >> > > > > >> >>> > in
> >> > > > > > > > >> > > > > >> >>> > > a
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > dedicated
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> repo
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    while streaming
> >> > > > integrations
> >> > > > > > > might
> >> > > > > > > > >> be
> >> > > > > > > > >> > in
> >> > > > > > > > >> > > a
> >> > > > > > > > >> > > > > >> single
> >> > > > > > > > >> > > > > >> >>> one.
> >> > > > > > > > >> > > > > >> >>> > > >> It's
> >> > > > > > > > >> > > > > >> >>> > > >> >> up
> >> > > > > > > > >> > > > > >> >>> > > >> >> > to
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > us.
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    - All the
> >> repositories
> >> > > and
> >> > > > > > > > >> integrations
> >> > > > > > > > >> > > > > belong
> >> > > > > > > > >> > > > > >> to
> >> > > > > > > > >> > > > > >> >>> ASF.
> >> > > > > > > > >> > > > > >> >>> > > >> We're
> >> > > > > > > > >> > > > > >> >>> > > >> >> not
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> dumping
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    them on Github
> but
> >> > > rather
> >> > > > > keep
> >> > > > > > > > >> > supporting
> >> > > > > > > > >> > > > and
> >> > > > > > > > >> > > > > >> >>> > developing
> >> > > > > > > > >> > > > > >> >>> > > >> in
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> accordance with
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    ASF vision and
> >> > > practices.
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    - There is a way
> to
> >> > > reward
> >> > > > > > > > >> contributors
> >> > > > > > > > >> > > via
> >> > > > > > > > >> > > > > >> >>> > > committership
> >> > > > > > > > >> > > > > >> >>> > > >> and
> >> > > > > > > > >> > > > > >> >>> > > >> >> > PMC
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    membership. You
> >> won't
> >> > > get
> >> > > > > this
> >> > > > > > > if
> >> > > > > > > > a
> >> > > > > > > > >> > > project
> >> > > > > > > > >> > > > > is
> >> > > > > > > > >> > > > > >> >>> just
> >> > > > > > > > >> > > > > >> >>> > one
> >> > > > > > > > >> > > > > >> >>> > > of
> >> > > > > > > > >> > > > > >> >>> > > >> >> the
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> millions of
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    Github projects.
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> Saikat, as Ignite
> PMC
> >> > > member,
> >> > > > > > could
> >> > > > > > > > you
> >> > > > > > > > >> > > please
> >> > > > > > > > >> > > > > >> >>> kick-off a
> >> > > > > > > > >> > > > > >> >>> > > >> >> separate
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> dev-list
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> discussion to
> involve
> >> > more
> >> > > > > > > community
> >> > > > > > > > >> > members
> >> > > > > > > > >> > > > and
> >> > > > > > > > >> > > > > >> >>> > referring
> >> > > > > > > > >> > > > > >> >>> > > to
> >> > > > > > > > >> > > > > >> >>> > > >> >> this
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> thread?
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> I think the primary
> >> > > question
> >> > > > > the
> >> > > > > > > > >> community
> >> > > > > > > > >> > > > needs
> >> > > > > > > > >> > > > > >> to
> >> > > > > > > > >> > > > > >> >>> > answer
> >> > > > > > > > >> > > > > >> >>> > > >> >> whether
> >> > > > > > > > >> > > > > >> >>> > > >> >> > we
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> should join the
> Bahir
> >> or
> >> > > > > kick-off
> >> > > > > > > > >> "Ignite
> >> > > > > > > > >> > > > > >> Extensions"
> >> > > > > > > > >> > > > > >> >>> > > >> project?
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> -
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> Denis
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> On Thu, Oct 17, 2019
> >> at
> >> > > 2:54
> >> > > > AM
> >> > > > > > > > Alexey
> >> > > > > > > > >> > > > Zinoviev
> >> > > > > > > > >> > > > > <
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > zaleslaw.sin@gmail.com>
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> wrote:
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > Maybe we could
> move
> >> all
> >> > > our
> >> > > > > > > > Streaming
> >> > > > > > > > >> > > > > >> Integrations
> >> > > > > > > > >> > > > > >> >>> > there,
> >> > > > > > > > >> > > > > >> >>> > > >> but
> >> > > > > > > > >> > > > > >> >>> > > >> >> > what
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > is
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> about
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > maintaining and
> >> > committer
> >> > > > > > > > >> permissions to
> >> > > > > > > > >> > > the
> >> > > > > > > > >> > > > > new
> >> > > > > > > > >> > > > > >> >>> > > >> repositories?
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > I see the list of
> >> the
> >> > > > > > committers
> >> > > > > > > > and
> >> > > > > > > > >> PMC
> >> > > > > > > > >> > > > > members
> >> > > > > > > > >> > > > > >> >>> there
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> >> > > > > > > > >> > > https://bahir.apache.org/community-members/
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > Could somebody
> from
> >> > > Ignite
> >> > > > > > > > community
> >> > > > > > > > >> be
> >> > > > > > > > >> > > > added
> >> > > > > > > > >> > > > > to
> >> > > > > > > > >> > > > > >> >>> this
> >> > > > > > > > >> > > > > >> >>> > > list
> >> > > > > > > > >> > > > > >> >>> > > >> as
> >> > > > > > > > >> > > > > >> >>> > > >> >> a
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > PMC/committer to
> >> > maintain
> >> > > > > > Ignite
> >> > > > > > > > >> > > > integrations?
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > If yes, I'd happy
> to
> >> > join
> >> > > > > this
> >> > > > > > > > >> project
> >> > > > > > > > >> > and
> >> > > > > > > > >> > > > > >> >>> collaborate
> >> > > > > > > > >> > > > > >> >>> > > with
> >> > > > > > > > >> > > > > >> >>> > > >> >> these
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > guys
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > together
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > If no, and we
> should
> >> > > start
> >> > > > > from
> >> > > > > > > the
> >> > > > > > > > >> zero
> >> > > > > > > > >> > > > level
> >> > > > > > > > >> > > > > >> with
> >> > > > > > > > >> > > > > >> >>> > > >> external
> >> > > > > > > > >> > > > > >> >>> > > >> >> PRs
> >> > > > > > > > >> > > > > >> >>> > > >> >> > -
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> hmmm,
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > it's better to
> have
> >> our
> >> > > own
> >> > > > > > > > external
> >> > > > > > > > >> > > > > repository
> >> > > > > > > > >> > > > > >> >>> with
> >> > > > > > > > >> > > > > >> >>> > > >> >> ApacheBahirr
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> ideology.
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > Also, I agree,
> that
> >> all
> >> > > > > > > connectors
> >> > > > > > > > >> could
> >> > > > > > > > >> > > be
> >> > > > > > > > >> > > > > >> moved
> >> > > > > > > > >> > > > > >> >>> there
> >> > > > > > > > >> > > > > >> >>> > > (in
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> ApacheBahirr
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > like repository),
> >> but
> >> > not
> >> > > > all
> >> > > > > > > > modules
> >> > > > > > > > >> > from
> >> > > > > > > > >> > > > the
> >> > > > > > > > >> > > > > >> page
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> >> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> > >
> >> > > > > > > > >> > > > > >> >>> > > >> >> >
> >> > > > > > > > >> > >
> >>
> >
>

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra <sa...@gmail.com>]

Hi Emmanouil,

Thank you for your contribution. I have merged your changes in master.

Regards,
Saikat


On Sun, Dec 29, 2019 at 12:20 PM Saikat Maitra <sa...@gmail.com>
wrote:

> Thank you, Emmanouil
>
> I am reviewing the changes.
>
> Regards,
> Saikat
>
> On Sat, Dec 28, 2019 at 3:48 PM Emmanouil Gkatziouras <
> gkatzioura@gmail.com> wrote:
>
>> Hi Saikat!
>>
>> The dependencies have been updated, and the merge conflicts resolved.
>> Kind regards
>>
>> On Sat, 28 Dec 2019, 23:27 Saikat Maitra, <sa...@gmail.com>
>> wrote:
>>
>> > Hi Emmanouil,
>> >
>> > Thank you for making the changes. I have merged my PR changes in master.
>> > Can you please take the latest from master branch and resolve the
>> conflicts
>> > in your PR.
>> >
>> > I have also reviewed the changes in your PR and requested minor changes
>> > specific to dependencies version and license information.
>> >
>> > Can you please review and share your feedback?
>> >
>> > Regards,
>> > Saikat
>> >
>> >
>> >
>> >
>> > On Sat, Dec 28, 2019 at 4:38 AM Emmanouil Gkatziouras <
>> > gkatzioura@gmail.com>
>> > wrote:
>> >
>> > > Hi Saikat!
>> > >
>> > > Thank you on this one! I tried it and it worked as expected [1]
>> > >
>> > > Kind regards
>> > > Emmanouil
>> > >
>> > > [1]
>> > >
>> > >
>> >
>> https://ci.ignite.apache.org/viewLog.html?buildId=4888357&tab=buildLog&_focus=2046#_state=2046
>> > >
>> > > *Emmanouil Gkatziouras*
>> > > https://egkatzioura.com/ |
>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > https://github.com/gkatzioura
>> > >
>> > >
>> > > On Thu, 26 Dec 2019 at 19:50, Saikat Maitra <sa...@gmail.com>
>> > > wrote:
>> > >
>> > > > Hi Emmanouil,
>> > > >
>> > > > I have been able to resolve the build failures for missing modules.
>> > > >
>> > > >
>> > > >
>> > >
>> >
>> https://ci.ignite.apache.org/viewLog.html?buildId=4883394&buildTypeId=IgniteExtensions_Build
>> > > >
>> > > > Will you please take the latest changes from my PR and run the build
>> > for
>> > > > your PR.
>> > > >
>> > > > https://github.com/apache/ignite-extensions/pull/1/files
>> > > >
>> > > > The new build configurations should let your module build also pass.
>> > > >
>> > > > The issue that was causing failure was that I had put ignite.zip as
>> > > > artifact dependency and it was downloading the artifacts including
>> pom
>> > > file
>> > > > inside the build checkout folder of ignite-extensions and as a
>> result
>> > the
>> > > > pom.xml of ignite-extensions was getting overwritten by ignite's
>> > pom.xml.
>> > > > This was resulting in missing flink-ext module error as it was not
>> even
>> > > > present in the ignite main pom.xml.
>> > > >
>> > > > To resolve the issue I just had to download and extract artifact of
>> > > > ignite.zip in a separate directory.
>> > > >
>> > > > Regards,
>> > > > Saikat
>> > > >
>> > > >
>> > > > On Sat, Nov 30, 2019 at 5:53 PM Emmanouil Gkatziouras <
>> > > > gkatzioura@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > Hi Saikat!
>> > > > >
>> > > > > Thank you for your assistance on that!
>> > > > >
>> > > > > Kind regards
>> > > > >
>> > > > > *Emmanouil Gkatziouras*
>> > > > > https://egkatzioura.com/ |
>> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > > https://github.com/gkatzioura
>> > > > >
>> > > > >
>> > > > > On Sat, 30 Nov 2019 at 23:51, Saikat Maitra <
>> saikat.maitra@gmail.com
>> > >
>> > > > > wrote:
>> > > > >
>> > > > > > Hi Emmanouil,
>> > > > > >
>> > > > > > I looked into the build logs and I observed that since I added
>> > > > artifacts
>> > > > > > dependencies to "Build apache Ignite"[1] it was able to pull the
>> > > > required
>> > > > > > dependencies and was able to run the existing tests. It is
>> however
>> > > not
>> > > > > > identifying new modules like I changed from flink to flink-ext
>> or
>> > > > > pub-sub.
>> > > > > >
>> > > > > > [1]
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://ci.ignite.apache.org/admin/editDependencies.html?id=buildType:IgniteExtensions_Build
>> > > > > >
>> > > > > > I will look into the issue and debug further.
>> > > > > >
>> > > > > > Regards,
>> > > > > > Saikat
>> > > > > >
>> > > > > >
>> > > > > > On Sat, Nov 30, 2019 at 5:35 PM Emmanouil Gkatziouras <
>> > > > > > gkatzioura@gmail.com>
>> > > > > > wrote:
>> > > > > >
>> > > > > > > Hi Saikat!
>> > > > > > >
>> > > > > > > From the logs[1] it seems that TC cannot find the project.
>> > However
>> > > I
>> > > > > did
>> > > > > > > include the project on the parent pom (modules sections) [2]
>> > > > > > > I tried to reproduce locally unfortunately I am not aware of
>> the
>> > > full
>> > > > > > > context on TC. It seems as if a different parent pom is being
>> > > picked
>> > > > > up.
>> > > > > > >
>> > > > > > > Kind regards
>> > > > > > >
>> > > > > > > [1]
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://ci.ignite.apache.org/viewLog.html?buildId=4804893&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=1704
>> > > > > > > [2]
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://github.com/gkatzioura/ignite-extensions/blob/d6a8beee7feff28f59d9a12be692016305564d25/pom.xml#L46
>> > > > > > >
>> > > > > > > *Emmanouil Gkatziouras*
>> > > > > > > https://egkatzioura.com/ |
>> > > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > > > > https://github.com/gkatzioura
>> > > > > > >
>> > > > > > >
>> > > > > > > On Sat, 30 Nov 2019 at 22:20, Saikat Maitra <
>> > > saikat.maitra@gmail.com
>> > > > >
>> > > > > > > wrote:
>> > > > > > >
>> > > > > > > > Hello,
>> > > > > > > >
>> > > > > > > > I have made changes in Ignite Extensions build
>> configurations.
>> > > > > > > >
>> > > > > > > > It is now running the tests as expected.
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://ci.ignite.apache.org/viewLog.html?buildId=4804480&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
>> > > > > > > >
>> > > > > > > > Regards,
>> > > > > > > > Saikat
>> > > > > > > >
>> > > > > > > > On Thu, Nov 28, 2019 at 1:29 PM Saikat Maitra <
>> > > > > saikat.maitra@gmail.com
>> > > > > > >
>> > > > > > > > wrote:
>> > > > > > > >
>> > > > > > > > > Hi,
>> > > > > > > > >
>> > > > > > > > > We do run build with -DskipTests and then configure
>> selective
>> > > > test
>> > > > > > > suites
>> > > > > > > > > to run.
>> > > > > > > > >
>> > > > > > > > > I will check why these test suites are not getting
>> executed.
>> > > > > > > > >
>> > > > > > > > > Regards
>> > > > > > > > > Saikat
>> > > > > > > > >
>> > > > > > > > > On Wed, 27 Nov 2019 at 3:05 PM, Emmanouil Gkatziouras <
>> > > > > > > > > gkatzioura@gmail.com> wrote:
>> > > > > > > > >
>> > > > > > > > >> Hi all,
>> > > > > > > > >>
>> > > > > > > > >> I did add a test suite and run the build with the
>> > > corresponding
>> > > > > > > > arguments,
>> > > > > > > > >> however the tests did not run [1].
>> > > > > > > > >> I checked the Flink's build logs and the same message is
>> > > > displayed
>> > > > > > > there
>> > > > > > > > >> too `No tests to run` [2].
>> > > > > > > > >>
>> > > > > > > > >> [1]
>> > > > > > > > >>
>> > > > > > > > >>
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://ci.ignite.apache.org/viewLog.html?buildId=4798203&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=248
>> > > > > > > > >> [2]
>> > > > > > > > >>
>> > > > > > > > >>
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4789199&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=1256
>> > > > > > > > >>
>> > > > > > > > >> Kind regards
>> > > > > > > > >> *Emmanouil Gkatziouras*
>> > > > > > > > >> https://egkatzioura.com/ |
>> > > > > > > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > > > > > >> https://github.com/gkatzioura
>> > > > > > > > >>
>> > > > > > > > >>
>> > > > > > > > >> On Wed, 27 Nov 2019 at 18:58, Saikat Maitra <
>> > > > > > saikat.maitra@gmail.com>
>> > > > > > > > >> wrote:
>> > > > > > > > >>
>> > > > > > > > >> > Hi,
>> > > > > > > > >> >
>> > > > > > > > >> > We will need to add test suites for tests to be
>> executed
>> > in
>> > > > > build.
>> > > > > > > We
>> > > > > > > > >> pass
>> > > > > > > > >> > them as TestSuites param. I had added Flink sink and
>> > source
>> > > > > > > > testsuites.
>> > > > > > > > >> >
>> > > > > > > > >> > Can you please review and confirm.
>> > > > > > > > >> >
>> > > > > > > > >> > Regards
>> > > > > > > > >> > Saikat
>> > > > > > > > >> >
>> > > > > > > > >> > On Wed, 27 Nov 2019 at 6:00 AM, Emmanouil Gkatziouras <
>> > > > > > > > >> > gkatzioura@gmail.com>
>> > > > > > > > >> > wrote:
>> > > > > > > > >> >
>> > > > > > > > >> > > Hi all!
>> > > > > > > > >> > >
>> > > > > > > > >> > > I was successful building and running my pull
>> requests.
>> > It
>> > > > > seems
>> > > > > > > > that
>> > > > > > > > >> the
>> > > > > > > > >> > > tests do not run both for Flink and Pub/Sub [1]
>> > > > > > > > >> > > If there is something I cannot do to make them
>> > > discoverable
>> > > > > > please
>> > > > > > > > >> let me
>> > > > > > > > >> > > know.
>> > > > > > > > >> > >
>> > > > > > > > >> > > [1]
>> > > > > > > > >> > >
>> > > > > > > > >> > >
>> > > > > > > > >> >
>> > > > > > > > >>
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4796722&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=283
>> > > > > > > > >> > >
>> > > > > > > > >> > > *Emmanouil Gkatziouras*
>> > > > > > > > >> > > https://egkatzioura.com/ |
>> > > > > > > > >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > > > > > >> > > https://github.com/gkatzioura
>> > > > > > > > >> > >
>> > > > > > > > >> > >
>> > > > > > > > >> > > On Wed, 27 Nov 2019 at 04:57, Saikat Maitra <
>> > > > > > > > saikat.maitra@gmail.com>
>> > > > > > > > >> > > wrote:
>> > > > > > > > >> > >
>> > > > > > > > >> > > > Hi Emmanouil,
>> > > > > > > > >> > > >
>> > > > > > > > >> > > > I have added you as contributor in Ignite
>> Extensions
>> > > > > project.
>> > > > > > > Can
>> > > > > > > > >> you
>> > > > > > > > >> > > > please check and confirm if you are able to see the
>> > > > project
>> > > > > > and
>> > > > > > > > >> execute
>> > > > > > > > >> > > > build on your pull request.
>> > > > > > > > >> > > >
>> > > > > > > > >> > > > Regards,
>> > > > > > > > >> > > > Saikat
>> > > > > > > > >> > > >
>> > > > > > > > >> > > > On Tue, Nov 26, 2019 at 12:49 PM Emmanouil
>> > Gkatziouras <
>> > > > > > > > >> > > > gkatzioura@gmail.com>
>> > > > > > > > >> > > > wrote:
>> > > > > > > > >> > > >
>> > > > > > > > >> > > > > Hi all!
>> > > > > > > > >> > > > >
>> > > > > > > > >> > > > > If someone could give me read access to the Job
>> of
>> > > > Ignite
>> > > > > > > > >> Extensions
>> > > > > > > > >> > on
>> > > > > > > > >> > > > > Team City will greatly help me (username
>> > gkatzioura).
>> > > > > > > > >> > > > > I suppose the builds get triggered automatically
>> on
>> > a
>> > > > pull
>> > > > > > > > >> request.
>> > > > > > > > >> > > > >
>> > > > > > > > >> > > > > Kind regards,
>> > > > > > > > >> > > > > Emmanouil
>> > > > > > > > >> > > > >
>> > > > > > > > >> > > > > *Emmanouil Gkatziouras*
>> > > > > > > > >> > > > > https://egkatzioura.com/ |
>> > > > > > > > >> > > > >
>> https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > > > > > >> > > > > https://github.com/gkatzioura
>> > > > > > > > >> > > > >
>> > > > > > > > >> > > > >
>> > > > > > > > >> > > > > On Sun, 24 Nov 2019 at 21:11, Emmanouil
>> Gkatziouras
>> > <
>> > > > > > > > >> > > > gkatzioura@gmail.com>
>> > > > > > > > >> > > > > wrote:
>> > > > > > > > >> > > > >
>> > > > > > > > >> > > > > > Hi Saikat!
>> > > > > > > > >> > > > > >
>> > > > > > > > >> > > > > > I just rebased with the flink branch.
>> > Unfortunately
>> > > I
>> > > > do
>> > > > > > not
>> > > > > > > > >> have
>> > > > > > > > >> > > read
>> > > > > > > > >> > > > > > access to the team city link you provided and
>> thus
>> > > > > > evaluate
>> > > > > > > > the
>> > > > > > > > >> > tests
>> > > > > > > > >> > > > for
>> > > > > > > > >> > > > > > my pull request.
>> > > > > > > > >> > > > > > I guess it has to do with ignite extensions
>> being
>> > > new.
>> > > > > > > > >> > > > > > My username is gkatzioura and email the one I
>> am
>> > > using
>> > > > > on
>> > > > > > > this
>> > > > > > > > >> > email
>> > > > > > > > >> > > > > >
>> > > > > > > > >> > > > > > Thank you for your time.
>> > > > > > > > >> > > > > >
>> > > > > > > > >> > > > > >
>> > > > > > > > >> > > > > > *Emmanouil Gkatziouras*
>> > > > > > > > >> > > > > > https://egkatzioura.com/ |
>> > > > > > > > >> > > > > >
>> https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > > > > > >> > > > > > https://github.com/gkatzioura
>> > > > > > > > >> > > > > >
>> > > > > > > > >> > > > > >
>> > > > > > > > >> > > > > > On Sun, 24 Nov 2019 at 18:03, Saikat Maitra <
>> > > > > > > > >> > saikat.maitra@gmail.com
>> > > > > > > > >> > > >
>> > > > > > > > >> > > > > > wrote:
>> > > > > > > > >> > > > > >
>> > > > > > > > >> > > > > >> Hi Emmanouil,
>> > > > > > > > >> > > > > >>
>> > > > > > > > >> > > > > >> The latest build on teamcity has passed on
>> Flink
>> > > pull
>> > > > > > > > request.
>> > > > > > > > >> > > > > >>
>> > > > > > > > >> > > > > >>
>> > > > > > > > >> > > > > >>
>> > > > > > > > >> > > > >
>> > > > > > > > >> > > >
>> > > > > > > > >> > >
>> > > > > > > > >> >
>> > > > > > > > >>
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://ci.ignite.apache.org/viewLog.html?buildId=4788928&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
>> > > > > > > > >> > > > > >>
>> > > > > > > > >> > > > > >> you should be able to take the changes I made
>> in
>> > my
>> > > > PR
>> > > > > > and
>> > > > > > > > run
>> > > > > > > > >> > build
>> > > > > > > > >> > > > on
>> > > > > > > > >> > > > > >> your pull request.
>> > > > > > > > >> > > > > >>
>> > > > > > > > >> > > > > >> Please let me know if you have any questions.
>> > > > > > > > >> > > > > >>
>> > > > > > > > >> > > > > >> Regards,
>> > > > > > > > >> > > > > >> Saikat
>> > > > > > > > >> > > > > >>
>> > > > > > > > >> > > > > >> On Sun, Nov 24, 2019 at 10:44 AM Saikat
>> Maitra <
>> > > > > > > > >> > > > saikat.maitra@gmail.com
>> > > > > > > > >> > > > > >
>> > > > > > > > >> > > > > >> wrote:
>> > > > > > > > >> > > > > >>
>> > > > > > > > >> > > > > >> > Hi Emmanouil,
>> > > > > > > > >> > > > > >> >
>> > > > > > > > >> > > > > >> > I have fixed the Flink module testsuites and
>> > have
>> > > > > setup
>> > > > > > > new
>> > > > > > > > >> > > project
>> > > > > > > > >> > > > in
>> > > > > > > > >> > > > > >> > teamcity.
>> > > > > > > > >> > > > > >> >
>> > > > > > > > >> > > > > >> >
>> > > > > > > > >> > > > > >> >
>> > > > > > > > >> > > > > >>
>> > > > > > > > >> > > > >
>> > > > > > > > >> > > >
>> > > > > > > > >> > >
>> > > > > > > > >> >
>> > > > > > > > >>
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview
>> > > > > > > > >> > > > > >> >
>> > > > > > > > >> > > > > >> > I am looking into setting up the build.
>> > > > > > > > >> > > > > >> >
>> > > > > > > > >> > > > > >> > Regards,
>> > > > > > > > >> > > > > >> > Saikat
>> > > > > > > > >> > > > > >> >
>> > > > > > > > >> > > > > >> > On Sat, Nov 23, 2019 at 9:52 AM Saikat
>> Maitra <
>> > > > > > > > >> > > > > saikat.maitra@gmail.com>
>> > > > > > > > >> > > > > >> > wrote:
>> > > > > > > > >> > > > > >> >
>> > > > > > > > >> > > > > >> >> Hi Emmanouil,
>> > > > > > > > >> > > > > >> >>
>> > > > > > > > >> > > > > >> >> Thank you for your email. Yes, the plan is
>> > once
>> > > > the
>> > > > > > > Flink
>> > > > > > > > >> PR is
>> > > > > > > > >> > > > > merged
>> > > > > > > > >> > > > > >> it
>> > > > > > > > >> > > > > >> >> will provide the Licence, parent POMs etc
>> and
>> > > you
>> > > > > can
>> > > > > > > > rebase
>> > > > > > > > >> > from
>> > > > > > > > >> > > > > >> master
>> > > > > > > > >> > > > > >> >> branch and apply the changes on top of it.
>> > > > > > > > >> > > > > >> >>
>> > > > > > > > >> > > > > >> >> I am getting some issues with test
>> failures in
>> > > > local
>> > > > > > > with
>> > > > > > > > >> > > > > >> >> GridTestProperties as the test.properties
>> is
>> > not
>> > > > > > present
>> > > > > > > > in
>> > > > > > > > >> > this
>> > > > > > > > >> > > > new
>> > > > > > > > >> > > > > >> >> project but should be available from
>> > > dependencies.
>> > > > > > > > >> > > > > >> >>
>> > > > > > > > >> > > > > >> >> Once I address this issue, I will go ahead
>> and
>> > > > merge
>> > > > > > the
>> > > > > > > > >> > changes
>> > > > > > > > >> > > > and
>> > > > > > > > >> > > > > >> then
>> > > > > > > > >> > > > > >> >> we can take it from there.
>> > > > > > > > >> > > > > >> >>
>> > > > > > > > >> > > > > >> >> Regards,
>> > > > > > > > >> > > > > >> >> Saikat
>> > > > > > > > >> > > > > >> >>
>> > > > > > > > >> > > > > >> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil
>> > > > > Gkatziouras
>> > > > > > <
>> > > > > > > > >> > > > > >> >> gkatzioura@gmail.com> wrote:
>> > > > > > > > >> > > > > >> >>
>> > > > > > > > >> > > > > >> >>> Hi all,
>> > > > > > > > >> > > > > >> >>>
>> > > > > > > > >> > > > > >> >>> I made my first pull request [1]. Since
>> this
>> > > > > project
>> > > > > > is
>> > > > > > > > >> brand
>> > > > > > > > >> > > new
>> > > > > > > > >> > > > > (no
>> > > > > > > > >> > > > > >> >>> parent poms, licensing), I reckoned it was
>> > best
>> > > > to
>> > > > > > use
>> > > > > > > > >> > Saikat's
>> > > > > > > > >> > > > > >> branch on
>> > > > > > > > >> > > > > >> >>> Flink.
>> > > > > > > > >> > > > > >> >>> I suppose the Flink branch will be merged
>> > > first.
>> > > > If
>> > > > > > not
>> > > > > > > > >> please
>> > > > > > > > >> > > > give
>> > > > > > > > >> > > > > me
>> > > > > > > > >> > > > > >> >>> guidelines on how I should proceed next.
>> > > > > > > > >> > > > > >> >>>
>> > > > > > > > >> > > > > >> >>> Kind regards
>> > > > > > > > >> > > > > >> >>> Emmanouil
>> > > > > > > > >> > > > > >> >>>
>> > > > > > > > >> > > > > >> >>> [1]
>> > > > > > https://github.com/apache/ignite-extensions/pull/2
>> > > > > > > > >> > > > > >> >>>
>> > > > > > > > >> > > > > >> >>> *Emmanouil Gkatziouras*
>> > > > > > > > >> > > > > >> >>> https://egkatzioura.com/ |
>> > > > > > > > >> > > > > >> >>>
>> > > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > > > > > >> > > > > >> >>> https://github.com/gkatzioura
>> > > > > > > > >> > > > > >> >>>
>> > > > > > > > >> > > > > >> >>>
>> > > > > > > > >> > > > > >> >>> On Fri, 22 Nov 2019 at 20:55, Denis Magda
>> <
>> > > > > > > > >> dmagda@apache.org>
>> > > > > > > > >> > > > > wrote:
>> > > > > > > > >> > > > > >> >>>
>> > > > > > > > >> > > > > >> >>> > Awesome, ping us whenever you're ready!
>> > > > > > > > >> > > > > >> >>> >
>> > > > > > > > >> > > > > >> >>> > -
>> > > > > > > > >> > > > > >> >>> > Denis
>> > > > > > > > >> > > > > >> >>> >
>> > > > > > > > >> > > > > >> >>> >
>> > > > > > > > >> > > > > >> >>> > On Fri, Nov 22, 2019 at 12:52 PM
>> Emmanouil
>> > > > > > > Gkatziouras
>> > > > > > > > <
>> > > > > > > > >> > > > > >> >>> > gkatzioura@gmail.com>
>> > > > > > > > >> > > > > >> >>> > wrote:
>> > > > > > > > >> > > > > >> >>> >
>> > > > > > > > >> > > > > >> >>> > > Hi all!
>> > > > > > > > >> > > > > >> >>> > >
>> > > > > > > > >> > > > > >> >>> > > I am sorry for being late on that. I
>> was
>> > > > trying
>> > > > > > to
>> > > > > > > > >> > refactor
>> > > > > > > > >> > > > the
>> > > > > > > > >> > > > > >> test
>> > > > > > > > >> > > > > >> >>> in
>> > > > > > > > >> > > > > >> >>> > > order not to be in need of any
>> external
>> > > tools
>> > > > > or
>> > > > > > > > >> spinning
>> > > > > > > > >> > > up a
>> > > > > > > > >> > > > > >> >>> server.
>> > > > > > > > >> > > > > >> >>> > > I did forked the new repo and indeed
>> my
>> > > > changes
>> > > > > > > > there,
>> > > > > > > > >> so
>> > > > > > > > >> > a
>> > > > > > > > >> > > > pull
>> > > > > > > > >> > > > > >> >>> request
>> > > > > > > > >> > > > > >> >>> > is
>> > > > > > > > >> > > > > >> >>> > > a matter of time!
>> > > > > > > > >> > > > > >> >>> > >
>> > > > > > > > >> > > > > >> >>> > > Kind regards
>> > > > > > > > >> > > > > >> >>> > >
>> > > > > > > > >> > > > > >> >>> > > *Emmanouil Gkatziouras*
>> > > > > > > > >> > > > > >> >>> > > https://egkatzioura.com/ |
>> > > > > > > > >> > > > > >> >>> > >
>> > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > > > > > >> > > > > >> >>> > > https://github.com/gkatzioura
>> > > > > > > > >> > > > > >> >>> > >
>> > > > > > > > >> > > > > >> >>> > >
>> > > > > > > > >> > > > > >> >>> > > On Fri, 22 Nov 2019 at 20:45, Denis
>> > Magda <
>> > > > > > > > >> > > dmagda@apache.org>
>> > > > > > > > >> > > > > >> wrote:
>> > > > > > > > >> > > > > >> >>> > >
>> > > > > > > > >> > > > > >> >>> > > > Hi Emmanouil,
>> > > > > > > > >> > > > > >> >>> > > >
>> > > > > > > > >> > > > > >> >>> > > > Do you have any questions or need
>> any
>> > > > support
>> > > > > > > from
>> > > > > > > > >> the
>> > > > > > > > >> > > > > >> community?
>> > > > > > > > >> > > > > >> >>> > > >
>> > > > > > > > >> > > > > >> >>> > > > -
>> > > > > > > > >> > > > > >> >>> > > > Denis
>> > > > > > > > >> > > > > >> >>> > > >
>> > > > > > > > >> > > > > >> >>> > > >
>> > > > > > > > >> > > > > >> >>> > > > On Sun, Nov 10, 2019 at 3:07 PM
>> Saikat
>> > > > > Maitra <
>> > > > > > > > >> > > > > >> >>> saikat.maitra@gmail.com
>> > > > > > > > >> > > > > >> >>> > >
>> > > > > > > > >> > > > > >> >>> > > > wrote:
>> > > > > > > > >> > > > > >> >>> > > >
>> > > > > > > > >> > > > > >> >>> > > >> Hi Emmanouil,
>> > > > > > > > >> > > > > >> >>> > > >>
>> > > > > > > > >> > > > > >> >>> > > >> Can you please take a looks at dev
>> > > utils,
>> > > > if
>> > > > > > > this
>> > > > > > > > is
>> > > > > > > > >> > > > > something
>> > > > > > > > >> > > > > >> >>> you are
>> > > > > > > > >> > > > > >> >>> > > >> looking for
>> > > > > > > > >> > > > > >> >>> > > >>
>> > > > > > > > >> > > > >
>> > > > > > >
>> https://github.com/apache/ignite/tree/master/modules/dev-utils
>> > > > > > > > >> > > > > >> ?
>> > > > > > > > >> > > > > >> >>> > > >>
>> > > > > > > > >> > > > > >> >>> > > >> IMO, if you can release Pub/Sub
>> server
>> > > in
>> > > > > > maven
>> > > > > > > > and
>> > > > > > > > >> > then
>> > > > > > > > >> > > > use
>> > > > > > > > >> > > > > >> it as
>> > > > > > > > >> > > > > >> >>> > > >> dependency, that would be great.
>> > > > > > > > >> > > > > >> >>> > > >>
>> > > > > > > > >> > > > > >> >>> > > >> Regards,
>> > > > > > > > >> > > > > >> >>> > > >> Saikat
>> > > > > > > > >> > > > > >> >>> > > >>
>> > > > > > > > >> > > > > >> >>> > > >> On Sun, Nov 10, 2019 at 5:00 PM
>> Saikat
>> > > > > Maitra
>> > > > > > <
>> > > > > > > > >> > > > > >> >>> > saikat.maitra@gmail.com>
>> > > > > > > > >> > > > > >> >>> > > >> wrote:
>> > > > > > > > >> > > > > >> >>> > > >>
>> > > > > > > > >> > > > > >> >>> > > >> > Hi Emmanouil,
>> > > > > > > > >> > > > > >> >>> > > >> >
>> > > > > > > > >> > > > > >> >>> > > >> > The master branch has been
>> > > initialized.
>> > > > I
>> > > > > > have
>> > > > > > > > >> > raised a
>> > > > > > > > >> > > > PR
>> > > > > > > > >> > > > > to
>> > > > > > > > >> > > > > >> >>> > migrate
>> > > > > > > > >> > > > > >> >>> > > >> > Flink module to
>> ignite-extensions.
>> > > > > > > > >> > > > > >> >>> > > >> >
>> > > > > > > > >> > > > > >> >>> > > >> > PR
>> > > > > > > > >> > https://github.com/apache/ignite-extensions/pull/1
>> > > > > > > > >> > > > > >> >>> > > >> > Jira :
>> > > > > > > > >> > > > https://issues.apache.org/jira/browse/IGNITE-12356
>> > > > > > > > >> > > > > >> >>> > > >> >
>> > > > > > > > >> > > > > >> >>> > > >> > Regards,
>> > > > > > > > >> > > > > >> >>> > > >> > Saikat
>> > > > > > > > >> > > > > >> >>> > > >> >
>> > > > > > > > >> > > > > >> >>> > > >> >
>> > > > > > > > >> > > > > >> >>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM
>> > > > Emmanouil
>> > > > > > > > >> > Gkatziouras <
>> > > > > > > > >> > > > > >> >>> > > >> > gkatzioura@gmail.com> wrote:
>> > > > > > > > >> > > > > >> >>> > > >> >
>> > > > > > > > >> > > > > >> >>> > > >> >> Hi all!
>> > > > > > > > >> > > > > >> >>> > > >> >>
>> > > > > > > > >> > > > > >> >>> > > >> >> This is an update. The
>> integration
>> > is
>> > > > > > tested
>> > > > > > > > with
>> > > > > > > > >> > > mocks
>> > > > > > > > >> > > > > >> based
>> > > > > > > > >> > > > > >> >>> on
>> > > > > > > > >> > > > > >> >>> > the
>> > > > > > > > >> > > > > >> >>> > > >> >> original Ignite repo [1].
>> > > > > > > > >> > > > > >> >>> > > >> >> I am currently waiting for the
>> > first
>> > > > > > commits
>> > > > > > > on
>> > > > > > > > >> the
>> > > > > > > > >> > > new
>> > > > > > > > >> > > > > >> >>> integration
>> > > > > > > > >> > > > > >> >>> > > >> >> project
>> > > > > > > > >> > > > > >> >>> > > >> >> to take place in order to
>> create my
>> > > > pull
>> > > > > > > > request
>> > > > > > > > >> > > > > accordingly
>> > > > > > > > >> > > > > >> >>> to the
>> > > > > > > > >> > > > > >> >>> > > >> >> initial
>> > > > > > > > >> > > > > >> >>> > > >> >> commits.
>> > > > > > > > >> > > > > >> >>> > > >> >> In order to simulate the Pub/Sub
>> > > server
>> > > > > in
>> > > > > > a
>> > > > > > > > >> test I
>> > > > > > > > >> > > > used a
>> > > > > > > > >> > > > > >> test
>> > > > > > > > >> > > > > >> >>> > > server
>> > > > > > > > >> > > > > >> >>> > > >> >> implementation [2].
>> Unfortunately
>> > > they
>> > > > > > don't
>> > > > > > > > have
>> > > > > > > > >> > this
>> > > > > > > > >> > > > > util
>> > > > > > > > >> > > > > >> >>> > uploaded
>> > > > > > > > >> > > > > >> >>> > > >> in a
>> > > > > > > > >> > > > > >> >>> > > >> >> maven repo, thus it needs to be
>> > > hosted
>> > > > > on a
>> > > > > > > > maven
>> > > > > > > > >> > repo
>> > > > > > > > >> > > > > >> (Apache
>> > > > > > > > >> > > > > >> >>> > > License
>> > > > > > > > >> > > > > >> >>> > > >> >> 2.0). Is there a repo for
>> utilities
>> > > > that
>> > > > > > > Ignite
>> > > > > > > > >> uses
>> > > > > > > > >> > > for
>> > > > > > > > >> > > > > the
>> > > > > > > > >> > > > > >> >>> > > >> test/build?
>> > > > > > > > >> > > > > >> >>> > > >> >> I have also created a ticket [3]
>> > > > > > > > >> > > > > >> >>> > > >> >>
>> > > > > > > > >> > > > > >> >>> > > >> >> Kind regards
>> > > > > > > > >> > > > > >> >>> > > >> >>
>> > > > > > > > >> > > > > >> >>> > > >> >> [1]
>> > > > > > > > >> > > > > >> >>> >
>> > > > > > > > >> > >
>> > > > > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
>> > > > > > > > >> > > > > >> >>> > > >> >> [2]
>> > > > > > > > >> > > > > >> >>>
>> > > > > > > > >>
>> > https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
>> > > > > > > > >> > > > > >> >>> > > >> >> [3]
>> > > > > > > > >> > > https://issues.apache.org/jira/browse/IGNITE-12262
>> > > > > > > > >> > > > > >> >>> > > >> >>
>> > > > > > > > >> > > > > >> >>> > > >> >> *Emmanouil Gkatziouras*
>> > > > > > > > >> > > > > >> >>> > > >> >> https://egkatzioura.com/ |
>> > > > > > > > >> > > > > >> >>> > > >> >>
>> > > > > > > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > > > > > >> > > > > >> >>> > > >> >> https://github.com/gkatzioura
>> > > > > > > > >> > > > > >> >>> > > >> >>
>> > > > > > > > >> > > > > >> >>> > > >> >>
>> > > > > > > > >> > > > > >> >>> > > >> >> On Fri, 25 Oct 2019 at 03:35,
>> > Saikat
>> > > > > > Maitra <
>> > > > > > > > >> > > > > >> >>> > saikat.maitra@gmail.com
>> > > > > > > > >> > > > > >> >>> > > >
>> > > > > > > > >> > > > > >> >>> > > >> >> wrote:
>> > > > > > > > >> > > > > >> >>> > > >> >>
>> > > > > > > > >> > > > > >> >>> > > >> >> > Hello Ilya,
>> > > > > > > > >> > > > > >> >>> > > >> >> >
>> > > > > > > > >> > > > > >> >>> > > >> >> > Thank you for your email. Yes,
>> > > before
>> > > > > we
>> > > > > > > > remove
>> > > > > > > > >> > any
>> > > > > > > > >> > > > > >> >>> integration
>> > > > > > > > >> > > > > >> >>> > we
>> > > > > > > > >> > > > > >> >>> > > >> will
>> > > > > > > > >> > > > > >> >>> > > >> >> > move them to another
>> repository
>> > and
>> > > > > > > release.
>> > > > > > > > >> > > > > >> >>> > > >> >> >
>> > > > > > > > >> > > > > >> >>> > > >> >> > We are thinking we will start
>> > with
>> > > > > > Pub/Sub
>> > > > > > > > >> > Streamer
>> > > > > > > > >> > > > > >> >>> > implementation
>> > > > > > > > >> > > > > >> >>> > > >> and
>> > > > > > > > >> > > > > >> >>> > > >> >> > continue to move other
>> > integration
>> > > as
>> > > > > > > > >> mentioned in
>> > > > > > > > >> > > the
>> > > > > > > > >> > > > > >> >>> following
>> > > > > > > > >> > > > > >> >>> > > doc.
>> > > > > > > > >> > > > > >> >>> > > >> >> >
>> > > > > > > > >> > > > > >> >>> > > >> >> >
>> > > > > > > > >> > > > > >> >>> > > >> >> >
>> > > > > > > > >> > > > > >> >>> > > >> >>
>> > > > > > > > >> > > > > >> >>> > > >>
>> > > > > > > > >> > > > > >> >>> > >
>> > > > > > > > >> > > > > >> >>> >
>> > > > > > > > >> > > > > >> >>>
>> > > > > > > > >> > > > > >>
>> > > > > > > > >> > > > >
>> > > > > > > > >> > > >
>> > > > > > > > >> > >
>> > > > > > > > >> >
>> > > > > > > > >>
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> > > > > > > > >> > > > > >> >>> > > >> >> >
>> > > > > > > > >> > > > > >> >>> > > >> >> > Regards,
>> > > > > > > > >> > > > > >> >>> > > >> >> > Saikat
>> > > > > > > > >> > > > > >> >>> > > >> >> >
>> > > > > > > > >> > > > > >> >>> > > >> >> >
>> > > > > > > > >> > > > > >> >>> > > >> >> > On Thu, Oct 24, 2019 at 11:00
>> AM
>> > > Ilya
>> > > > > > > > >> Kasnacheev <
>> > > > > > > > >> > > > > >> >>> > > >> >> > ilya.kasnacheev@gmail.com>
>> > > > > > > > >> > > > > >> >>> > > >> >> > wrote:
>> > > > > > > > >> > > > > >> >>> > > >> >> >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > Hello!
>> > > > > > > > >> > > > > >> >>> > > >> >> > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > My take on this: we want to
>> > > offload
>> > > > > > some
>> > > > > > > of
>> > > > > > > > >> > > > > >> integrations,
>> > > > > > > > >> > > > > >> >>> but
>> > > > > > > > >> > > > > >> >>> > > >> please
>> > > > > > > > >> > > > > >> >>> > > >> >> > commit
>> > > > > > > > >> > > > > >> >>> > > >> >> > > and release them to another
>> > > project
>> > > > > > > first,
>> > > > > > > > >> and
>> > > > > > > > >> > > only
>> > > > > > > > >> > > > > >> then we
>> > > > > > > > >> > > > > >> >>> > will
>> > > > > > > > >> > > > > >> >>> > > >> >> decide
>> > > > > > > > >> > > > > >> >>> > > >> >> > to
>> > > > > > > > >> > > > > >> >>> > > >> >> > > remove them from our
>> > repository.
>> > > > > > > > >> > > > > >> >>> > > >> >> > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > Please also make sure they
>> are
>> > > > > > available
>> > > > > > > > with
>> > > > > > > > >> > same
>> > > > > > > > >> > > > > >> artifact
>> > > > > > > > >> > > > > >> >>> > name
>> > > > > > > > >> > > > > >> >>> > > >> and
>> > > > > > > > >> > > > > >> >>> > > >> >> API.
>> > > > > > > > >> > > > > >> >>> > > >> >> > > If this has to be changed,
>> > let's
>> > > > wait
>> > > > > > for
>> > > > > > > > AI
>> > > > > > > > >> 3.0
>> > > > > > > > >> > > > > >> >>> > > >> >> > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > Regards,
>> > > > > > > > >> > > > > >> >>> > > >> >> > > --
>> > > > > > > > >> > > > > >> >>> > > >> >> > > Ilya Kasnacheev
>> > > > > > > > >> > > > > >> >>> > > >> >> > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > ср, 23 окт. 2019 г. в 03:23,
>> > > Saikat
>> > > > > > > Maitra
>> > > > > > > > <
>> > > > > > > > >> > > > > >> >>> > > >> saikat.maitra@gmail.com>:
>> > > > > > > > >> > > > > >> >>> > > >> >> > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > Hello Emmanouil,
>> > > > > > > > >> > > > > >> >>> > > >> >> > > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > As discussed earlier, I
>> > > discussed
>> > > > > > with
>> > > > > > > > >> Apache
>> > > > > > > > >> > > > Bahir
>> > > > > > > > >> > > > > >> >>> community
>> > > > > > > > >> > > > > >> >>> > > and
>> > > > > > > > >> > > > > >> >>> > > >> >> they
>> > > > > > > > >> > > > > >> >>> > > >> >> > > are
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > interested to have Apache
>> > > Ignite
>> > > > > > > > >> extensions as
>> > > > > > > > >> > > > part
>> > > > > > > > >> > > > > of
>> > > > > > > > >> > > > > >> >>> Apache
>> > > > > > > > >> > > > > >> >>> > > >> Bahir
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > project.
>> > > > > > > > >> > > > > >> >>> > > >> >> > > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > Can you please share
>> Pub/Sub
>> > > > > streamer
>> > > > > > > > >> > > > implementation
>> > > > > > > > >> > > > > >> >>> details
>> > > > > > > > >> > > > > >> >>> > > with
>> > > > > > > > >> > > > > >> >>> > > >> >> > Apache
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > Bahir community and
>> request
>> > for
>> > > > > > > feedback.
>> > > > > > > > >> > > > > >> >>> > > >> >> > > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > https://bahir.apache.org/
>> > > > > > > > >> > > > > >> >>> > > >> >> > > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > I have also requested for
>> > > > > contributor
>> > > > > > > > >> access
>> > > > > > > > >> > in
>> > > > > > > > >> > > > Jira
>> > > > > > > > >> > > > > >> for
>> > > > > > > > >> > > > > >> >>> > Apache
>> > > > > > > > >> > > > > >> >>> > > >> >> Bahir
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > project so that I can
>> create
>> > > > issues
>> > > > > > and
>> > > > > > > > >> assign
>> > > > > > > > >> > > to
>> > > > > > > > >> > > > > >> myself.
>> > > > > > > > >> > > > > >> >>> > > >> >> > > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > I can help with code
>> reviews
>> > as
>> > > > > well.
>> > > > > > > > >> > > > > >> >>> > > >> >> > > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > Here is the email thread
>> for
>> > > > > > reference
>> > > > > > > > >> > > > > >> >>> > > >> >> > > >
>> > > > > > > > >> > > > > >> >>> > >
>> > > > > > > > >> > > >
>> > > > > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
>> > > > > > > > >> > > > > >> >>> > > >> >> > > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > Regards,
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > Saikat
>> > > > > > > > >> > > > > >> >>> > > >> >> > > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > On Mon, Oct 21, 2019 at
>> 7:54
>> > PM
>> > > > > > Saikat
>> > > > > > > > >> Maitra
>> > > > > > > > >> > <
>> > > > > > > > >> > > > > >> >>> > > >> >> saikat.maitra@gmail.com
>> > > > > > > > >> > > > > >> >>> > > >> >> > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > wrote:
>> > > > > > > > >> > > > > >> >>> > > >> >> > > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > > Hello,
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > > As discussed I have
>> created
>> > > > > > following
>> > > > > > > > >> > > discussion
>> > > > > > > > >> > > > > >> >>> threads on
>> > > > > > > > >> > > > > >> >>> > > our
>> > > > > > > > >> > > > > >> >>> > > >> >> > > migration
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > > proposals for Apache
>> Ignite
>> > > > > > > extensions:
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > >
>> > > > > > > > >> > > > > >> >>> > > >> >> >
>> > > > > > > > >> > > > > >> >>> > > >> >>
>> > > > > > > > >> > > > > >> >>> > > >>
>> > > > > > > > >> > > > > >> >>> > >
>> > > > > > > > >> > > > > >> >>> >
>> > > > > > > > >> > > > > >> >>>
>> > > > > > > > >> > > > > >>
>> > > > > > > > >> > > > >
>> > > > > > > > >> > > >
>> > > > > > > > >> > >
>> > > > > > > > >> >
>> > > > > > > > >>
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
>> > > > > > > > >> > > > > >> >>> > > >>
>> > > > > > > > >> > > > > >>
>> > > > > > > > >>
>> > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > > I will share update as I
>> > hear
>> > > > > more
>> > > > > > > > >> > > information.
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > > Regards,
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > > Saikat
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > > On Fri, Oct 18, 2019 at
>> > 9:03
>> > > PM
>> > > > > > > Saikat
>> > > > > > > > >> > Maitra
>> > > > > > > > >> > > <
>> > > > > > > > >> > > > > >> >>> > > >> >> > saikat.maitra@gmail.com
>> > > > > > > > >> > > > > >> >>> > > >> >> > > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > > wrote:
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >> Hello Denis,
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >> Sure, sounds good. I
>> will
>> > > > > create a
>> > > > > > > > >> separate
>> > > > > > > > >> > > > > >> discussion
>> > > > > > > > >> > > > > >> >>> > > thread
>> > > > > > > > >> > > > > >> >>> > > >> to
>> > > > > > > > >> > > > > >> >>> > > >> >> get
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >> community feedback on
>> > > whether
>> > > > we
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >> should join the Bahir
>> or
>> > > > > kick-off
>> > > > > > > > >> "Ignite
>> > > > > > > > >> > > > > >> Extensions"
>> > > > > > > > >> > > > > >> >>> > > project.
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >> Regards,
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >> Saikat
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >> On Thu, Oct 17, 2019 at
>> > 2:21
>> > > > PM
>> > > > > > > Denis
>> > > > > > > > >> > Magda <
>> > > > > > > > >> > > > > >> >>> > > >> dmagda@apache.org>
>> > > > > > > > >> > > > > >> >>> > > >> >> > > wrote:
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> Folks,
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> The concept of Apache
>> > Bahir
>> > > > is
>> > > > > > > > >> precisely
>> > > > > > > > >> > > what
>> > > > > > > > >> > > > we
>> > > > > > > > >> > > > > >> >>> need!
>> > > > > > > > >> > > > > >> >>> > > >> Saikat,
>> > > > > > > > >> > > > > >> >>> > > >> >> > thanks
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > for
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> doing the research.
>> Why I
>> > > > > believe
>> > > > > > > > the *
>> > > > > > > > >> > > idea*
>> > > > > > > > >> > > > > >> fits us
>> > > > > > > > >> > > > > >> >>> > well:
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    - All integrations
>> can
>> > > be
>> > > > > > stored
>> > > > > > > > in
>> > > > > > > > >> > > > separate
>> > > > > > > > >> > > > > >> >>> Github
>> > > > > > > > >> > > > > >> >>> > > >> >> repositories
>> > > > > > > > >> > > > > >> >>> > > >> >> > > and
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    have their dev
>> > > lifecycles.
>> > > > > > We've
>> > > > > > > > not
>> > > > > > > > >> > > > obliged
>> > > > > > > > >> > > > > to
>> > > > > > > > >> > > > > >> >>> couple
>> > > > > > > > >> > > > > >> >>> > > all
>> > > > > > > > >> > > > > >> >>> > > >> >> the
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> integrations
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    in a single repo.
>> For
>> > > > > > instance,
>> > > > > > > > >> Spark
>> > > > > > > > >> > can
>> > > > > > > > >> > > > be
>> > > > > > > > >> > > > > >> >>> located
>> > > > > > > > >> > > > > >> >>> > in
>> > > > > > > > >> > > > > >> >>> > > a
>> > > > > > > > >> > > > > >> >>> > > >> >> > > dedicated
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> repo
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    while streaming
>> > > > integrations
>> > > > > > > might
>> > > > > > > > >> be
>> > > > > > > > >> > in
>> > > > > > > > >> > > a
>> > > > > > > > >> > > > > >> single
>> > > > > > > > >> > > > > >> >>> one.
>> > > > > > > > >> > > > > >> >>> > > >> It's
>> > > > > > > > >> > > > > >> >>> > > >> >> up
>> > > > > > > > >> > > > > >> >>> > > >> >> > to
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > us.
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    - All the
>> repositories
>> > > and
>> > > > > > > > >> integrations
>> > > > > > > > >> > > > > belong
>> > > > > > > > >> > > > > >> to
>> > > > > > > > >> > > > > >> >>> ASF.
>> > > > > > > > >> > > > > >> >>> > > >> We're
>> > > > > > > > >> > > > > >> >>> > > >> >> not
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> dumping
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    them on Github but
>> > > rather
>> > > > > keep
>> > > > > > > > >> > supporting
>> > > > > > > > >> > > > and
>> > > > > > > > >> > > > > >> >>> > developing
>> > > > > > > > >> > > > > >> >>> > > >> in
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> accordance with
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    ASF vision and
>> > > practices.
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    - There is a way to
>> > > reward
>> > > > > > > > >> contributors
>> > > > > > > > >> > > via
>> > > > > > > > >> > > > > >> >>> > > committership
>> > > > > > > > >> > > > > >> >>> > > >> and
>> > > > > > > > >> > > > > >> >>> > > >> >> > PMC
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    membership. You
>> won't
>> > > get
>> > > > > this
>> > > > > > > if
>> > > > > > > > a
>> > > > > > > > >> > > project
>> > > > > > > > >> > > > > is
>> > > > > > > > >> > > > > >> >>> just
>> > > > > > > > >> > > > > >> >>> > one
>> > > > > > > > >> > > > > >> >>> > > of
>> > > > > > > > >> > > > > >> >>> > > >> >> the
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> millions of
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    Github projects.
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> Saikat, as Ignite PMC
>> > > member,
>> > > > > > could
>> > > > > > > > you
>> > > > > > > > >> > > please
>> > > > > > > > >> > > > > >> >>> kick-off a
>> > > > > > > > >> > > > > >> >>> > > >> >> separate
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> dev-list
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> discussion to involve
>> > more
>> > > > > > > community
>> > > > > > > > >> > members
>> > > > > > > > >> > > > and
>> > > > > > > > >> > > > > >> >>> > referring
>> > > > > > > > >> > > > > >> >>> > > to
>> > > > > > > > >> > > > > >> >>> > > >> >> this
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> thread?
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> I think the primary
>> > > question
>> > > > > the
>> > > > > > > > >> community
>> > > > > > > > >> > > > needs
>> > > > > > > > >> > > > > >> to
>> > > > > > > > >> > > > > >> >>> > answer
>> > > > > > > > >> > > > > >> >>> > > >> >> whether
>> > > > > > > > >> > > > > >> >>> > > >> >> > we
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> should join the Bahir
>> or
>> > > > > kick-off
>> > > > > > > > >> "Ignite
>> > > > > > > > >> > > > > >> Extensions"
>> > > > > > > > >> > > > > >> >>> > > >> project?
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> -
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> Denis
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> On Thu, Oct 17, 2019
>> at
>> > > 2:54
>> > > > AM
>> > > > > > > > Alexey
>> > > > > > > > >> > > > Zinoviev
>> > > > > > > > >> > > > > <
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > zaleslaw.sin@gmail.com>
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> wrote:
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > Maybe we could move
>> all
>> > > our
>> > > > > > > > Streaming
>> > > > > > > > >> > > > > >> Integrations
>> > > > > > > > >> > > > > >> >>> > there,
>> > > > > > > > >> > > > > >> >>> > > >> but
>> > > > > > > > >> > > > > >> >>> > > >> >> > what
>> > > > > > > > >> > > > > >> >>> > > >> >> > > is
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> about
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > maintaining and
>> > committer
>> > > > > > > > >> permissions to
>> > > > > > > > >> > > the
>> > > > > > > > >> > > > > new
>> > > > > > > > >> > > > > >> >>> > > >> repositories?
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > I see the list of
>> the
>> > > > > > committers
>> > > > > > > > and
>> > > > > > > > >> PMC
>> > > > > > > > >> > > > > members
>> > > > > > > > >> > > > > >> >>> there
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
>> > > > > > > > >> > > https://bahir.apache.org/community-members/
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > Could somebody from
>> > > Ignite
>> > > > > > > > community
>> > > > > > > > >> be
>> > > > > > > > >> > > > added
>> > > > > > > > >> > > > > to
>> > > > > > > > >> > > > > >> >>> this
>> > > > > > > > >> > > > > >> >>> > > list
>> > > > > > > > >> > > > > >> >>> > > >> as
>> > > > > > > > >> > > > > >> >>> > > >> >> a
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > PMC/committer to
>> > maintain
>> > > > > > Ignite
>> > > > > > > > >> > > > integrations?
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > If yes, I'd happy to
>> > join
>> > > > > this
>> > > > > > > > >> project
>> > > > > > > > >> > and
>> > > > > > > > >> > > > > >> >>> collaborate
>> > > > > > > > >> > > > > >> >>> > > with
>> > > > > > > > >> > > > > >> >>> > > >> >> these
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > guys
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > together
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > If no, and we should
>> > > start
>> > > > > from
>> > > > > > > the
>> > > > > > > > >> zero
>> > > > > > > > >> > > > level
>> > > > > > > > >> > > > > >> with
>> > > > > > > > >> > > > > >> >>> > > >> external
>> > > > > > > > >> > > > > >> >>> > > >> >> PRs
>> > > > > > > > >> > > > > >> >>> > > >> >> > -
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> hmmm,
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > it's better to have
>> our
>> > > own
>> > > > > > > > external
>> > > > > > > > >> > > > > repository
>> > > > > > > > >> > > > > >> >>> with
>> > > > > > > > >> > > > > >> >>> > > >> >> ApacheBahirr
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> ideology.
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > Also, I agree, that
>> all
>> > > > > > > connectors
>> > > > > > > > >> could
>> > > > > > > > >> > > be
>> > > > > > > > >> > > > > >> moved
>> > > > > > > > >> > > > > >> >>> there
>> > > > > > > > >> > > > > >> >>> > > (in
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> ApacheBahirr
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > like repository),
>> but
>> > not
>> > > > all
>> > > > > > > > modules
>> > > > > > > > >> > from
>> > > > > > > > >> > > > the
>> > > > > > > > >> > > > > >> page
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
>> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
>> > > > > > > > >> > > > > >> >>> > > >> >> > > >
>> > > > > > > > >> > > > > >> >>> > > >> >> > >
>> > > > > > > > >> > > > > >> >>> > > >> >> >
>> > > > > > > > >> > >
>>
>

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra <sa...@gmail.com>]

Thank you, Emmanouil

I am reviewing the changes.

Regards,
Saikat

On Sat, Dec 28, 2019 at 3:48 PM Emmanouil Gkatziouras <gk...@gmail.com>
wrote:

> Hi Saikat!
>
> The dependencies have been updated, and the merge conflicts resolved.
> Kind regards
>
> On Sat, 28 Dec 2019, 23:27 Saikat Maitra, <sa...@gmail.com> wrote:
>
> > Hi Emmanouil,
> >
> > Thank you for making the changes. I have merged my PR changes in master.
> > Can you please take the latest from master branch and resolve the
> conflicts
> > in your PR.
> >
> > I have also reviewed the changes in your PR and requested minor changes
> > specific to dependencies version and license information.
> >
> > Can you please review and share your feedback?
> >
> > Regards,
> > Saikat
> >
> >
> >
> >
> > On Sat, Dec 28, 2019 at 4:38 AM Emmanouil Gkatziouras <
> > gkatzioura@gmail.com>
> > wrote:
> >
> > > Hi Saikat!
> > >
> > > Thank you on this one! I tried it and it worked as expected [1]
> > >
> > > Kind regards
> > > Emmanouil
> > >
> > > [1]
> > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4888357&tab=buildLog&_focus=2046#_state=2046
> > >
> > > *Emmanouil Gkatziouras*
> > > https://egkatzioura.com/ |
> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > https://github.com/gkatzioura
> > >
> > >
> > > On Thu, 26 Dec 2019 at 19:50, Saikat Maitra <sa...@gmail.com>
> > > wrote:
> > >
> > > > Hi Emmanouil,
> > > >
> > > > I have been able to resolve the build failures for missing modules.
> > > >
> > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4883394&buildTypeId=IgniteExtensions_Build
> > > >
> > > > Will you please take the latest changes from my PR and run the build
> > for
> > > > your PR.
> > > >
> > > > https://github.com/apache/ignite-extensions/pull/1/files
> > > >
> > > > The new build configurations should let your module build also pass.
> > > >
> > > > The issue that was causing failure was that I had put ignite.zip as
> > > > artifact dependency and it was downloading the artifacts including
> pom
> > > file
> > > > inside the build checkout folder of ignite-extensions and as a result
> > the
> > > > pom.xml of ignite-extensions was getting overwritten by ignite's
> > pom.xml.
> > > > This was resulting in missing flink-ext module error as it was not
> even
> > > > present in the ignite main pom.xml.
> > > >
> > > > To resolve the issue I just had to download and extract artifact of
> > > > ignite.zip in a separate directory.
> > > >
> > > > Regards,
> > > > Saikat
> > > >
> > > >
> > > > On Sat, Nov 30, 2019 at 5:53 PM Emmanouil Gkatziouras <
> > > > gkatzioura@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi Saikat!
> > > > >
> > > > > Thank you for your assistance on that!
> > > > >
> > > > > Kind regards
> > > > >
> > > > > *Emmanouil Gkatziouras*
> > > > > https://egkatzioura.com/ |
> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > https://github.com/gkatzioura
> > > > >
> > > > >
> > > > > On Sat, 30 Nov 2019 at 23:51, Saikat Maitra <
> saikat.maitra@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > > > Hi Emmanouil,
> > > > > >
> > > > > > I looked into the build logs and I observed that since I added
> > > > artifacts
> > > > > > dependencies to "Build apache Ignite"[1] it was able to pull the
> > > > required
> > > > > > dependencies and was able to run the existing tests. It is
> however
> > > not
> > > > > > identifying new modules like I changed from flink to flink-ext or
> > > > > pub-sub.
> > > > > >
> > > > > > [1]
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/admin/editDependencies.html?id=buildType:IgniteExtensions_Build
> > > > > >
> > > > > > I will look into the issue and debug further.
> > > > > >
> > > > > > Regards,
> > > > > > Saikat
> > > > > >
> > > > > >
> > > > > > On Sat, Nov 30, 2019 at 5:35 PM Emmanouil Gkatziouras <
> > > > > > gkatzioura@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Saikat!
> > > > > > >
> > > > > > > From the logs[1] it seems that TC cannot find the project.
> > However
> > > I
> > > > > did
> > > > > > > include the project on the parent pom (modules sections) [2]
> > > > > > > I tried to reproduce locally unfortunately I am not aware of
> the
> > > full
> > > > > > > context on TC. It seems as if a different parent pom is being
> > > picked
> > > > > up.
> > > > > > >
> > > > > > > Kind regards
> > > > > > >
> > > > > > > [1]
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4804893&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=1704
> > > > > > > [2]
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/gkatzioura/ignite-extensions/blob/d6a8beee7feff28f59d9a12be692016305564d25/pom.xml#L46
> > > > > > >
> > > > > > > *Emmanouil Gkatziouras*
> > > > > > > https://egkatzioura.com/ |
> > > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > > > https://github.com/gkatzioura
> > > > > > >
> > > > > > >
> > > > > > > On Sat, 30 Nov 2019 at 22:20, Saikat Maitra <
> > > saikat.maitra@gmail.com
> > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hello,
> > > > > > > >
> > > > > > > > I have made changes in Ignite Extensions build
> configurations.
> > > > > > > >
> > > > > > > > It is now running the tests as expected.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4804480&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> > > > > > > >
> > > > > > > > Regards,
> > > > > > > > Saikat
> > > > > > > >
> > > > > > > > On Thu, Nov 28, 2019 at 1:29 PM Saikat Maitra <
> > > > > saikat.maitra@gmail.com
> > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi,
> > > > > > > > >
> > > > > > > > > We do run build with -DskipTests and then configure
> selective
> > > > test
> > > > > > > suites
> > > > > > > > > to run.
> > > > > > > > >
> > > > > > > > > I will check why these test suites are not getting
> executed.
> > > > > > > > >
> > > > > > > > > Regards
> > > > > > > > > Saikat
> > > > > > > > >
> > > > > > > > > On Wed, 27 Nov 2019 at 3:05 PM, Emmanouil Gkatziouras <
> > > > > > > > > gkatzioura@gmail.com> wrote:
> > > > > > > > >
> > > > > > > > >> Hi all,
> > > > > > > > >>
> > > > > > > > >> I did add a test suite and run the build with the
> > > corresponding
> > > > > > > > arguments,
> > > > > > > > >> however the tests did not run [1].
> > > > > > > > >> I checked the Flink's build logs and the same message is
> > > > displayed
> > > > > > > there
> > > > > > > > >> too `No tests to run` [2].
> > > > > > > > >>
> > > > > > > > >> [1]
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4798203&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=248
> > > > > > > > >> [2]
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4789199&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=1256
> > > > > > > > >>
> > > > > > > > >> Kind regards
> > > > > > > > >> *Emmanouil Gkatziouras*
> > > > > > > > >> https://egkatzioura.com/ |
> > > > > > > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > > > > >> https://github.com/gkatzioura
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > > >> On Wed, 27 Nov 2019 at 18:58, Saikat Maitra <
> > > > > > saikat.maitra@gmail.com>
> > > > > > > > >> wrote:
> > > > > > > > >>
> > > > > > > > >> > Hi,
> > > > > > > > >> >
> > > > > > > > >> > We will need to add test suites for tests to be executed
> > in
> > > > > build.
> > > > > > > We
> > > > > > > > >> pass
> > > > > > > > >> > them as TestSuites param. I had added Flink sink and
> > source
> > > > > > > > testsuites.
> > > > > > > > >> >
> > > > > > > > >> > Can you please review and confirm.
> > > > > > > > >> >
> > > > > > > > >> > Regards
> > > > > > > > >> > Saikat
> > > > > > > > >> >
> > > > > > > > >> > On Wed, 27 Nov 2019 at 6:00 AM, Emmanouil Gkatziouras <
> > > > > > > > >> > gkatzioura@gmail.com>
> > > > > > > > >> > wrote:
> > > > > > > > >> >
> > > > > > > > >> > > Hi all!
> > > > > > > > >> > >
> > > > > > > > >> > > I was successful building and running my pull
> requests.
> > It
> > > > > seems
> > > > > > > > that
> > > > > > > > >> the
> > > > > > > > >> > > tests do not run both for Flink and Pub/Sub [1]
> > > > > > > > >> > > If there is something I cannot do to make them
> > > discoverable
> > > > > > please
> > > > > > > > >> let me
> > > > > > > > >> > > know.
> > > > > > > > >> > >
> > > > > > > > >> > > [1]
> > > > > > > > >> > >
> > > > > > > > >> > >
> > > > > > > > >> >
> > > > > > > > >>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4796722&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=283
> > > > > > > > >> > >
> > > > > > > > >> > > *Emmanouil Gkatziouras*
> > > > > > > > >> > > https://egkatzioura.com/ |
> > > > > > > > >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > > > > >> > > https://github.com/gkatzioura
> > > > > > > > >> > >
> > > > > > > > >> > >
> > > > > > > > >> > > On Wed, 27 Nov 2019 at 04:57, Saikat Maitra <
> > > > > > > > saikat.maitra@gmail.com>
> > > > > > > > >> > > wrote:
> > > > > > > > >> > >
> > > > > > > > >> > > > Hi Emmanouil,
> > > > > > > > >> > > >
> > > > > > > > >> > > > I have added you as contributor in Ignite Extensions
> > > > > project.
> > > > > > > Can
> > > > > > > > >> you
> > > > > > > > >> > > > please check and confirm if you are able to see the
> > > > project
> > > > > > and
> > > > > > > > >> execute
> > > > > > > > >> > > > build on your pull request.
> > > > > > > > >> > > >
> > > > > > > > >> > > > Regards,
> > > > > > > > >> > > > Saikat
> > > > > > > > >> > > >
> > > > > > > > >> > > > On Tue, Nov 26, 2019 at 12:49 PM Emmanouil
> > Gkatziouras <
> > > > > > > > >> > > > gkatzioura@gmail.com>
> > > > > > > > >> > > > wrote:
> > > > > > > > >> > > >
> > > > > > > > >> > > > > Hi all!
> > > > > > > > >> > > > >
> > > > > > > > >> > > > > If someone could give me read access to the Job of
> > > > Ignite
> > > > > > > > >> Extensions
> > > > > > > > >> > on
> > > > > > > > >> > > > > Team City will greatly help me (username
> > gkatzioura).
> > > > > > > > >> > > > > I suppose the builds get triggered automatically
> on
> > a
> > > > pull
> > > > > > > > >> request.
> > > > > > > > >> > > > >
> > > > > > > > >> > > > > Kind regards,
> > > > > > > > >> > > > > Emmanouil
> > > > > > > > >> > > > >
> > > > > > > > >> > > > > *Emmanouil Gkatziouras*
> > > > > > > > >> > > > > https://egkatzioura.com/ |
> > > > > > > > >> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > > > > >> > > > > https://github.com/gkatzioura
> > > > > > > > >> > > > >
> > > > > > > > >> > > > >
> > > > > > > > >> > > > > On Sun, 24 Nov 2019 at 21:11, Emmanouil
> Gkatziouras
> > <
> > > > > > > > >> > > > gkatzioura@gmail.com>
> > > > > > > > >> > > > > wrote:
> > > > > > > > >> > > > >
> > > > > > > > >> > > > > > Hi Saikat!
> > > > > > > > >> > > > > >
> > > > > > > > >> > > > > > I just rebased with the flink branch.
> > Unfortunately
> > > I
> > > > do
> > > > > > not
> > > > > > > > >> have
> > > > > > > > >> > > read
> > > > > > > > >> > > > > > access to the team city link you provided and
> thus
> > > > > > evaluate
> > > > > > > > the
> > > > > > > > >> > tests
> > > > > > > > >> > > > for
> > > > > > > > >> > > > > > my pull request.
> > > > > > > > >> > > > > > I guess it has to do with ignite extensions
> being
> > > new.
> > > > > > > > >> > > > > > My username is gkatzioura and email the one I am
> > > using
> > > > > on
> > > > > > > this
> > > > > > > > >> > email
> > > > > > > > >> > > > > >
> > > > > > > > >> > > > > > Thank you for your time.
> > > > > > > > >> > > > > >
> > > > > > > > >> > > > > >
> > > > > > > > >> > > > > > *Emmanouil Gkatziouras*
> > > > > > > > >> > > > > > https://egkatzioura.com/ |
> > > > > > > > >> > > > > >
> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > > > > >> > > > > > https://github.com/gkatzioura
> > > > > > > > >> > > > > >
> > > > > > > > >> > > > > >
> > > > > > > > >> > > > > > On Sun, 24 Nov 2019 at 18:03, Saikat Maitra <
> > > > > > > > >> > saikat.maitra@gmail.com
> > > > > > > > >> > > >
> > > > > > > > >> > > > > > wrote:
> > > > > > > > >> > > > > >
> > > > > > > > >> > > > > >> Hi Emmanouil,
> > > > > > > > >> > > > > >>
> > > > > > > > >> > > > > >> The latest build on teamcity has passed on
> Flink
> > > pull
> > > > > > > > request.
> > > > > > > > >> > > > > >>
> > > > > > > > >> > > > > >>
> > > > > > > > >> > > > > >>
> > > > > > > > >> > > > >
> > > > > > > > >> > > >
> > > > > > > > >> > >
> > > > > > > > >> >
> > > > > > > > >>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4788928&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> > > > > > > > >> > > > > >>
> > > > > > > > >> > > > > >> you should be able to take the changes I made
> in
> > my
> > > > PR
> > > > > > and
> > > > > > > > run
> > > > > > > > >> > build
> > > > > > > > >> > > > on
> > > > > > > > >> > > > > >> your pull request.
> > > > > > > > >> > > > > >>
> > > > > > > > >> > > > > >> Please let me know if you have any questions.
> > > > > > > > >> > > > > >>
> > > > > > > > >> > > > > >> Regards,
> > > > > > > > >> > > > > >> Saikat
> > > > > > > > >> > > > > >>
> > > > > > > > >> > > > > >> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra
> <
> > > > > > > > >> > > > saikat.maitra@gmail.com
> > > > > > > > >> > > > > >
> > > > > > > > >> > > > > >> wrote:
> > > > > > > > >> > > > > >>
> > > > > > > > >> > > > > >> > Hi Emmanouil,
> > > > > > > > >> > > > > >> >
> > > > > > > > >> > > > > >> > I have fixed the Flink module testsuites and
> > have
> > > > > setup
> > > > > > > new
> > > > > > > > >> > > project
> > > > > > > > >> > > > in
> > > > > > > > >> > > > > >> > teamcity.
> > > > > > > > >> > > > > >> >
> > > > > > > > >> > > > > >> >
> > > > > > > > >> > > > > >> >
> > > > > > > > >> > > > > >>
> > > > > > > > >> > > > >
> > > > > > > > >> > > >
> > > > > > > > >> > >
> > > > > > > > >> >
> > > > > > > > >>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview
> > > > > > > > >> > > > > >> >
> > > > > > > > >> > > > > >> > I am looking into setting up the build.
> > > > > > > > >> > > > > >> >
> > > > > > > > >> > > > > >> > Regards,
> > > > > > > > >> > > > > >> > Saikat
> > > > > > > > >> > > > > >> >
> > > > > > > > >> > > > > >> > On Sat, Nov 23, 2019 at 9:52 AM Saikat
> Maitra <
> > > > > > > > >> > > > > saikat.maitra@gmail.com>
> > > > > > > > >> > > > > >> > wrote:
> > > > > > > > >> > > > > >> >
> > > > > > > > >> > > > > >> >> Hi Emmanouil,
> > > > > > > > >> > > > > >> >>
> > > > > > > > >> > > > > >> >> Thank you for your email. Yes, the plan is
> > once
> > > > the
> > > > > > > Flink
> > > > > > > > >> PR is
> > > > > > > > >> > > > > merged
> > > > > > > > >> > > > > >> it
> > > > > > > > >> > > > > >> >> will provide the Licence, parent POMs etc
> and
> > > you
> > > > > can
> > > > > > > > rebase
> > > > > > > > >> > from
> > > > > > > > >> > > > > >> master
> > > > > > > > >> > > > > >> >> branch and apply the changes on top of it.
> > > > > > > > >> > > > > >> >>
> > > > > > > > >> > > > > >> >> I am getting some issues with test failures
> in
> > > > local
> > > > > > > with
> > > > > > > > >> > > > > >> >> GridTestProperties as the test.properties is
> > not
> > > > > > present
> > > > > > > > in
> > > > > > > > >> > this
> > > > > > > > >> > > > new
> > > > > > > > >> > > > > >> >> project but should be available from
> > > dependencies.
> > > > > > > > >> > > > > >> >>
> > > > > > > > >> > > > > >> >> Once I address this issue, I will go ahead
> and
> > > > merge
> > > > > > the
> > > > > > > > >> > changes
> > > > > > > > >> > > > and
> > > > > > > > >> > > > > >> then
> > > > > > > > >> > > > > >> >> we can take it from there.
> > > > > > > > >> > > > > >> >>
> > > > > > > > >> > > > > >> >> Regards,
> > > > > > > > >> > > > > >> >> Saikat
> > > > > > > > >> > > > > >> >>
> > > > > > > > >> > > > > >> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil
> > > > > Gkatziouras
> > > > > > <
> > > > > > > > >> > > > > >> >> gkatzioura@gmail.com> wrote:
> > > > > > > > >> > > > > >> >>
> > > > > > > > >> > > > > >> >>> Hi all,
> > > > > > > > >> > > > > >> >>>
> > > > > > > > >> > > > > >> >>> I made my first pull request [1]. Since
> this
> > > > > project
> > > > > > is
> > > > > > > > >> brand
> > > > > > > > >> > > new
> > > > > > > > >> > > > > (no
> > > > > > > > >> > > > > >> >>> parent poms, licensing), I reckoned it was
> > best
> > > > to
> > > > > > use
> > > > > > > > >> > Saikat's
> > > > > > > > >> > > > > >> branch on
> > > > > > > > >> > > > > >> >>> Flink.
> > > > > > > > >> > > > > >> >>> I suppose the Flink branch will be merged
> > > first.
> > > > If
> > > > > > not
> > > > > > > > >> please
> > > > > > > > >> > > > give
> > > > > > > > >> > > > > me
> > > > > > > > >> > > > > >> >>> guidelines on how I should proceed next.
> > > > > > > > >> > > > > >> >>>
> > > > > > > > >> > > > > >> >>> Kind regards
> > > > > > > > >> > > > > >> >>> Emmanouil
> > > > > > > > >> > > > > >> >>>
> > > > > > > > >> > > > > >> >>> [1]
> > > > > > https://github.com/apache/ignite-extensions/pull/2
> > > > > > > > >> > > > > >> >>>
> > > > > > > > >> > > > > >> >>> *Emmanouil Gkatziouras*
> > > > > > > > >> > > > > >> >>> https://egkatzioura.com/ |
> > > > > > > > >> > > > > >> >>>
> > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > > > > >> > > > > >> >>> https://github.com/gkatzioura
> > > > > > > > >> > > > > >> >>>
> > > > > > > > >> > > > > >> >>>
> > > > > > > > >> > > > > >> >>> On Fri, 22 Nov 2019 at 20:55, Denis Magda <
> > > > > > > > >> dmagda@apache.org>
> > > > > > > > >> > > > > wrote:
> > > > > > > > >> > > > > >> >>>
> > > > > > > > >> > > > > >> >>> > Awesome, ping us whenever you're ready!
> > > > > > > > >> > > > > >> >>> >
> > > > > > > > >> > > > > >> >>> > -
> > > > > > > > >> > > > > >> >>> > Denis
> > > > > > > > >> > > > > >> >>> >
> > > > > > > > >> > > > > >> >>> >
> > > > > > > > >> > > > > >> >>> > On Fri, Nov 22, 2019 at 12:52 PM
> Emmanouil
> > > > > > > Gkatziouras
> > > > > > > > <
> > > > > > > > >> > > > > >> >>> > gkatzioura@gmail.com>
> > > > > > > > >> > > > > >> >>> > wrote:
> > > > > > > > >> > > > > >> >>> >
> > > > > > > > >> > > > > >> >>> > > Hi all!
> > > > > > > > >> > > > > >> >>> > >
> > > > > > > > >> > > > > >> >>> > > I am sorry for being late on that. I
> was
> > > > trying
> > > > > > to
> > > > > > > > >> > refactor
> > > > > > > > >> > > > the
> > > > > > > > >> > > > > >> test
> > > > > > > > >> > > > > >> >>> in
> > > > > > > > >> > > > > >> >>> > > order not to be in need of any external
> > > tools
> > > > > or
> > > > > > > > >> spinning
> > > > > > > > >> > > up a
> > > > > > > > >> > > > > >> >>> server.
> > > > > > > > >> > > > > >> >>> > > I did forked the new repo and indeed my
> > > > changes
> > > > > > > > there,
> > > > > > > > >> so
> > > > > > > > >> > a
> > > > > > > > >> > > > pull
> > > > > > > > >> > > > > >> >>> request
> > > > > > > > >> > > > > >> >>> > is
> > > > > > > > >> > > > > >> >>> > > a matter of time!
> > > > > > > > >> > > > > >> >>> > >
> > > > > > > > >> > > > > >> >>> > > Kind regards
> > > > > > > > >> > > > > >> >>> > >
> > > > > > > > >> > > > > >> >>> > > *Emmanouil Gkatziouras*
> > > > > > > > >> > > > > >> >>> > > https://egkatzioura.com/ |
> > > > > > > > >> > > > > >> >>> > >
> > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > > > > >> > > > > >> >>> > > https://github.com/gkatzioura
> > > > > > > > >> > > > > >> >>> > >
> > > > > > > > >> > > > > >> >>> > >
> > > > > > > > >> > > > > >> >>> > > On Fri, 22 Nov 2019 at 20:45, Denis
> > Magda <
> > > > > > > > >> > > dmagda@apache.org>
> > > > > > > > >> > > > > >> wrote:
> > > > > > > > >> > > > > >> >>> > >
> > > > > > > > >> > > > > >> >>> > > > Hi Emmanouil,
> > > > > > > > >> > > > > >> >>> > > >
> > > > > > > > >> > > > > >> >>> > > > Do you have any questions or need any
> > > > support
> > > > > > > from
> > > > > > > > >> the
> > > > > > > > >> > > > > >> community?
> > > > > > > > >> > > > > >> >>> > > >
> > > > > > > > >> > > > > >> >>> > > > -
> > > > > > > > >> > > > > >> >>> > > > Denis
> > > > > > > > >> > > > > >> >>> > > >
> > > > > > > > >> > > > > >> >>> > > >
> > > > > > > > >> > > > > >> >>> > > > On Sun, Nov 10, 2019 at 3:07 PM
> Saikat
> > > > > Maitra <
> > > > > > > > >> > > > > >> >>> saikat.maitra@gmail.com
> > > > > > > > >> > > > > >> >>> > >
> > > > > > > > >> > > > > >> >>> > > > wrote:
> > > > > > > > >> > > > > >> >>> > > >
> > > > > > > > >> > > > > >> >>> > > >> Hi Emmanouil,
> > > > > > > > >> > > > > >> >>> > > >>
> > > > > > > > >> > > > > >> >>> > > >> Can you please take a looks at dev
> > > utils,
> > > > if
> > > > > > > this
> > > > > > > > is
> > > > > > > > >> > > > > something
> > > > > > > > >> > > > > >> >>> you are
> > > > > > > > >> > > > > >> >>> > > >> looking for
> > > > > > > > >> > > > > >> >>> > > >>
> > > > > > > > >> > > > >
> > > > > > > https://github.com/apache/ignite/tree/master/modules/dev-utils
> > > > > > > > >> > > > > >> ?
> > > > > > > > >> > > > > >> >>> > > >>
> > > > > > > > >> > > > > >> >>> > > >> IMO, if you can release Pub/Sub
> server
> > > in
> > > > > > maven
> > > > > > > > and
> > > > > > > > >> > then
> > > > > > > > >> > > > use
> > > > > > > > >> > > > > >> it as
> > > > > > > > >> > > > > >> >>> > > >> dependency, that would be great.
> > > > > > > > >> > > > > >> >>> > > >>
> > > > > > > > >> > > > > >> >>> > > >> Regards,
> > > > > > > > >> > > > > >> >>> > > >> Saikat
> > > > > > > > >> > > > > >> >>> > > >>
> > > > > > > > >> > > > > >> >>> > > >> On Sun, Nov 10, 2019 at 5:00 PM
> Saikat
> > > > > Maitra
> > > > > > <
> > > > > > > > >> > > > > >> >>> > saikat.maitra@gmail.com>
> > > > > > > > >> > > > > >> >>> > > >> wrote:
> > > > > > > > >> > > > > >> >>> > > >>
> > > > > > > > >> > > > > >> >>> > > >> > Hi Emmanouil,
> > > > > > > > >> > > > > >> >>> > > >> >
> > > > > > > > >> > > > > >> >>> > > >> > The master branch has been
> > > initialized.
> > > > I
> > > > > > have
> > > > > > > > >> > raised a
> > > > > > > > >> > > > PR
> > > > > > > > >> > > > > to
> > > > > > > > >> > > > > >> >>> > migrate
> > > > > > > > >> > > > > >> >>> > > >> > Flink module to ignite-extensions.
> > > > > > > > >> > > > > >> >>> > > >> >
> > > > > > > > >> > > > > >> >>> > > >> > PR
> > > > > > > > >> > https://github.com/apache/ignite-extensions/pull/1
> > > > > > > > >> > > > > >> >>> > > >> > Jira :
> > > > > > > > >> > > > https://issues.apache.org/jira/browse/IGNITE-12356
> > > > > > > > >> > > > > >> >>> > > >> >
> > > > > > > > >> > > > > >> >>> > > >> > Regards,
> > > > > > > > >> > > > > >> >>> > > >> > Saikat
> > > > > > > > >> > > > > >> >>> > > >> >
> > > > > > > > >> > > > > >> >>> > > >> >
> > > > > > > > >> > > > > >> >>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM
> > > > Emmanouil
> > > > > > > > >> > Gkatziouras <
> > > > > > > > >> > > > > >> >>> > > >> > gkatzioura@gmail.com> wrote:
> > > > > > > > >> > > > > >> >>> > > >> >
> > > > > > > > >> > > > > >> >>> > > >> >> Hi all!
> > > > > > > > >> > > > > >> >>> > > >> >>
> > > > > > > > >> > > > > >> >>> > > >> >> This is an update. The
> integration
> > is
> > > > > > tested
> > > > > > > > with
> > > > > > > > >> > > mocks
> > > > > > > > >> > > > > >> based
> > > > > > > > >> > > > > >> >>> on
> > > > > > > > >> > > > > >> >>> > the
> > > > > > > > >> > > > > >> >>> > > >> >> original Ignite repo [1].
> > > > > > > > >> > > > > >> >>> > > >> >> I am currently waiting for the
> > first
> > > > > > commits
> > > > > > > on
> > > > > > > > >> the
> > > > > > > > >> > > new
> > > > > > > > >> > > > > >> >>> integration
> > > > > > > > >> > > > > >> >>> > > >> >> project
> > > > > > > > >> > > > > >> >>> > > >> >> to take place in order to create
> my
> > > > pull
> > > > > > > > request
> > > > > > > > >> > > > > accordingly
> > > > > > > > >> > > > > >> >>> to the
> > > > > > > > >> > > > > >> >>> > > >> >> initial
> > > > > > > > >> > > > > >> >>> > > >> >> commits.
> > > > > > > > >> > > > > >> >>> > > >> >> In order to simulate the Pub/Sub
> > > server
> > > > > in
> > > > > > a
> > > > > > > > >> test I
> > > > > > > > >> > > > used a
> > > > > > > > >> > > > > >> test
> > > > > > > > >> > > > > >> >>> > > server
> > > > > > > > >> > > > > >> >>> > > >> >> implementation [2]. Unfortunately
> > > they
> > > > > > don't
> > > > > > > > have
> > > > > > > > >> > this
> > > > > > > > >> > > > > util
> > > > > > > > >> > > > > >> >>> > uploaded
> > > > > > > > >> > > > > >> >>> > > >> in a
> > > > > > > > >> > > > > >> >>> > > >> >> maven repo, thus it needs to be
> > > hosted
> > > > > on a
> > > > > > > > maven
> > > > > > > > >> > repo
> > > > > > > > >> > > > > >> (Apache
> > > > > > > > >> > > > > >> >>> > > License
> > > > > > > > >> > > > > >> >>> > > >> >> 2.0). Is there a repo for
> utilities
> > > > that
> > > > > > > Ignite
> > > > > > > > >> uses
> > > > > > > > >> > > for
> > > > > > > > >> > > > > the
> > > > > > > > >> > > > > >> >>> > > >> test/build?
> > > > > > > > >> > > > > >> >>> > > >> >> I have also created a ticket [3]
> > > > > > > > >> > > > > >> >>> > > >> >>
> > > > > > > > >> > > > > >> >>> > > >> >> Kind regards
> > > > > > > > >> > > > > >> >>> > > >> >>
> > > > > > > > >> > > > > >> >>> > > >> >> [1]
> > > > > > > > >> > > > > >> >>> >
> > > > > > > > >> > >
> > > > > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > > > > > > > >> > > > > >> >>> > > >> >> [2]
> > > > > > > > >> > > > > >> >>>
> > > > > > > > >>
> > https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> > > > > > > > >> > > > > >> >>> > > >> >> [3]
> > > > > > > > >> > > https://issues.apache.org/jira/browse/IGNITE-12262
> > > > > > > > >> > > > > >> >>> > > >> >>
> > > > > > > > >> > > > > >> >>> > > >> >> *Emmanouil Gkatziouras*
> > > > > > > > >> > > > > >> >>> > > >> >> https://egkatzioura.com/ |
> > > > > > > > >> > > > > >> >>> > > >> >>
> > > > > > > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > > > > >> > > > > >> >>> > > >> >> https://github.com/gkatzioura
> > > > > > > > >> > > > > >> >>> > > >> >>
> > > > > > > > >> > > > > >> >>> > > >> >>
> > > > > > > > >> > > > > >> >>> > > >> >> On Fri, 25 Oct 2019 at 03:35,
> > Saikat
> > > > > > Maitra <
> > > > > > > > >> > > > > >> >>> > saikat.maitra@gmail.com
> > > > > > > > >> > > > > >> >>> > > >
> > > > > > > > >> > > > > >> >>> > > >> >> wrote:
> > > > > > > > >> > > > > >> >>> > > >> >>
> > > > > > > > >> > > > > >> >>> > > >> >> > Hello Ilya,
> > > > > > > > >> > > > > >> >>> > > >> >> >
> > > > > > > > >> > > > > >> >>> > > >> >> > Thank you for your email. Yes,
> > > before
> > > > > we
> > > > > > > > remove
> > > > > > > > >> > any
> > > > > > > > >> > > > > >> >>> integration
> > > > > > > > >> > > > > >> >>> > we
> > > > > > > > >> > > > > >> >>> > > >> will
> > > > > > > > >> > > > > >> >>> > > >> >> > move them to another repository
> > and
> > > > > > > release.
> > > > > > > > >> > > > > >> >>> > > >> >> >
> > > > > > > > >> > > > > >> >>> > > >> >> > We are thinking we will start
> > with
> > > > > > Pub/Sub
> > > > > > > > >> > Streamer
> > > > > > > > >> > > > > >> >>> > implementation
> > > > > > > > >> > > > > >> >>> > > >> and
> > > > > > > > >> > > > > >> >>> > > >> >> > continue to move other
> > integration
> > > as
> > > > > > > > >> mentioned in
> > > > > > > > >> > > the
> > > > > > > > >> > > > > >> >>> following
> > > > > > > > >> > > > > >> >>> > > doc.
> > > > > > > > >> > > > > >> >>> > > >> >> >
> > > > > > > > >> > > > > >> >>> > > >> >> >
> > > > > > > > >> > > > > >> >>> > > >> >> >
> > > > > > > > >> > > > > >> >>> > > >> >>
> > > > > > > > >> > > > > >> >>> > > >>
> > > > > > > > >> > > > > >> >>> > >
> > > > > > > > >> > > > > >> >>> >
> > > > > > > > >> > > > > >> >>>
> > > > > > > > >> > > > > >>
> > > > > > > > >> > > > >
> > > > > > > > >> > > >
> > > > > > > > >> > >
> > > > > > > > >> >
> > > > > > > > >>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > > > > > >> > > > > >> >>> > > >> >> >
> > > > > > > > >> > > > > >> >>> > > >> >> > Regards,
> > > > > > > > >> > > > > >> >>> > > >> >> > Saikat
> > > > > > > > >> > > > > >> >>> > > >> >> >
> > > > > > > > >> > > > > >> >>> > > >> >> >
> > > > > > > > >> > > > > >> >>> > > >> >> > On Thu, Oct 24, 2019 at 11:00
> AM
> > > Ilya
> > > > > > > > >> Kasnacheev <
> > > > > > > > >> > > > > >> >>> > > >> >> > ilya.kasnacheev@gmail.com>
> > > > > > > > >> > > > > >> >>> > > >> >> > wrote:
> > > > > > > > >> > > > > >> >>> > > >> >> >
> > > > > > > > >> > > > > >> >>> > > >> >> > > Hello!
> > > > > > > > >> > > > > >> >>> > > >> >> > >
> > > > > > > > >> > > > > >> >>> > > >> >> > > My take on this: we want to
> > > offload
> > > > > > some
> > > > > > > of
> > > > > > > > >> > > > > >> integrations,
> > > > > > > > >> > > > > >> >>> but
> > > > > > > > >> > > > > >> >>> > > >> please
> > > > > > > > >> > > > > >> >>> > > >> >> > commit
> > > > > > > > >> > > > > >> >>> > > >> >> > > and release them to another
> > > project
> > > > > > > first,
> > > > > > > > >> and
> > > > > > > > >> > > only
> > > > > > > > >> > > > > >> then we
> > > > > > > > >> > > > > >> >>> > will
> > > > > > > > >> > > > > >> >>> > > >> >> decide
> > > > > > > > >> > > > > >> >>> > > >> >> > to
> > > > > > > > >> > > > > >> >>> > > >> >> > > remove them from our
> > repository.
> > > > > > > > >> > > > > >> >>> > > >> >> > >
> > > > > > > > >> > > > > >> >>> > > >> >> > > Please also make sure they
> are
> > > > > > available
> > > > > > > > with
> > > > > > > > >> > same
> > > > > > > > >> > > > > >> artifact
> > > > > > > > >> > > > > >> >>> > name
> > > > > > > > >> > > > > >> >>> > > >> and
> > > > > > > > >> > > > > >> >>> > > >> >> API.
> > > > > > > > >> > > > > >> >>> > > >> >> > > If this has to be changed,
> > let's
> > > > wait
> > > > > > for
> > > > > > > > AI
> > > > > > > > >> 3.0
> > > > > > > > >> > > > > >> >>> > > >> >> > >
> > > > > > > > >> > > > > >> >>> > > >> >> > > Regards,
> > > > > > > > >> > > > > >> >>> > > >> >> > > --
> > > > > > > > >> > > > > >> >>> > > >> >> > > Ilya Kasnacheev
> > > > > > > > >> > > > > >> >>> > > >> >> > >
> > > > > > > > >> > > > > >> >>> > > >> >> > >
> > > > > > > > >> > > > > >> >>> > > >> >> > > ср, 23 окт. 2019 г. в 03:23,
> > > Saikat
> > > > > > > Maitra
> > > > > > > > <
> > > > > > > > >> > > > > >> >>> > > >> saikat.maitra@gmail.com>:
> > > > > > > > >> > > > > >> >>> > > >> >> > >
> > > > > > > > >> > > > > >> >>> > > >> >> > > > Hello Emmanouil,
> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > > >> > > > > >> >>> > > >> >> > > > As discussed earlier, I
> > > discussed
> > > > > > with
> > > > > > > > >> Apache
> > > > > > > > >> > > > Bahir
> > > > > > > > >> > > > > >> >>> community
> > > > > > > > >> > > > > >> >>> > > and
> > > > > > > > >> > > > > >> >>> > > >> >> they
> > > > > > > > >> > > > > >> >>> > > >> >> > > are
> > > > > > > > >> > > > > >> >>> > > >> >> > > > interested to have Apache
> > > Ignite
> > > > > > > > >> extensions as
> > > > > > > > >> > > > part
> > > > > > > > >> > > > > of
> > > > > > > > >> > > > > >> >>> Apache
> > > > > > > > >> > > > > >> >>> > > >> Bahir
> > > > > > > > >> > > > > >> >>> > > >> >> > > > project.
> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > > >> > > > > >> >>> > > >> >> > > > Can you please share
> Pub/Sub
> > > > > streamer
> > > > > > > > >> > > > implementation
> > > > > > > > >> > > > > >> >>> details
> > > > > > > > >> > > > > >> >>> > > with
> > > > > > > > >> > > > > >> >>> > > >> >> > Apache
> > > > > > > > >> > > > > >> >>> > > >> >> > > > Bahir community and request
> > for
> > > > > > > feedback.
> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > > >> > > > > >> >>> > > >> >> > > > https://bahir.apache.org/
> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > > >> > > > > >> >>> > > >> >> > > > I have also requested for
> > > > > contributor
> > > > > > > > >> access
> > > > > > > > >> > in
> > > > > > > > >> > > > Jira
> > > > > > > > >> > > > > >> for
> > > > > > > > >> > > > > >> >>> > Apache
> > > > > > > > >> > > > > >> >>> > > >> >> Bahir
> > > > > > > > >> > > > > >> >>> > > >> >> > > > project so that I can
> create
> > > > issues
> > > > > > and
> > > > > > > > >> assign
> > > > > > > > >> > > to
> > > > > > > > >> > > > > >> myself.
> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > > >> > > > > >> >>> > > >> >> > > > I can help with code
> reviews
> > as
> > > > > well.
> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > > >> > > > > >> >>> > > >> >> > > > Here is the email thread
> for
> > > > > > reference
> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > > >> > > > > >> >>> > >
> > > > > > > > >> > > >
> > > > > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > > >> > > > > >> >>> > > >> >> > > > Regards,
> > > > > > > > >> > > > > >> >>> > > >> >> > > > Saikat
> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > > >> > > > > >> >>> > > >> >> > > > On Mon, Oct 21, 2019 at
> 7:54
> > PM
> > > > > > Saikat
> > > > > > > > >> Maitra
> > > > > > > > >> > <
> > > > > > > > >> > > > > >> >>> > > >> >> saikat.maitra@gmail.com
> > > > > > > > >> > > > > >> >>> > > >> >> > >
> > > > > > > > >> > > > > >> >>> > > >> >> > > > wrote:
> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > > >> > > > > >> >>> > > >> >> > > > > Hello,
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > > > >> > > > > >> >>> > > >> >> > > > > As discussed I have
> created
> > > > > > following
> > > > > > > > >> > > discussion
> > > > > > > > >> > > > > >> >>> threads on
> > > > > > > > >> > > > > >> >>> > > our
> > > > > > > > >> > > > > >> >>> > > >> >> > > migration
> > > > > > > > >> > > > > >> >>> > > >> >> > > > > proposals for Apache
> Ignite
> > > > > > > extensions:
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > > >> > > > > >> >>> > > >> >> > >
> > > > > > > > >> > > > > >> >>> > > >> >> >
> > > > > > > > >> > > > > >> >>> > > >> >>
> > > > > > > > >> > > > > >> >>> > > >>
> > > > > > > > >> > > > > >> >>> > >
> > > > > > > > >> > > > > >> >>> >
> > > > > > > > >> > > > > >> >>>
> > > > > > > > >> > > > > >>
> > > > > > > > >> > > > >
> > > > > > > > >> > > >
> > > > > > > > >> > >
> > > > > > > > >> >
> > > > > > > > >>
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > > > >> > > > > >> >>> > > >>
> > > > > > > > >> > > > > >>
> > > > > > > > >>
> > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > > > >> > > > > >> >>> > > >> >> > > > > I will share update as I
> > hear
> > > > > more
> > > > > > > > >> > > information.
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > > > >> > > > > >> >>> > > >> >> > > > > Regards,
> > > > > > > > >> > > > > >> >>> > > >> >> > > > > Saikat
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > > > >> > > > > >> >>> > > >> >> > > > > On Fri, Oct 18, 2019 at
> > 9:03
> > > PM
> > > > > > > Saikat
> > > > > > > > >> > Maitra
> > > > > > > > >> > > <
> > > > > > > > >> > > > > >> >>> > > >> >> > saikat.maitra@gmail.com
> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > > >> > > > > >> >>> > > >> >> > > > > wrote:
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >> Hello Denis,
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >> Sure, sounds good. I
> will
> > > > > create a
> > > > > > > > >> separate
> > > > > > > > >> > > > > >> discussion
> > > > > > > > >> > > > > >> >>> > > thread
> > > > > > > > >> > > > > >> >>> > > >> to
> > > > > > > > >> > > > > >> >>> > > >> >> get
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >> community feedback on
> > > whether
> > > > we
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >> should join the Bahir or
> > > > > kick-off
> > > > > > > > >> "Ignite
> > > > > > > > >> > > > > >> Extensions"
> > > > > > > > >> > > > > >> >>> > > project.
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >> Regards,
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >> Saikat
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >> On Thu, Oct 17, 2019 at
> > 2:21
> > > > PM
> > > > > > > Denis
> > > > > > > > >> > Magda <
> > > > > > > > >> > > > > >> >>> > > >> dmagda@apache.org>
> > > > > > > > >> > > > > >> >>> > > >> >> > > wrote:
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> Folks,
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> The concept of Apache
> > Bahir
> > > > is
> > > > > > > > >> precisely
> > > > > > > > >> > > what
> > > > > > > > >> > > > we
> > > > > > > > >> > > > > >> >>> need!
> > > > > > > > >> > > > > >> >>> > > >> Saikat,
> > > > > > > > >> > > > > >> >>> > > >> >> > thanks
> > > > > > > > >> > > > > >> >>> > > >> >> > > > for
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> doing the research.
> Why I
> > > > > believe
> > > > > > > > the *
> > > > > > > > >> > > idea*
> > > > > > > > >> > > > > >> fits us
> > > > > > > > >> > > > > >> >>> > well:
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    - All integrations
> can
> > > be
> > > > > > stored
> > > > > > > > in
> > > > > > > > >> > > > separate
> > > > > > > > >> > > > > >> >>> Github
> > > > > > > > >> > > > > >> >>> > > >> >> repositories
> > > > > > > > >> > > > > >> >>> > > >> >> > > and
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    have their dev
> > > lifecycles.
> > > > > > We've
> > > > > > > > not
> > > > > > > > >> > > > obliged
> > > > > > > > >> > > > > to
> > > > > > > > >> > > > > >> >>> couple
> > > > > > > > >> > > > > >> >>> > > all
> > > > > > > > >> > > > > >> >>> > > >> >> the
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> integrations
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    in a single repo.
> For
> > > > > > instance,
> > > > > > > > >> Spark
> > > > > > > > >> > can
> > > > > > > > >> > > > be
> > > > > > > > >> > > > > >> >>> located
> > > > > > > > >> > > > > >> >>> > in
> > > > > > > > >> > > > > >> >>> > > a
> > > > > > > > >> > > > > >> >>> > > >> >> > > dedicated
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> repo
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    while streaming
> > > > integrations
> > > > > > > might
> > > > > > > > >> be
> > > > > > > > >> > in
> > > > > > > > >> > > a
> > > > > > > > >> > > > > >> single
> > > > > > > > >> > > > > >> >>> one.
> > > > > > > > >> > > > > >> >>> > > >> It's
> > > > > > > > >> > > > > >> >>> > > >> >> up
> > > > > > > > >> > > > > >> >>> > > >> >> > to
> > > > > > > > >> > > > > >> >>> > > >> >> > > > us.
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    - All the
> repositories
> > > and
> > > > > > > > >> integrations
> > > > > > > > >> > > > > belong
> > > > > > > > >> > > > > >> to
> > > > > > > > >> > > > > >> >>> ASF.
> > > > > > > > >> > > > > >> >>> > > >> We're
> > > > > > > > >> > > > > >> >>> > > >> >> not
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> dumping
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    them on Github but
> > > rather
> > > > > keep
> > > > > > > > >> > supporting
> > > > > > > > >> > > > and
> > > > > > > > >> > > > > >> >>> > developing
> > > > > > > > >> > > > > >> >>> > > >> in
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> accordance with
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    ASF vision and
> > > practices.
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    - There is a way to
> > > reward
> > > > > > > > >> contributors
> > > > > > > > >> > > via
> > > > > > > > >> > > > > >> >>> > > committership
> > > > > > > > >> > > > > >> >>> > > >> and
> > > > > > > > >> > > > > >> >>> > > >> >> > PMC
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    membership. You
> won't
> > > get
> > > > > this
> > > > > > > if
> > > > > > > > a
> > > > > > > > >> > > project
> > > > > > > > >> > > > > is
> > > > > > > > >> > > > > >> >>> just
> > > > > > > > >> > > > > >> >>> > one
> > > > > > > > >> > > > > >> >>> > > of
> > > > > > > > >> > > > > >> >>> > > >> >> the
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> millions of
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    Github projects.
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> Saikat, as Ignite PMC
> > > member,
> > > > > > could
> > > > > > > > you
> > > > > > > > >> > > please
> > > > > > > > >> > > > > >> >>> kick-off a
> > > > > > > > >> > > > > >> >>> > > >> >> separate
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> dev-list
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> discussion to involve
> > more
> > > > > > > community
> > > > > > > > >> > members
> > > > > > > > >> > > > and
> > > > > > > > >> > > > > >> >>> > referring
> > > > > > > > >> > > > > >> >>> > > to
> > > > > > > > >> > > > > >> >>> > > >> >> this
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> thread?
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> I think the primary
> > > question
> > > > > the
> > > > > > > > >> community
> > > > > > > > >> > > > needs
> > > > > > > > >> > > > > >> to
> > > > > > > > >> > > > > >> >>> > answer
> > > > > > > > >> > > > > >> >>> > > >> >> whether
> > > > > > > > >> > > > > >> >>> > > >> >> > we
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> should join the Bahir
> or
> > > > > kick-off
> > > > > > > > >> "Ignite
> > > > > > > > >> > > > > >> Extensions"
> > > > > > > > >> > > > > >> >>> > > >> project?
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> -
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> Denis
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> On Thu, Oct 17, 2019 at
> > > 2:54
> > > > AM
> > > > > > > > Alexey
> > > > > > > > >> > > > Zinoviev
> > > > > > > > >> > > > > <
> > > > > > > > >> > > > > >> >>> > > >> >> > > > zaleslaw.sin@gmail.com>
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> wrote:
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > Maybe we could move
> all
> > > our
> > > > > > > > Streaming
> > > > > > > > >> > > > > >> Integrations
> > > > > > > > >> > > > > >> >>> > there,
> > > > > > > > >> > > > > >> >>> > > >> but
> > > > > > > > >> > > > > >> >>> > > >> >> > what
> > > > > > > > >> > > > > >> >>> > > >> >> > > is
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> about
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > maintaining and
> > committer
> > > > > > > > >> permissions to
> > > > > > > > >> > > the
> > > > > > > > >> > > > > new
> > > > > > > > >> > > > > >> >>> > > >> repositories?
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > I see the list of the
> > > > > > committers
> > > > > > > > and
> > > > > > > > >> PMC
> > > > > > > > >> > > > > members
> > > > > > > > >> > > > > >> >>> there
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > > > > > >> > > https://bahir.apache.org/community-members/
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > Could somebody from
> > > Ignite
> > > > > > > > community
> > > > > > > > >> be
> > > > > > > > >> > > > added
> > > > > > > > >> > > > > to
> > > > > > > > >> > > > > >> >>> this
> > > > > > > > >> > > > > >> >>> > > list
> > > > > > > > >> > > > > >> >>> > > >> as
> > > > > > > > >> > > > > >> >>> > > >> >> a
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > PMC/committer to
> > maintain
> > > > > > Ignite
> > > > > > > > >> > > > integrations?
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > If yes, I'd happy to
> > join
> > > > > this
> > > > > > > > >> project
> > > > > > > > >> > and
> > > > > > > > >> > > > > >> >>> collaborate
> > > > > > > > >> > > > > >> >>> > > with
> > > > > > > > >> > > > > >> >>> > > >> >> these
> > > > > > > > >> > > > > >> >>> > > >> >> > > > guys
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > together
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > If no, and we should
> > > start
> > > > > from
> > > > > > > the
> > > > > > > > >> zero
> > > > > > > > >> > > > level
> > > > > > > > >> > > > > >> with
> > > > > > > > >> > > > > >> >>> > > >> external
> > > > > > > > >> > > > > >> >>> > > >> >> PRs
> > > > > > > > >> > > > > >> >>> > > >> >> > -
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> hmmm,
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > it's better to have
> our
> > > own
> > > > > > > > external
> > > > > > > > >> > > > > repository
> > > > > > > > >> > > > > >> >>> with
> > > > > > > > >> > > > > >> >>> > > >> >> ApacheBahirr
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> ideology.
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > Also, I agree, that
> all
> > > > > > > connectors
> > > > > > > > >> could
> > > > > > > > >> > > be
> > > > > > > > >> > > > > >> moved
> > > > > > > > >> > > > > >> >>> there
> > > > > > > > >> > > > > >> >>> > > (in
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> ApacheBahirr
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > like repository), but
> > not
> > > > all
> > > > > > > > modules
> > > > > > > > >> > from
> > > > > > > > >> > > > the
> > > > > > > > >> > > > > >> page
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > > >> > > > > >> >>> > > >> >> > >
> > > > > > > > >> > > > > >> >>> > > >> >> >
> > > > > > > > >> > >
>

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Emmanouil Gkatziouras <gk...@gmail.com>]

Hi Saikat!

The dependencies have been updated, and the merge conflicts resolved.
Kind regards

On Sat, 28 Dec 2019, 23:27 Saikat Maitra, <sa...@gmail.com> wrote:

> Hi Emmanouil,
>
> Thank you for making the changes. I have merged my PR changes in master.
> Can you please take the latest from master branch and resolve the conflicts
> in your PR.
>
> I have also reviewed the changes in your PR and requested minor changes
> specific to dependencies version and license information.
>
> Can you please review and share your feedback?
>
> Regards,
> Saikat
>
>
>
>
> On Sat, Dec 28, 2019 at 4:38 AM Emmanouil Gkatziouras <
> gkatzioura@gmail.com>
> wrote:
>
> > Hi Saikat!
> >
> > Thank you on this one! I tried it and it worked as expected [1]
> >
> > Kind regards
> > Emmanouil
> >
> > [1]
> >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4888357&tab=buildLog&_focus=2046#_state=2046
> >
> > *Emmanouil Gkatziouras*
> > https://egkatzioura.com/ |
> > https://www.linkedin.com/in/gkatziourasemmanouil/
> > https://github.com/gkatzioura
> >
> >
> > On Thu, 26 Dec 2019 at 19:50, Saikat Maitra <sa...@gmail.com>
> > wrote:
> >
> > > Hi Emmanouil,
> > >
> > > I have been able to resolve the build failures for missing modules.
> > >
> > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4883394&buildTypeId=IgniteExtensions_Build
> > >
> > > Will you please take the latest changes from my PR and run the build
> for
> > > your PR.
> > >
> > > https://github.com/apache/ignite-extensions/pull/1/files
> > >
> > > The new build configurations should let your module build also pass.
> > >
> > > The issue that was causing failure was that I had put ignite.zip as
> > > artifact dependency and it was downloading the artifacts including pom
> > file
> > > inside the build checkout folder of ignite-extensions and as a result
> the
> > > pom.xml of ignite-extensions was getting overwritten by ignite's
> pom.xml.
> > > This was resulting in missing flink-ext module error as it was not even
> > > present in the ignite main pom.xml.
> > >
> > > To resolve the issue I just had to download and extract artifact of
> > > ignite.zip in a separate directory.
> > >
> > > Regards,
> > > Saikat
> > >
> > >
> > > On Sat, Nov 30, 2019 at 5:53 PM Emmanouil Gkatziouras <
> > > gkatzioura@gmail.com>
> > > wrote:
> > >
> > > > Hi Saikat!
> > > >
> > > > Thank you for your assistance on that!
> > > >
> > > > Kind regards
> > > >
> > > > *Emmanouil Gkatziouras*
> > > > https://egkatzioura.com/ |
> > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > https://github.com/gkatzioura
> > > >
> > > >
> > > > On Sat, 30 Nov 2019 at 23:51, Saikat Maitra <saikat.maitra@gmail.com
> >
> > > > wrote:
> > > >
> > > > > Hi Emmanouil,
> > > > >
> > > > > I looked into the build logs and I observed that since I added
> > > artifacts
> > > > > dependencies to "Build apache Ignite"[1] it was able to pull the
> > > required
> > > > > dependencies and was able to run the existing tests. It is however
> > not
> > > > > identifying new modules like I changed from flink to flink-ext or
> > > > pub-sub.
> > > > >
> > > > > [1]
> > > > >
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/admin/editDependencies.html?id=buildType:IgniteExtensions_Build
> > > > >
> > > > > I will look into the issue and debug further.
> > > > >
> > > > > Regards,
> > > > > Saikat
> > > > >
> > > > >
> > > > > On Sat, Nov 30, 2019 at 5:35 PM Emmanouil Gkatziouras <
> > > > > gkatzioura@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi Saikat!
> > > > > >
> > > > > > From the logs[1] it seems that TC cannot find the project.
> However
> > I
> > > > did
> > > > > > include the project on the parent pom (modules sections) [2]
> > > > > > I tried to reproduce locally unfortunately I am not aware of the
> > full
> > > > > > context on TC. It seems as if a different parent pom is being
> > picked
> > > > up.
> > > > > >
> > > > > > Kind regards
> > > > > >
> > > > > > [1]
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4804893&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=1704
> > > > > > [2]
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/gkatzioura/ignite-extensions/blob/d6a8beee7feff28f59d9a12be692016305564d25/pom.xml#L46
> > > > > >
> > > > > > *Emmanouil Gkatziouras*
> > > > > > https://egkatzioura.com/ |
> > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > > https://github.com/gkatzioura
> > > > > >
> > > > > >
> > > > > > On Sat, 30 Nov 2019 at 22:20, Saikat Maitra <
> > saikat.maitra@gmail.com
> > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hello,
> > > > > > >
> > > > > > > I have made changes in Ignite Extensions build configurations.
> > > > > > >
> > > > > > > It is now running the tests as expected.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4804480&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> > > > > > >
> > > > > > > Regards,
> > > > > > > Saikat
> > > > > > >
> > > > > > > On Thu, Nov 28, 2019 at 1:29 PM Saikat Maitra <
> > > > saikat.maitra@gmail.com
> > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > We do run build with -DskipTests and then configure selective
> > > test
> > > > > > suites
> > > > > > > > to run.
> > > > > > > >
> > > > > > > > I will check why these test suites are not getting executed.
> > > > > > > >
> > > > > > > > Regards
> > > > > > > > Saikat
> > > > > > > >
> > > > > > > > On Wed, 27 Nov 2019 at 3:05 PM, Emmanouil Gkatziouras <
> > > > > > > > gkatzioura@gmail.com> wrote:
> > > > > > > >
> > > > > > > >> Hi all,
> > > > > > > >>
> > > > > > > >> I did add a test suite and run the build with the
> > corresponding
> > > > > > > arguments,
> > > > > > > >> however the tests did not run [1].
> > > > > > > >> I checked the Flink's build logs and the same message is
> > > displayed
> > > > > > there
> > > > > > > >> too `No tests to run` [2].
> > > > > > > >>
> > > > > > > >> [1]
> > > > > > > >>
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4798203&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=248
> > > > > > > >> [2]
> > > > > > > >>
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4789199&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=1256
> > > > > > > >>
> > > > > > > >> Kind regards
> > > > > > > >> *Emmanouil Gkatziouras*
> > > > > > > >> https://egkatzioura.com/ |
> > > > > > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > > > >> https://github.com/gkatzioura
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> On Wed, 27 Nov 2019 at 18:58, Saikat Maitra <
> > > > > saikat.maitra@gmail.com>
> > > > > > > >> wrote:
> > > > > > > >>
> > > > > > > >> > Hi,
> > > > > > > >> >
> > > > > > > >> > We will need to add test suites for tests to be executed
> in
> > > > build.
> > > > > > We
> > > > > > > >> pass
> > > > > > > >> > them as TestSuites param. I had added Flink sink and
> source
> > > > > > > testsuites.
> > > > > > > >> >
> > > > > > > >> > Can you please review and confirm.
> > > > > > > >> >
> > > > > > > >> > Regards
> > > > > > > >> > Saikat
> > > > > > > >> >
> > > > > > > >> > On Wed, 27 Nov 2019 at 6:00 AM, Emmanouil Gkatziouras <
> > > > > > > >> > gkatzioura@gmail.com>
> > > > > > > >> > wrote:
> > > > > > > >> >
> > > > > > > >> > > Hi all!
> > > > > > > >> > >
> > > > > > > >> > > I was successful building and running my pull requests.
> It
> > > > seems
> > > > > > > that
> > > > > > > >> the
> > > > > > > >> > > tests do not run both for Flink and Pub/Sub [1]
> > > > > > > >> > > If there is something I cannot do to make them
> > discoverable
> > > > > please
> > > > > > > >> let me
> > > > > > > >> > > know.
> > > > > > > >> > >
> > > > > > > >> > > [1]
> > > > > > > >> > >
> > > > > > > >> > >
> > > > > > > >> >
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4796722&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=283
> > > > > > > >> > >
> > > > > > > >> > > *Emmanouil Gkatziouras*
> > > > > > > >> > > https://egkatzioura.com/ |
> > > > > > > >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > > > >> > > https://github.com/gkatzioura
> > > > > > > >> > >
> > > > > > > >> > >
> > > > > > > >> > > On Wed, 27 Nov 2019 at 04:57, Saikat Maitra <
> > > > > > > saikat.maitra@gmail.com>
> > > > > > > >> > > wrote:
> > > > > > > >> > >
> > > > > > > >> > > > Hi Emmanouil,
> > > > > > > >> > > >
> > > > > > > >> > > > I have added you as contributor in Ignite Extensions
> > > > project.
> > > > > > Can
> > > > > > > >> you
> > > > > > > >> > > > please check and confirm if you are able to see the
> > > project
> > > > > and
> > > > > > > >> execute
> > > > > > > >> > > > build on your pull request.
> > > > > > > >> > > >
> > > > > > > >> > > > Regards,
> > > > > > > >> > > > Saikat
> > > > > > > >> > > >
> > > > > > > >> > > > On Tue, Nov 26, 2019 at 12:49 PM Emmanouil
> Gkatziouras <
> > > > > > > >> > > > gkatzioura@gmail.com>
> > > > > > > >> > > > wrote:
> > > > > > > >> > > >
> > > > > > > >> > > > > Hi all!
> > > > > > > >> > > > >
> > > > > > > >> > > > > If someone could give me read access to the Job of
> > > Ignite
> > > > > > > >> Extensions
> > > > > > > >> > on
> > > > > > > >> > > > > Team City will greatly help me (username
> gkatzioura).
> > > > > > > >> > > > > I suppose the builds get triggered automatically on
> a
> > > pull
> > > > > > > >> request.
> > > > > > > >> > > > >
> > > > > > > >> > > > > Kind regards,
> > > > > > > >> > > > > Emmanouil
> > > > > > > >> > > > >
> > > > > > > >> > > > > *Emmanouil Gkatziouras*
> > > > > > > >> > > > > https://egkatzioura.com/ |
> > > > > > > >> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > > > >> > > > > https://github.com/gkatzioura
> > > > > > > >> > > > >
> > > > > > > >> > > > >
> > > > > > > >> > > > > On Sun, 24 Nov 2019 at 21:11, Emmanouil Gkatziouras
> <
> > > > > > > >> > > > gkatzioura@gmail.com>
> > > > > > > >> > > > > wrote:
> > > > > > > >> > > > >
> > > > > > > >> > > > > > Hi Saikat!
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > I just rebased with the flink branch.
> Unfortunately
> > I
> > > do
> > > > > not
> > > > > > > >> have
> > > > > > > >> > > read
> > > > > > > >> > > > > > access to the team city link you provided and thus
> > > > > evaluate
> > > > > > > the
> > > > > > > >> > tests
> > > > > > > >> > > > for
> > > > > > > >> > > > > > my pull request.
> > > > > > > >> > > > > > I guess it has to do with ignite extensions being
> > new.
> > > > > > > >> > > > > > My username is gkatzioura and email the one I am
> > using
> > > > on
> > > > > > this
> > > > > > > >> > email
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > Thank you for your time.
> > > > > > > >> > > > > >
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > *Emmanouil Gkatziouras*
> > > > > > > >> > > > > > https://egkatzioura.com/ |
> > > > > > > >> > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > > > >> > > > > > https://github.com/gkatzioura
> > > > > > > >> > > > > >
> > > > > > > >> > > > > >
> > > > > > > >> > > > > > On Sun, 24 Nov 2019 at 18:03, Saikat Maitra <
> > > > > > > >> > saikat.maitra@gmail.com
> > > > > > > >> > > >
> > > > > > > >> > > > > > wrote:
> > > > > > > >> > > > > >
> > > > > > > >> > > > > >> Hi Emmanouil,
> > > > > > > >> > > > > >>
> > > > > > > >> > > > > >> The latest build on teamcity has passed on Flink
> > pull
> > > > > > > request.
> > > > > > > >> > > > > >>
> > > > > > > >> > > > > >>
> > > > > > > >> > > > > >>
> > > > > > > >> > > > >
> > > > > > > >> > > >
> > > > > > > >> > >
> > > > > > > >> >
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4788928&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> > > > > > > >> > > > > >>
> > > > > > > >> > > > > >> you should be able to take the changes I made in
> my
> > > PR
> > > > > and
> > > > > > > run
> > > > > > > >> > build
> > > > > > > >> > > > on
> > > > > > > >> > > > > >> your pull request.
> > > > > > > >> > > > > >>
> > > > > > > >> > > > > >> Please let me know if you have any questions.
> > > > > > > >> > > > > >>
> > > > > > > >> > > > > >> Regards,
> > > > > > > >> > > > > >> Saikat
> > > > > > > >> > > > > >>
> > > > > > > >> > > > > >> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra <
> > > > > > > >> > > > saikat.maitra@gmail.com
> > > > > > > >> > > > > >
> > > > > > > >> > > > > >> wrote:
> > > > > > > >> > > > > >>
> > > > > > > >> > > > > >> > Hi Emmanouil,
> > > > > > > >> > > > > >> >
> > > > > > > >> > > > > >> > I have fixed the Flink module testsuites and
> have
> > > > setup
> > > > > > new
> > > > > > > >> > > project
> > > > > > > >> > > > in
> > > > > > > >> > > > > >> > teamcity.
> > > > > > > >> > > > > >> >
> > > > > > > >> > > > > >> >
> > > > > > > >> > > > > >> >
> > > > > > > >> > > > > >>
> > > > > > > >> > > > >
> > > > > > > >> > > >
> > > > > > > >> > >
> > > > > > > >> >
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview
> > > > > > > >> > > > > >> >
> > > > > > > >> > > > > >> > I am looking into setting up the build.
> > > > > > > >> > > > > >> >
> > > > > > > >> > > > > >> > Regards,
> > > > > > > >> > > > > >> > Saikat
> > > > > > > >> > > > > >> >
> > > > > > > >> > > > > >> > On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra <
> > > > > > > >> > > > > saikat.maitra@gmail.com>
> > > > > > > >> > > > > >> > wrote:
> > > > > > > >> > > > > >> >
> > > > > > > >> > > > > >> >> Hi Emmanouil,
> > > > > > > >> > > > > >> >>
> > > > > > > >> > > > > >> >> Thank you for your email. Yes, the plan is
> once
> > > the
> > > > > > Flink
> > > > > > > >> PR is
> > > > > > > >> > > > > merged
> > > > > > > >> > > > > >> it
> > > > > > > >> > > > > >> >> will provide the Licence, parent POMs etc and
> > you
> > > > can
> > > > > > > rebase
> > > > > > > >> > from
> > > > > > > >> > > > > >> master
> > > > > > > >> > > > > >> >> branch and apply the changes on top of it.
> > > > > > > >> > > > > >> >>
> > > > > > > >> > > > > >> >> I am getting some issues with test failures in
> > > local
> > > > > > with
> > > > > > > >> > > > > >> >> GridTestProperties as the test.properties is
> not
> > > > > present
> > > > > > > in
> > > > > > > >> > this
> > > > > > > >> > > > new
> > > > > > > >> > > > > >> >> project but should be available from
> > dependencies.
> > > > > > > >> > > > > >> >>
> > > > > > > >> > > > > >> >> Once I address this issue, I will go ahead and
> > > merge
> > > > > the
> > > > > > > >> > changes
> > > > > > > >> > > > and
> > > > > > > >> > > > > >> then
> > > > > > > >> > > > > >> >> we can take it from there.
> > > > > > > >> > > > > >> >>
> > > > > > > >> > > > > >> >> Regards,
> > > > > > > >> > > > > >> >> Saikat
> > > > > > > >> > > > > >> >>
> > > > > > > >> > > > > >> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil
> > > > Gkatziouras
> > > > > <
> > > > > > > >> > > > > >> >> gkatzioura@gmail.com> wrote:
> > > > > > > >> > > > > >> >>
> > > > > > > >> > > > > >> >>> Hi all,
> > > > > > > >> > > > > >> >>>
> > > > > > > >> > > > > >> >>> I made my first pull request [1]. Since this
> > > > project
> > > > > is
> > > > > > > >> brand
> > > > > > > >> > > new
> > > > > > > >> > > > > (no
> > > > > > > >> > > > > >> >>> parent poms, licensing), I reckoned it was
> best
> > > to
> > > > > use
> > > > > > > >> > Saikat's
> > > > > > > >> > > > > >> branch on
> > > > > > > >> > > > > >> >>> Flink.
> > > > > > > >> > > > > >> >>> I suppose the Flink branch will be merged
> > first.
> > > If
> > > > > not
> > > > > > > >> please
> > > > > > > >> > > > give
> > > > > > > >> > > > > me
> > > > > > > >> > > > > >> >>> guidelines on how I should proceed next.
> > > > > > > >> > > > > >> >>>
> > > > > > > >> > > > > >> >>> Kind regards
> > > > > > > >> > > > > >> >>> Emmanouil
> > > > > > > >> > > > > >> >>>
> > > > > > > >> > > > > >> >>> [1]
> > > > > https://github.com/apache/ignite-extensions/pull/2
> > > > > > > >> > > > > >> >>>
> > > > > > > >> > > > > >> >>> *Emmanouil Gkatziouras*
> > > > > > > >> > > > > >> >>> https://egkatzioura.com/ |
> > > > > > > >> > > > > >> >>>
> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > > > >> > > > > >> >>> https://github.com/gkatzioura
> > > > > > > >> > > > > >> >>>
> > > > > > > >> > > > > >> >>>
> > > > > > > >> > > > > >> >>> On Fri, 22 Nov 2019 at 20:55, Denis Magda <
> > > > > > > >> dmagda@apache.org>
> > > > > > > >> > > > > wrote:
> > > > > > > >> > > > > >> >>>
> > > > > > > >> > > > > >> >>> > Awesome, ping us whenever you're ready!
> > > > > > > >> > > > > >> >>> >
> > > > > > > >> > > > > >> >>> > -
> > > > > > > >> > > > > >> >>> > Denis
> > > > > > > >> > > > > >> >>> >
> > > > > > > >> > > > > >> >>> >
> > > > > > > >> > > > > >> >>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil
> > > > > > Gkatziouras
> > > > > > > <
> > > > > > > >> > > > > >> >>> > gkatzioura@gmail.com>
> > > > > > > >> > > > > >> >>> > wrote:
> > > > > > > >> > > > > >> >>> >
> > > > > > > >> > > > > >> >>> > > Hi all!
> > > > > > > >> > > > > >> >>> > >
> > > > > > > >> > > > > >> >>> > > I am sorry for being late on that. I was
> > > trying
> > > > > to
> > > > > > > >> > refactor
> > > > > > > >> > > > the
> > > > > > > >> > > > > >> test
> > > > > > > >> > > > > >> >>> in
> > > > > > > >> > > > > >> >>> > > order not to be in need of any external
> > tools
> > > > or
> > > > > > > >> spinning
> > > > > > > >> > > up a
> > > > > > > >> > > > > >> >>> server.
> > > > > > > >> > > > > >> >>> > > I did forked the new repo and indeed my
> > > changes
> > > > > > > there,
> > > > > > > >> so
> > > > > > > >> > a
> > > > > > > >> > > > pull
> > > > > > > >> > > > > >> >>> request
> > > > > > > >> > > > > >> >>> > is
> > > > > > > >> > > > > >> >>> > > a matter of time!
> > > > > > > >> > > > > >> >>> > >
> > > > > > > >> > > > > >> >>> > > Kind regards
> > > > > > > >> > > > > >> >>> > >
> > > > > > > >> > > > > >> >>> > > *Emmanouil Gkatziouras*
> > > > > > > >> > > > > >> >>> > > https://egkatzioura.com/ |
> > > > > > > >> > > > > >> >>> > >
> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > > > >> > > > > >> >>> > > https://github.com/gkatzioura
> > > > > > > >> > > > > >> >>> > >
> > > > > > > >> > > > > >> >>> > >
> > > > > > > >> > > > > >> >>> > > On Fri, 22 Nov 2019 at 20:45, Denis
> Magda <
> > > > > > > >> > > dmagda@apache.org>
> > > > > > > >> > > > > >> wrote:
> > > > > > > >> > > > > >> >>> > >
> > > > > > > >> > > > > >> >>> > > > Hi Emmanouil,
> > > > > > > >> > > > > >> >>> > > >
> > > > > > > >> > > > > >> >>> > > > Do you have any questions or need any
> > > support
> > > > > > from
> > > > > > > >> the
> > > > > > > >> > > > > >> community?
> > > > > > > >> > > > > >> >>> > > >
> > > > > > > >> > > > > >> >>> > > > -
> > > > > > > >> > > > > >> >>> > > > Denis
> > > > > > > >> > > > > >> >>> > > >
> > > > > > > >> > > > > >> >>> > > >
> > > > > > > >> > > > > >> >>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat
> > > > Maitra <
> > > > > > > >> > > > > >> >>> saikat.maitra@gmail.com
> > > > > > > >> > > > > >> >>> > >
> > > > > > > >> > > > > >> >>> > > > wrote:
> > > > > > > >> > > > > >> >>> > > >
> > > > > > > >> > > > > >> >>> > > >> Hi Emmanouil,
> > > > > > > >> > > > > >> >>> > > >>
> > > > > > > >> > > > > >> >>> > > >> Can you please take a looks at dev
> > utils,
> > > if
> > > > > > this
> > > > > > > is
> > > > > > > >> > > > > something
> > > > > > > >> > > > > >> >>> you are
> > > > > > > >> > > > > >> >>> > > >> looking for
> > > > > > > >> > > > > >> >>> > > >>
> > > > > > > >> > > > >
> > > > > > https://github.com/apache/ignite/tree/master/modules/dev-utils
> > > > > > > >> > > > > >> ?
> > > > > > > >> > > > > >> >>> > > >>
> > > > > > > >> > > > > >> >>> > > >> IMO, if you can release Pub/Sub server
> > in
> > > > > maven
> > > > > > > and
> > > > > > > >> > then
> > > > > > > >> > > > use
> > > > > > > >> > > > > >> it as
> > > > > > > >> > > > > >> >>> > > >> dependency, that would be great.
> > > > > > > >> > > > > >> >>> > > >>
> > > > > > > >> > > > > >> >>> > > >> Regards,
> > > > > > > >> > > > > >> >>> > > >> Saikat
> > > > > > > >> > > > > >> >>> > > >>
> > > > > > > >> > > > > >> >>> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat
> > > > Maitra
> > > > > <
> > > > > > > >> > > > > >> >>> > saikat.maitra@gmail.com>
> > > > > > > >> > > > > >> >>> > > >> wrote:
> > > > > > > >> > > > > >> >>> > > >>
> > > > > > > >> > > > > >> >>> > > >> > Hi Emmanouil,
> > > > > > > >> > > > > >> >>> > > >> >
> > > > > > > >> > > > > >> >>> > > >> > The master branch has been
> > initialized.
> > > I
> > > > > have
> > > > > > > >> > raised a
> > > > > > > >> > > > PR
> > > > > > > >> > > > > to
> > > > > > > >> > > > > >> >>> > migrate
> > > > > > > >> > > > > >> >>> > > >> > Flink module to ignite-extensions.
> > > > > > > >> > > > > >> >>> > > >> >
> > > > > > > >> > > > > >> >>> > > >> > PR
> > > > > > > >> > https://github.com/apache/ignite-extensions/pull/1
> > > > > > > >> > > > > >> >>> > > >> > Jira :
> > > > > > > >> > > > https://issues.apache.org/jira/browse/IGNITE-12356
> > > > > > > >> > > > > >> >>> > > >> >
> > > > > > > >> > > > > >> >>> > > >> > Regards,
> > > > > > > >> > > > > >> >>> > > >> > Saikat
> > > > > > > >> > > > > >> >>> > > >> >
> > > > > > > >> > > > > >> >>> > > >> >
> > > > > > > >> > > > > >> >>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM
> > > Emmanouil
> > > > > > > >> > Gkatziouras <
> > > > > > > >> > > > > >> >>> > > >> > gkatzioura@gmail.com> wrote:
> > > > > > > >> > > > > >> >>> > > >> >
> > > > > > > >> > > > > >> >>> > > >> >> Hi all!
> > > > > > > >> > > > > >> >>> > > >> >>
> > > > > > > >> > > > > >> >>> > > >> >> This is an update. The integration
> is
> > > > > tested
> > > > > > > with
> > > > > > > >> > > mocks
> > > > > > > >> > > > > >> based
> > > > > > > >> > > > > >> >>> on
> > > > > > > >> > > > > >> >>> > the
> > > > > > > >> > > > > >> >>> > > >> >> original Ignite repo [1].
> > > > > > > >> > > > > >> >>> > > >> >> I am currently waiting for the
> first
> > > > > commits
> > > > > > on
> > > > > > > >> the
> > > > > > > >> > > new
> > > > > > > >> > > > > >> >>> integration
> > > > > > > >> > > > > >> >>> > > >> >> project
> > > > > > > >> > > > > >> >>> > > >> >> to take place in order to create my
> > > pull
> > > > > > > request
> > > > > > > >> > > > > accordingly
> > > > > > > >> > > > > >> >>> to the
> > > > > > > >> > > > > >> >>> > > >> >> initial
> > > > > > > >> > > > > >> >>> > > >> >> commits.
> > > > > > > >> > > > > >> >>> > > >> >> In order to simulate the Pub/Sub
> > server
> > > > in
> > > > > a
> > > > > > > >> test I
> > > > > > > >> > > > used a
> > > > > > > >> > > > > >> test
> > > > > > > >> > > > > >> >>> > > server
> > > > > > > >> > > > > >> >>> > > >> >> implementation [2]. Unfortunately
> > they
> > > > > don't
> > > > > > > have
> > > > > > > >> > this
> > > > > > > >> > > > > util
> > > > > > > >> > > > > >> >>> > uploaded
> > > > > > > >> > > > > >> >>> > > >> in a
> > > > > > > >> > > > > >> >>> > > >> >> maven repo, thus it needs to be
> > hosted
> > > > on a
> > > > > > > maven
> > > > > > > >> > repo
> > > > > > > >> > > > > >> (Apache
> > > > > > > >> > > > > >> >>> > > License
> > > > > > > >> > > > > >> >>> > > >> >> 2.0). Is there a repo for utilities
> > > that
> > > > > > Ignite
> > > > > > > >> uses
> > > > > > > >> > > for
> > > > > > > >> > > > > the
> > > > > > > >> > > > > >> >>> > > >> test/build?
> > > > > > > >> > > > > >> >>> > > >> >> I have also created a ticket [3]
> > > > > > > >> > > > > >> >>> > > >> >>
> > > > > > > >> > > > > >> >>> > > >> >> Kind regards
> > > > > > > >> > > > > >> >>> > > >> >>
> > > > > > > >> > > > > >> >>> > > >> >> [1]
> > > > > > > >> > > > > >> >>> >
> > > > > > > >> > >
> > > > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > > > > > > >> > > > > >> >>> > > >> >> [2]
> > > > > > > >> > > > > >> >>>
> > > > > > > >>
> https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> > > > > > > >> > > > > >> >>> > > >> >> [3]
> > > > > > > >> > > https://issues.apache.org/jira/browse/IGNITE-12262
> > > > > > > >> > > > > >> >>> > > >> >>
> > > > > > > >> > > > > >> >>> > > >> >> *Emmanouil Gkatziouras*
> > > > > > > >> > > > > >> >>> > > >> >> https://egkatzioura.com/ |
> > > > > > > >> > > > > >> >>> > > >> >>
> > > > > > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > > > >> > > > > >> >>> > > >> >> https://github.com/gkatzioura
> > > > > > > >> > > > > >> >>> > > >> >>
> > > > > > > >> > > > > >> >>> > > >> >>
> > > > > > > >> > > > > >> >>> > > >> >> On Fri, 25 Oct 2019 at 03:35,
> Saikat
> > > > > Maitra <
> > > > > > > >> > > > > >> >>> > saikat.maitra@gmail.com
> > > > > > > >> > > > > >> >>> > > >
> > > > > > > >> > > > > >> >>> > > >> >> wrote:
> > > > > > > >> > > > > >> >>> > > >> >>
> > > > > > > >> > > > > >> >>> > > >> >> > Hello Ilya,
> > > > > > > >> > > > > >> >>> > > >> >> >
> > > > > > > >> > > > > >> >>> > > >> >> > Thank you for your email. Yes,
> > before
> > > > we
> > > > > > > remove
> > > > > > > >> > any
> > > > > > > >> > > > > >> >>> integration
> > > > > > > >> > > > > >> >>> > we
> > > > > > > >> > > > > >> >>> > > >> will
> > > > > > > >> > > > > >> >>> > > >> >> > move them to another repository
> and
> > > > > > release.
> > > > > > > >> > > > > >> >>> > > >> >> >
> > > > > > > >> > > > > >> >>> > > >> >> > We are thinking we will start
> with
> > > > > Pub/Sub
> > > > > > > >> > Streamer
> > > > > > > >> > > > > >> >>> > implementation
> > > > > > > >> > > > > >> >>> > > >> and
> > > > > > > >> > > > > >> >>> > > >> >> > continue to move other
> integration
> > as
> > > > > > > >> mentioned in
> > > > > > > >> > > the
> > > > > > > >> > > > > >> >>> following
> > > > > > > >> > > > > >> >>> > > doc.
> > > > > > > >> > > > > >> >>> > > >> >> >
> > > > > > > >> > > > > >> >>> > > >> >> >
> > > > > > > >> > > > > >> >>> > > >> >> >
> > > > > > > >> > > > > >> >>> > > >> >>
> > > > > > > >> > > > > >> >>> > > >>
> > > > > > > >> > > > > >> >>> > >
> > > > > > > >> > > > > >> >>> >
> > > > > > > >> > > > > >> >>>
> > > > > > > >> > > > > >>
> > > > > > > >> > > > >
> > > > > > > >> > > >
> > > > > > > >> > >
> > > > > > > >> >
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > > > > >> > > > > >> >>> > > >> >> >
> > > > > > > >> > > > > >> >>> > > >> >> > Regards,
> > > > > > > >> > > > > >> >>> > > >> >> > Saikat
> > > > > > > >> > > > > >> >>> > > >> >> >
> > > > > > > >> > > > > >> >>> > > >> >> >
> > > > > > > >> > > > > >> >>> > > >> >> > On Thu, Oct 24, 2019 at 11:00 AM
> > Ilya
> > > > > > > >> Kasnacheev <
> > > > > > > >> > > > > >> >>> > > >> >> > ilya.kasnacheev@gmail.com>
> > > > > > > >> > > > > >> >>> > > >> >> > wrote:
> > > > > > > >> > > > > >> >>> > > >> >> >
> > > > > > > >> > > > > >> >>> > > >> >> > > Hello!
> > > > > > > >> > > > > >> >>> > > >> >> > >
> > > > > > > >> > > > > >> >>> > > >> >> > > My take on this: we want to
> > offload
> > > > > some
> > > > > > of
> > > > > > > >> > > > > >> integrations,
> > > > > > > >> > > > > >> >>> but
> > > > > > > >> > > > > >> >>> > > >> please
> > > > > > > >> > > > > >> >>> > > >> >> > commit
> > > > > > > >> > > > > >> >>> > > >> >> > > and release them to another
> > project
> > > > > > first,
> > > > > > > >> and
> > > > > > > >> > > only
> > > > > > > >> > > > > >> then we
> > > > > > > >> > > > > >> >>> > will
> > > > > > > >> > > > > >> >>> > > >> >> decide
> > > > > > > >> > > > > >> >>> > > >> >> > to
> > > > > > > >> > > > > >> >>> > > >> >> > > remove them from our
> repository.
> > > > > > > >> > > > > >> >>> > > >> >> > >
> > > > > > > >> > > > > >> >>> > > >> >> > > Please also make sure they are
> > > > > available
> > > > > > > with
> > > > > > > >> > same
> > > > > > > >> > > > > >> artifact
> > > > > > > >> > > > > >> >>> > name
> > > > > > > >> > > > > >> >>> > > >> and
> > > > > > > >> > > > > >> >>> > > >> >> API.
> > > > > > > >> > > > > >> >>> > > >> >> > > If this has to be changed,
> let's
> > > wait
> > > > > for
> > > > > > > AI
> > > > > > > >> 3.0
> > > > > > > >> > > > > >> >>> > > >> >> > >
> > > > > > > >> > > > > >> >>> > > >> >> > > Regards,
> > > > > > > >> > > > > >> >>> > > >> >> > > --
> > > > > > > >> > > > > >> >>> > > >> >> > > Ilya Kasnacheev
> > > > > > > >> > > > > >> >>> > > >> >> > >
> > > > > > > >> > > > > >> >>> > > >> >> > >
> > > > > > > >> > > > > >> >>> > > >> >> > > ср, 23 окт. 2019 г. в 03:23,
> > Saikat
> > > > > > Maitra
> > > > > > > <
> > > > > > > >> > > > > >> >>> > > >> saikat.maitra@gmail.com>:
> > > > > > > >> > > > > >> >>> > > >> >> > >
> > > > > > > >> > > > > >> >>> > > >> >> > > > Hello Emmanouil,
> > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > >> > > > > >> >>> > > >> >> > > > As discussed earlier, I
> > discussed
> > > > > with
> > > > > > > >> Apache
> > > > > > > >> > > > Bahir
> > > > > > > >> > > > > >> >>> community
> > > > > > > >> > > > > >> >>> > > and
> > > > > > > >> > > > > >> >>> > > >> >> they
> > > > > > > >> > > > > >> >>> > > >> >> > > are
> > > > > > > >> > > > > >> >>> > > >> >> > > > interested to have Apache
> > Ignite
> > > > > > > >> extensions as
> > > > > > > >> > > > part
> > > > > > > >> > > > > of
> > > > > > > >> > > > > >> >>> Apache
> > > > > > > >> > > > > >> >>> > > >> Bahir
> > > > > > > >> > > > > >> >>> > > >> >> > > > project.
> > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > >> > > > > >> >>> > > >> >> > > > Can you please share Pub/Sub
> > > > streamer
> > > > > > > >> > > > implementation
> > > > > > > >> > > > > >> >>> details
> > > > > > > >> > > > > >> >>> > > with
> > > > > > > >> > > > > >> >>> > > >> >> > Apache
> > > > > > > >> > > > > >> >>> > > >> >> > > > Bahir community and request
> for
> > > > > > feedback.
> > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > >> > > > > >> >>> > > >> >> > > > https://bahir.apache.org/
> > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > >> > > > > >> >>> > > >> >> > > > I have also requested for
> > > > contributor
> > > > > > > >> access
> > > > > > > >> > in
> > > > > > > >> > > > Jira
> > > > > > > >> > > > > >> for
> > > > > > > >> > > > > >> >>> > Apache
> > > > > > > >> > > > > >> >>> > > >> >> Bahir
> > > > > > > >> > > > > >> >>> > > >> >> > > > project so that I can create
> > > issues
> > > > > and
> > > > > > > >> assign
> > > > > > > >> > > to
> > > > > > > >> > > > > >> myself.
> > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > >> > > > > >> >>> > > >> >> > > > I can help with code reviews
> as
> > > > well.
> > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > >> > > > > >> >>> > > >> >> > > > Here is the email thread for
> > > > > reference
> > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > >> > > > > >> >>> > >
> > > > > > > >> > > >
> > > > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > >> > > > > >> >>> > > >> >> > > > Regards,
> > > > > > > >> > > > > >> >>> > > >> >> > > > Saikat
> > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > >> > > > > >> >>> > > >> >> > > > On Mon, Oct 21, 2019 at 7:54
> PM
> > > > > Saikat
> > > > > > > >> Maitra
> > > > > > > >> > <
> > > > > > > >> > > > > >> >>> > > >> >> saikat.maitra@gmail.com
> > > > > > > >> > > > > >> >>> > > >> >> > >
> > > > > > > >> > > > > >> >>> > > >> >> > > > wrote:
> > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > >> > > > > >> >>> > > >> >> > > > > Hello,
> > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > > >> > > > > >> >>> > > >> >> > > > > As discussed I have created
> > > > > following
> > > > > > > >> > > discussion
> > > > > > > >> > > > > >> >>> threads on
> > > > > > > >> > > > > >> >>> > > our
> > > > > > > >> > > > > >> >>> > > >> >> > > migration
> > > > > > > >> > > > > >> >>> > > >> >> > > > > proposals for Apache Ignite
> > > > > > extensions:
> > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > >> > > > > >> >>> > > >> >> > >
> > > > > > > >> > > > > >> >>> > > >> >> >
> > > > > > > >> > > > > >> >>> > > >> >>
> > > > > > > >> > > > > >> >>> > > >>
> > > > > > > >> > > > > >> >>> > >
> > > > > > > >> > > > > >> >>> >
> > > > > > > >> > > > > >> >>>
> > > > > > > >> > > > > >>
> > > > > > > >> > > > >
> > > > > > > >> > > >
> > > > > > > >> > >
> > > > > > > >> >
> > > > > > > >>
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > > >> > > > > >> >>> > > >>
> > > > > > > >> > > > > >>
> > > > > > > >>
> > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > > >> > > > > >> >>> > > >> >> > > > > I will share update as I
> hear
> > > > more
> > > > > > > >> > > information.
> > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > > >> > > > > >> >>> > > >> >> > > > > Regards,
> > > > > > > >> > > > > >> >>> > > >> >> > > > > Saikat
> > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > > >> > > > > >> >>> > > >> >> > > > > On Fri, Oct 18, 2019 at
> 9:03
> > PM
> > > > > > Saikat
> > > > > > > >> > Maitra
> > > > > > > >> > > <
> > > > > > > >> > > > > >> >>> > > >> >> > saikat.maitra@gmail.com
> > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > >> > > > > >> >>> > > >> >> > > > > wrote:
> > > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > > >> > > > > >> >>> > > >> >> > > > >> Hello Denis,
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>
> > > > > > > >> > > > > >> >>> > > >> >> > > > >> Sure, sounds good. I will
> > > > create a
> > > > > > > >> separate
> > > > > > > >> > > > > >> discussion
> > > > > > > >> > > > > >> >>> > > thread
> > > > > > > >> > > > > >> >>> > > >> to
> > > > > > > >> > > > > >> >>> > > >> >> get
> > > > > > > >> > > > > >> >>> > > >> >> > > > >> community feedback on
> > whether
> > > we
> > > > > > > >> > > > > >> >>> > > >> >> > > > >> should join the Bahir or
> > > > kick-off
> > > > > > > >> "Ignite
> > > > > > > >> > > > > >> Extensions"
> > > > > > > >> > > > > >> >>> > > project.
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>
> > > > > > > >> > > > > >> >>> > > >> >> > > > >> Regards,
> > > > > > > >> > > > > >> >>> > > >> >> > > > >> Saikat
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>
> > > > > > > >> > > > > >> >>> > > >> >> > > > >> On Thu, Oct 17, 2019 at
> 2:21
> > > PM
> > > > > > Denis
> > > > > > > >> > Magda <
> > > > > > > >> > > > > >> >>> > > >> dmagda@apache.org>
> > > > > > > >> > > > > >> >>> > > >> >> > > wrote:
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> Folks,
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> The concept of Apache
> Bahir
> > > is
> > > > > > > >> precisely
> > > > > > > >> > > what
> > > > > > > >> > > > we
> > > > > > > >> > > > > >> >>> need!
> > > > > > > >> > > > > >> >>> > > >> Saikat,
> > > > > > > >> > > > > >> >>> > > >> >> > thanks
> > > > > > > >> > > > > >> >>> > > >> >> > > > for
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> doing the research. Why I
> > > > believe
> > > > > > > the *
> > > > > > > >> > > idea*
> > > > > > > >> > > > > >> fits us
> > > > > > > >> > > > > >> >>> > well:
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    - All integrations can
> > be
> > > > > stored
> > > > > > > in
> > > > > > > >> > > > separate
> > > > > > > >> > > > > >> >>> Github
> > > > > > > >> > > > > >> >>> > > >> >> repositories
> > > > > > > >> > > > > >> >>> > > >> >> > > and
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    have their dev
> > lifecycles.
> > > > > We've
> > > > > > > not
> > > > > > > >> > > > obliged
> > > > > > > >> > > > > to
> > > > > > > >> > > > > >> >>> couple
> > > > > > > >> > > > > >> >>> > > all
> > > > > > > >> > > > > >> >>> > > >> >> the
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> integrations
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    in a single repo. For
> > > > > instance,
> > > > > > > >> Spark
> > > > > > > >> > can
> > > > > > > >> > > > be
> > > > > > > >> > > > > >> >>> located
> > > > > > > >> > > > > >> >>> > in
> > > > > > > >> > > > > >> >>> > > a
> > > > > > > >> > > > > >> >>> > > >> >> > > dedicated
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> repo
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    while streaming
> > > integrations
> > > > > > might
> > > > > > > >> be
> > > > > > > >> > in
> > > > > > > >> > > a
> > > > > > > >> > > > > >> single
> > > > > > > >> > > > > >> >>> one.
> > > > > > > >> > > > > >> >>> > > >> It's
> > > > > > > >> > > > > >> >>> > > >> >> up
> > > > > > > >> > > > > >> >>> > > >> >> > to
> > > > > > > >> > > > > >> >>> > > >> >> > > > us.
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    - All the repositories
> > and
> > > > > > > >> integrations
> > > > > > > >> > > > > belong
> > > > > > > >> > > > > >> to
> > > > > > > >> > > > > >> >>> ASF.
> > > > > > > >> > > > > >> >>> > > >> We're
> > > > > > > >> > > > > >> >>> > > >> >> not
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> dumping
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    them on Github but
> > rather
> > > > keep
> > > > > > > >> > supporting
> > > > > > > >> > > > and
> > > > > > > >> > > > > >> >>> > developing
> > > > > > > >> > > > > >> >>> > > >> in
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> accordance with
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    ASF vision and
> > practices.
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    - There is a way to
> > reward
> > > > > > > >> contributors
> > > > > > > >> > > via
> > > > > > > >> > > > > >> >>> > > committership
> > > > > > > >> > > > > >> >>> > > >> and
> > > > > > > >> > > > > >> >>> > > >> >> > PMC
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    membership. You won't
> > get
> > > > this
> > > > > > if
> > > > > > > a
> > > > > > > >> > > project
> > > > > > > >> > > > > is
> > > > > > > >> > > > > >> >>> just
> > > > > > > >> > > > > >> >>> > one
> > > > > > > >> > > > > >> >>> > > of
> > > > > > > >> > > > > >> >>> > > >> >> the
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> millions of
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>>    Github projects.
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> Saikat, as Ignite PMC
> > member,
> > > > > could
> > > > > > > you
> > > > > > > >> > > please
> > > > > > > >> > > > > >> >>> kick-off a
> > > > > > > >> > > > > >> >>> > > >> >> separate
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> dev-list
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> discussion to involve
> more
> > > > > > community
> > > > > > > >> > members
> > > > > > > >> > > > and
> > > > > > > >> > > > > >> >>> > referring
> > > > > > > >> > > > > >> >>> > > to
> > > > > > > >> > > > > >> >>> > > >> >> this
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> thread?
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> I think the primary
> > question
> > > > the
> > > > > > > >> community
> > > > > > > >> > > > needs
> > > > > > > >> > > > > >> to
> > > > > > > >> > > > > >> >>> > answer
> > > > > > > >> > > > > >> >>> > > >> >> whether
> > > > > > > >> > > > > >> >>> > > >> >> > we
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> should join the Bahir or
> > > > kick-off
> > > > > > > >> "Ignite
> > > > > > > >> > > > > >> Extensions"
> > > > > > > >> > > > > >> >>> > > >> project?
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> -
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> Denis
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> On Thu, Oct 17, 2019 at
> > 2:54
> > > AM
> > > > > > > Alexey
> > > > > > > >> > > > Zinoviev
> > > > > > > >> > > > > <
> > > > > > > >> > > > > >> >>> > > >> >> > > > zaleslaw.sin@gmail.com>
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> wrote:
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > Maybe we could move all
> > our
> > > > > > > Streaming
> > > > > > > >> > > > > >> Integrations
> > > > > > > >> > > > > >> >>> > there,
> > > > > > > >> > > > > >> >>> > > >> but
> > > > > > > >> > > > > >> >>> > > >> >> > what
> > > > > > > >> > > > > >> >>> > > >> >> > > is
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> about
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > maintaining and
> committer
> > > > > > > >> permissions to
> > > > > > > >> > > the
> > > > > > > >> > > > > new
> > > > > > > >> > > > > >> >>> > > >> repositories?
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > I see the list of the
> > > > > committers
> > > > > > > and
> > > > > > > >> PMC
> > > > > > > >> > > > > members
> > > > > > > >> > > > > >> >>> there
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > > > > >> > > https://bahir.apache.org/community-members/
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > Could somebody from
> > Ignite
> > > > > > > community
> > > > > > > >> be
> > > > > > > >> > > > added
> > > > > > > >> > > > > to
> > > > > > > >> > > > > >> >>> this
> > > > > > > >> > > > > >> >>> > > list
> > > > > > > >> > > > > >> >>> > > >> as
> > > > > > > >> > > > > >> >>> > > >> >> a
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > PMC/committer to
> maintain
> > > > > Ignite
> > > > > > > >> > > > integrations?
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > If yes, I'd happy to
> join
> > > > this
> > > > > > > >> project
> > > > > > > >> > and
> > > > > > > >> > > > > >> >>> collaborate
> > > > > > > >> > > > > >> >>> > > with
> > > > > > > >> > > > > >> >>> > > >> >> these
> > > > > > > >> > > > > >> >>> > > >> >> > > > guys
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > together
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > If no, and we should
> > start
> > > > from
> > > > > > the
> > > > > > > >> zero
> > > > > > > >> > > > level
> > > > > > > >> > > > > >> with
> > > > > > > >> > > > > >> >>> > > >> external
> > > > > > > >> > > > > >> >>> > > >> >> PRs
> > > > > > > >> > > > > >> >>> > > >> >> > -
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> hmmm,
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > it's better to have our
> > own
> > > > > > > external
> > > > > > > >> > > > > repository
> > > > > > > >> > > > > >> >>> with
> > > > > > > >> > > > > >> >>> > > >> >> ApacheBahirr
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> ideology.
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > Also, I agree, that all
> > > > > > connectors
> > > > > > > >> could
> > > > > > > >> > > be
> > > > > > > >> > > > > >> moved
> > > > > > > >> > > > > >> >>> there
> > > > > > > >> > > > > >> >>> > > (in
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> ApacheBahirr
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> > like repository), but
> not
> > > all
> > > > > > > modules
> > > > > > > >> > from
> > > > > > > >> > > > the
> > > > > > > >> > > > > >> page
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > > >> > > > > >> >>> > > >> >> > >
> > > > > > > >> > > > > >> >>> > > >> >> >
> > > > > > > >> > >

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra <sa...@gmail.com>]

Hi Emmanouil,

Thank you for making the changes. I have merged my PR changes in master.
Can you please take the latest from master branch and resolve the conflicts
in your PR.

I have also reviewed the changes in your PR and requested minor changes
specific to dependencies version and license information.

Can you please review and share your feedback?

Regards,
Saikat




On Sat, Dec 28, 2019 at 4:38 AM Emmanouil Gkatziouras <gk...@gmail.com>
wrote:

> Hi Saikat!
>
> Thank you on this one! I tried it and it worked as expected [1]
>
> Kind regards
> Emmanouil
>
> [1]
>
> https://ci.ignite.apache.org/viewLog.html?buildId=4888357&tab=buildLog&_focus=2046#_state=2046
>
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>
>
> On Thu, 26 Dec 2019 at 19:50, Saikat Maitra <sa...@gmail.com>
> wrote:
>
> > Hi Emmanouil,
> >
> > I have been able to resolve the build failures for missing modules.
> >
> >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4883394&buildTypeId=IgniteExtensions_Build
> >
> > Will you please take the latest changes from my PR and run the build for
> > your PR.
> >
> > https://github.com/apache/ignite-extensions/pull/1/files
> >
> > The new build configurations should let your module build also pass.
> >
> > The issue that was causing failure was that I had put ignite.zip as
> > artifact dependency and it was downloading the artifacts including pom
> file
> > inside the build checkout folder of ignite-extensions and as a result the
> > pom.xml of ignite-extensions was getting overwritten by ignite's pom.xml.
> > This was resulting in missing flink-ext module error as it was not even
> > present in the ignite main pom.xml.
> >
> > To resolve the issue I just had to download and extract artifact of
> > ignite.zip in a separate directory.
> >
> > Regards,
> > Saikat
> >
> >
> > On Sat, Nov 30, 2019 at 5:53 PM Emmanouil Gkatziouras <
> > gkatzioura@gmail.com>
> > wrote:
> >
> > > Hi Saikat!
> > >
> > > Thank you for your assistance on that!
> > >
> > > Kind regards
> > >
> > > *Emmanouil Gkatziouras*
> > > https://egkatzioura.com/ |
> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > https://github.com/gkatzioura
> > >
> > >
> > > On Sat, 30 Nov 2019 at 23:51, Saikat Maitra <sa...@gmail.com>
> > > wrote:
> > >
> > > > Hi Emmanouil,
> > > >
> > > > I looked into the build logs and I observed that since I added
> > artifacts
> > > > dependencies to "Build apache Ignite"[1] it was able to pull the
> > required
> > > > dependencies and was able to run the existing tests. It is however
> not
> > > > identifying new modules like I changed from flink to flink-ext or
> > > pub-sub.
> > > >
> > > > [1]
> > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/admin/editDependencies.html?id=buildType:IgniteExtensions_Build
> > > >
> > > > I will look into the issue and debug further.
> > > >
> > > > Regards,
> > > > Saikat
> > > >
> > > >
> > > > On Sat, Nov 30, 2019 at 5:35 PM Emmanouil Gkatziouras <
> > > > gkatzioura@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi Saikat!
> > > > >
> > > > > From the logs[1] it seems that TC cannot find the project. However
> I
> > > did
> > > > > include the project on the parent pom (modules sections) [2]
> > > > > I tried to reproduce locally unfortunately I am not aware of the
> full
> > > > > context on TC. It seems as if a different parent pom is being
> picked
> > > up.
> > > > >
> > > > > Kind regards
> > > > >
> > > > > [1]
> > > > >
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4804893&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=1704
> > > > > [2]
> > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/gkatzioura/ignite-extensions/blob/d6a8beee7feff28f59d9a12be692016305564d25/pom.xml#L46
> > > > >
> > > > > *Emmanouil Gkatziouras*
> > > > > https://egkatzioura.com/ |
> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > https://github.com/gkatzioura
> > > > >
> > > > >
> > > > > On Sat, 30 Nov 2019 at 22:20, Saikat Maitra <
> saikat.maitra@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > > > Hello,
> > > > > >
> > > > > > I have made changes in Ignite Extensions build configurations.
> > > > > >
> > > > > > It is now running the tests as expected.
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4804480&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> > > > > >
> > > > > > Regards,
> > > > > > Saikat
> > > > > >
> > > > > > On Thu, Nov 28, 2019 at 1:29 PM Saikat Maitra <
> > > saikat.maitra@gmail.com
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > We do run build with -DskipTests and then configure selective
> > test
> > > > > suites
> > > > > > > to run.
> > > > > > >
> > > > > > > I will check why these test suites are not getting executed.
> > > > > > >
> > > > > > > Regards
> > > > > > > Saikat
> > > > > > >
> > > > > > > On Wed, 27 Nov 2019 at 3:05 PM, Emmanouil Gkatziouras <
> > > > > > > gkatzioura@gmail.com> wrote:
> > > > > > >
> > > > > > >> Hi all,
> > > > > > >>
> > > > > > >> I did add a test suite and run the build with the
> corresponding
> > > > > > arguments,
> > > > > > >> however the tests did not run [1].
> > > > > > >> I checked the Flink's build logs and the same message is
> > displayed
> > > > > there
> > > > > > >> too `No tests to run` [2].
> > > > > > >>
> > > > > > >> [1]
> > > > > > >>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4798203&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=248
> > > > > > >> [2]
> > > > > > >>
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4789199&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=1256
> > > > > > >>
> > > > > > >> Kind regards
> > > > > > >> *Emmanouil Gkatziouras*
> > > > > > >> https://egkatzioura.com/ |
> > > > > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > > >> https://github.com/gkatzioura
> > > > > > >>
> > > > > > >>
> > > > > > >> On Wed, 27 Nov 2019 at 18:58, Saikat Maitra <
> > > > saikat.maitra@gmail.com>
> > > > > > >> wrote:
> > > > > > >>
> > > > > > >> > Hi,
> > > > > > >> >
> > > > > > >> > We will need to add test suites for tests to be executed in
> > > build.
> > > > > We
> > > > > > >> pass
> > > > > > >> > them as TestSuites param. I had added Flink sink and source
> > > > > > testsuites.
> > > > > > >> >
> > > > > > >> > Can you please review and confirm.
> > > > > > >> >
> > > > > > >> > Regards
> > > > > > >> > Saikat
> > > > > > >> >
> > > > > > >> > On Wed, 27 Nov 2019 at 6:00 AM, Emmanouil Gkatziouras <
> > > > > > >> > gkatzioura@gmail.com>
> > > > > > >> > wrote:
> > > > > > >> >
> > > > > > >> > > Hi all!
> > > > > > >> > >
> > > > > > >> > > I was successful building and running my pull requests. It
> > > seems
> > > > > > that
> > > > > > >> the
> > > > > > >> > > tests do not run both for Flink and Pub/Sub [1]
> > > > > > >> > > If there is something I cannot do to make them
> discoverable
> > > > please
> > > > > > >> let me
> > > > > > >> > > know.
> > > > > > >> > >
> > > > > > >> > > [1]
> > > > > > >> > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4796722&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=283
> > > > > > >> > >
> > > > > > >> > > *Emmanouil Gkatziouras*
> > > > > > >> > > https://egkatzioura.com/ |
> > > > > > >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > > >> > > https://github.com/gkatzioura
> > > > > > >> > >
> > > > > > >> > >
> > > > > > >> > > On Wed, 27 Nov 2019 at 04:57, Saikat Maitra <
> > > > > > saikat.maitra@gmail.com>
> > > > > > >> > > wrote:
> > > > > > >> > >
> > > > > > >> > > > Hi Emmanouil,
> > > > > > >> > > >
> > > > > > >> > > > I have added you as contributor in Ignite Extensions
> > > project.
> > > > > Can
> > > > > > >> you
> > > > > > >> > > > please check and confirm if you are able to see the
> > project
> > > > and
> > > > > > >> execute
> > > > > > >> > > > build on your pull request.
> > > > > > >> > > >
> > > > > > >> > > > Regards,
> > > > > > >> > > > Saikat
> > > > > > >> > > >
> > > > > > >> > > > On Tue, Nov 26, 2019 at 12:49 PM Emmanouil Gkatziouras <
> > > > > > >> > > > gkatzioura@gmail.com>
> > > > > > >> > > > wrote:
> > > > > > >> > > >
> > > > > > >> > > > > Hi all!
> > > > > > >> > > > >
> > > > > > >> > > > > If someone could give me read access to the Job of
> > Ignite
> > > > > > >> Extensions
> > > > > > >> > on
> > > > > > >> > > > > Team City will greatly help me (username gkatzioura).
> > > > > > >> > > > > I suppose the builds get triggered automatically on a
> > pull
> > > > > > >> request.
> > > > > > >> > > > >
> > > > > > >> > > > > Kind regards,
> > > > > > >> > > > > Emmanouil
> > > > > > >> > > > >
> > > > > > >> > > > > *Emmanouil Gkatziouras*
> > > > > > >> > > > > https://egkatzioura.com/ |
> > > > > > >> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > > >> > > > > https://github.com/gkatzioura
> > > > > > >> > > > >
> > > > > > >> > > > >
> > > > > > >> > > > > On Sun, 24 Nov 2019 at 21:11, Emmanouil Gkatziouras <
> > > > > > >> > > > gkatzioura@gmail.com>
> > > > > > >> > > > > wrote:
> > > > > > >> > > > >
> > > > > > >> > > > > > Hi Saikat!
> > > > > > >> > > > > >
> > > > > > >> > > > > > I just rebased with the flink branch. Unfortunately
> I
> > do
> > > > not
> > > > > > >> have
> > > > > > >> > > read
> > > > > > >> > > > > > access to the team city link you provided and thus
> > > > evaluate
> > > > > > the
> > > > > > >> > tests
> > > > > > >> > > > for
> > > > > > >> > > > > > my pull request.
> > > > > > >> > > > > > I guess it has to do with ignite extensions being
> new.
> > > > > > >> > > > > > My username is gkatzioura and email the one I am
> using
> > > on
> > > > > this
> > > > > > >> > email
> > > > > > >> > > > > >
> > > > > > >> > > > > > Thank you for your time.
> > > > > > >> > > > > >
> > > > > > >> > > > > >
> > > > > > >> > > > > > *Emmanouil Gkatziouras*
> > > > > > >> > > > > > https://egkatzioura.com/ |
> > > > > > >> > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > > >> > > > > > https://github.com/gkatzioura
> > > > > > >> > > > > >
> > > > > > >> > > > > >
> > > > > > >> > > > > > On Sun, 24 Nov 2019 at 18:03, Saikat Maitra <
> > > > > > >> > saikat.maitra@gmail.com
> > > > > > >> > > >
> > > > > > >> > > > > > wrote:
> > > > > > >> > > > > >
> > > > > > >> > > > > >> Hi Emmanouil,
> > > > > > >> > > > > >>
> > > > > > >> > > > > >> The latest build on teamcity has passed on Flink
> pull
> > > > > > request.
> > > > > > >> > > > > >>
> > > > > > >> > > > > >>
> > > > > > >> > > > > >>
> > > > > > >> > > > >
> > > > > > >> > > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4788928&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> > > > > > >> > > > > >>
> > > > > > >> > > > > >> you should be able to take the changes I made in my
> > PR
> > > > and
> > > > > > run
> > > > > > >> > build
> > > > > > >> > > > on
> > > > > > >> > > > > >> your pull request.
> > > > > > >> > > > > >>
> > > > > > >> > > > > >> Please let me know if you have any questions.
> > > > > > >> > > > > >>
> > > > > > >> > > > > >> Regards,
> > > > > > >> > > > > >> Saikat
> > > > > > >> > > > > >>
> > > > > > >> > > > > >> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra <
> > > > > > >> > > > saikat.maitra@gmail.com
> > > > > > >> > > > > >
> > > > > > >> > > > > >> wrote:
> > > > > > >> > > > > >>
> > > > > > >> > > > > >> > Hi Emmanouil,
> > > > > > >> > > > > >> >
> > > > > > >> > > > > >> > I have fixed the Flink module testsuites and have
> > > setup
> > > > > new
> > > > > > >> > > project
> > > > > > >> > > > in
> > > > > > >> > > > > >> > teamcity.
> > > > > > >> > > > > >> >
> > > > > > >> > > > > >> >
> > > > > > >> > > > > >> >
> > > > > > >> > > > > >>
> > > > > > >> > > > >
> > > > > > >> > > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview
> > > > > > >> > > > > >> >
> > > > > > >> > > > > >> > I am looking into setting up the build.
> > > > > > >> > > > > >> >
> > > > > > >> > > > > >> > Regards,
> > > > > > >> > > > > >> > Saikat
> > > > > > >> > > > > >> >
> > > > > > >> > > > > >> > On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra <
> > > > > > >> > > > > saikat.maitra@gmail.com>
> > > > > > >> > > > > >> > wrote:
> > > > > > >> > > > > >> >
> > > > > > >> > > > > >> >> Hi Emmanouil,
> > > > > > >> > > > > >> >>
> > > > > > >> > > > > >> >> Thank you for your email. Yes, the plan is once
> > the
> > > > > Flink
> > > > > > >> PR is
> > > > > > >> > > > > merged
> > > > > > >> > > > > >> it
> > > > > > >> > > > > >> >> will provide the Licence, parent POMs etc and
> you
> > > can
> > > > > > rebase
> > > > > > >> > from
> > > > > > >> > > > > >> master
> > > > > > >> > > > > >> >> branch and apply the changes on top of it.
> > > > > > >> > > > > >> >>
> > > > > > >> > > > > >> >> I am getting some issues with test failures in
> > local
> > > > > with
> > > > > > >> > > > > >> >> GridTestProperties as the test.properties is not
> > > > present
> > > > > > in
> > > > > > >> > this
> > > > > > >> > > > new
> > > > > > >> > > > > >> >> project but should be available from
> dependencies.
> > > > > > >> > > > > >> >>
> > > > > > >> > > > > >> >> Once I address this issue, I will go ahead and
> > merge
> > > > the
> > > > > > >> > changes
> > > > > > >> > > > and
> > > > > > >> > > > > >> then
> > > > > > >> > > > > >> >> we can take it from there.
> > > > > > >> > > > > >> >>
> > > > > > >> > > > > >> >> Regards,
> > > > > > >> > > > > >> >> Saikat
> > > > > > >> > > > > >> >>
> > > > > > >> > > > > >> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil
> > > Gkatziouras
> > > > <
> > > > > > >> > > > > >> >> gkatzioura@gmail.com> wrote:
> > > > > > >> > > > > >> >>
> > > > > > >> > > > > >> >>> Hi all,
> > > > > > >> > > > > >> >>>
> > > > > > >> > > > > >> >>> I made my first pull request [1]. Since this
> > > project
> > > > is
> > > > > > >> brand
> > > > > > >> > > new
> > > > > > >> > > > > (no
> > > > > > >> > > > > >> >>> parent poms, licensing), I reckoned it was best
> > to
> > > > use
> > > > > > >> > Saikat's
> > > > > > >> > > > > >> branch on
> > > > > > >> > > > > >> >>> Flink.
> > > > > > >> > > > > >> >>> I suppose the Flink branch will be merged
> first.
> > If
> > > > not
> > > > > > >> please
> > > > > > >> > > > give
> > > > > > >> > > > > me
> > > > > > >> > > > > >> >>> guidelines on how I should proceed next.
> > > > > > >> > > > > >> >>>
> > > > > > >> > > > > >> >>> Kind regards
> > > > > > >> > > > > >> >>> Emmanouil
> > > > > > >> > > > > >> >>>
> > > > > > >> > > > > >> >>> [1]
> > > > https://github.com/apache/ignite-extensions/pull/2
> > > > > > >> > > > > >> >>>
> > > > > > >> > > > > >> >>> *Emmanouil Gkatziouras*
> > > > > > >> > > > > >> >>> https://egkatzioura.com/ |
> > > > > > >> > > > > >> >>>
> > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > > >> > > > > >> >>> https://github.com/gkatzioura
> > > > > > >> > > > > >> >>>
> > > > > > >> > > > > >> >>>
> > > > > > >> > > > > >> >>> On Fri, 22 Nov 2019 at 20:55, Denis Magda <
> > > > > > >> dmagda@apache.org>
> > > > > > >> > > > > wrote:
> > > > > > >> > > > > >> >>>
> > > > > > >> > > > > >> >>> > Awesome, ping us whenever you're ready!
> > > > > > >> > > > > >> >>> >
> > > > > > >> > > > > >> >>> > -
> > > > > > >> > > > > >> >>> > Denis
> > > > > > >> > > > > >> >>> >
> > > > > > >> > > > > >> >>> >
> > > > > > >> > > > > >> >>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil
> > > > > Gkatziouras
> > > > > > <
> > > > > > >> > > > > >> >>> > gkatzioura@gmail.com>
> > > > > > >> > > > > >> >>> > wrote:
> > > > > > >> > > > > >> >>> >
> > > > > > >> > > > > >> >>> > > Hi all!
> > > > > > >> > > > > >> >>> > >
> > > > > > >> > > > > >> >>> > > I am sorry for being late on that. I was
> > trying
> > > > to
> > > > > > >> > refactor
> > > > > > >> > > > the
> > > > > > >> > > > > >> test
> > > > > > >> > > > > >> >>> in
> > > > > > >> > > > > >> >>> > > order not to be in need of any external
> tools
> > > or
> > > > > > >> spinning
> > > > > > >> > > up a
> > > > > > >> > > > > >> >>> server.
> > > > > > >> > > > > >> >>> > > I did forked the new repo and indeed my
> > changes
> > > > > > there,
> > > > > > >> so
> > > > > > >> > a
> > > > > > >> > > > pull
> > > > > > >> > > > > >> >>> request
> > > > > > >> > > > > >> >>> > is
> > > > > > >> > > > > >> >>> > > a matter of time!
> > > > > > >> > > > > >> >>> > >
> > > > > > >> > > > > >> >>> > > Kind regards
> > > > > > >> > > > > >> >>> > >
> > > > > > >> > > > > >> >>> > > *Emmanouil Gkatziouras*
> > > > > > >> > > > > >> >>> > > https://egkatzioura.com/ |
> > > > > > >> > > > > >> >>> > >
> > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > > >> > > > > >> >>> > > https://github.com/gkatzioura
> > > > > > >> > > > > >> >>> > >
> > > > > > >> > > > > >> >>> > >
> > > > > > >> > > > > >> >>> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda <
> > > > > > >> > > dmagda@apache.org>
> > > > > > >> > > > > >> wrote:
> > > > > > >> > > > > >> >>> > >
> > > > > > >> > > > > >> >>> > > > Hi Emmanouil,
> > > > > > >> > > > > >> >>> > > >
> > > > > > >> > > > > >> >>> > > > Do you have any questions or need any
> > support
> > > > > from
> > > > > > >> the
> > > > > > >> > > > > >> community?
> > > > > > >> > > > > >> >>> > > >
> > > > > > >> > > > > >> >>> > > > -
> > > > > > >> > > > > >> >>> > > > Denis
> > > > > > >> > > > > >> >>> > > >
> > > > > > >> > > > > >> >>> > > >
> > > > > > >> > > > > >> >>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat
> > > Maitra <
> > > > > > >> > > > > >> >>> saikat.maitra@gmail.com
> > > > > > >> > > > > >> >>> > >
> > > > > > >> > > > > >> >>> > > > wrote:
> > > > > > >> > > > > >> >>> > > >
> > > > > > >> > > > > >> >>> > > >> Hi Emmanouil,
> > > > > > >> > > > > >> >>> > > >>
> > > > > > >> > > > > >> >>> > > >> Can you please take a looks at dev
> utils,
> > if
> > > > > this
> > > > > > is
> > > > > > >> > > > > something
> > > > > > >> > > > > >> >>> you are
> > > > > > >> > > > > >> >>> > > >> looking for
> > > > > > >> > > > > >> >>> > > >>
> > > > > > >> > > > >
> > > > > https://github.com/apache/ignite/tree/master/modules/dev-utils
> > > > > > >> > > > > >> ?
> > > > > > >> > > > > >> >>> > > >>
> > > > > > >> > > > > >> >>> > > >> IMO, if you can release Pub/Sub server
> in
> > > > maven
> > > > > > and
> > > > > > >> > then
> > > > > > >> > > > use
> > > > > > >> > > > > >> it as
> > > > > > >> > > > > >> >>> > > >> dependency, that would be great.
> > > > > > >> > > > > >> >>> > > >>
> > > > > > >> > > > > >> >>> > > >> Regards,
> > > > > > >> > > > > >> >>> > > >> Saikat
> > > > > > >> > > > > >> >>> > > >>
> > > > > > >> > > > > >> >>> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat
> > > Maitra
> > > > <
> > > > > > >> > > > > >> >>> > saikat.maitra@gmail.com>
> > > > > > >> > > > > >> >>> > > >> wrote:
> > > > > > >> > > > > >> >>> > > >>
> > > > > > >> > > > > >> >>> > > >> > Hi Emmanouil,
> > > > > > >> > > > > >> >>> > > >> >
> > > > > > >> > > > > >> >>> > > >> > The master branch has been
> initialized.
> > I
> > > > have
> > > > > > >> > raised a
> > > > > > >> > > > PR
> > > > > > >> > > > > to
> > > > > > >> > > > > >> >>> > migrate
> > > > > > >> > > > > >> >>> > > >> > Flink module to ignite-extensions.
> > > > > > >> > > > > >> >>> > > >> >
> > > > > > >> > > > > >> >>> > > >> > PR
> > > > > > >> > https://github.com/apache/ignite-extensions/pull/1
> > > > > > >> > > > > >> >>> > > >> > Jira :
> > > > > > >> > > > https://issues.apache.org/jira/browse/IGNITE-12356
> > > > > > >> > > > > >> >>> > > >> >
> > > > > > >> > > > > >> >>> > > >> > Regards,
> > > > > > >> > > > > >> >>> > > >> > Saikat
> > > > > > >> > > > > >> >>> > > >> >
> > > > > > >> > > > > >> >>> > > >> >
> > > > > > >> > > > > >> >>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM
> > Emmanouil
> > > > > > >> > Gkatziouras <
> > > > > > >> > > > > >> >>> > > >> > gkatzioura@gmail.com> wrote:
> > > > > > >> > > > > >> >>> > > >> >
> > > > > > >> > > > > >> >>> > > >> >> Hi all!
> > > > > > >> > > > > >> >>> > > >> >>
> > > > > > >> > > > > >> >>> > > >> >> This is an update. The integration is
> > > > tested
> > > > > > with
> > > > > > >> > > mocks
> > > > > > >> > > > > >> based
> > > > > > >> > > > > >> >>> on
> > > > > > >> > > > > >> >>> > the
> > > > > > >> > > > > >> >>> > > >> >> original Ignite repo [1].
> > > > > > >> > > > > >> >>> > > >> >> I am currently waiting for the first
> > > > commits
> > > > > on
> > > > > > >> the
> > > > > > >> > > new
> > > > > > >> > > > > >> >>> integration
> > > > > > >> > > > > >> >>> > > >> >> project
> > > > > > >> > > > > >> >>> > > >> >> to take place in order to create my
> > pull
> > > > > > request
> > > > > > >> > > > > accordingly
> > > > > > >> > > > > >> >>> to the
> > > > > > >> > > > > >> >>> > > >> >> initial
> > > > > > >> > > > > >> >>> > > >> >> commits.
> > > > > > >> > > > > >> >>> > > >> >> In order to simulate the Pub/Sub
> server
> > > in
> > > > a
> > > > > > >> test I
> > > > > > >> > > > used a
> > > > > > >> > > > > >> test
> > > > > > >> > > > > >> >>> > > server
> > > > > > >> > > > > >> >>> > > >> >> implementation [2]. Unfortunately
> they
> > > > don't
> > > > > > have
> > > > > > >> > this
> > > > > > >> > > > > util
> > > > > > >> > > > > >> >>> > uploaded
> > > > > > >> > > > > >> >>> > > >> in a
> > > > > > >> > > > > >> >>> > > >> >> maven repo, thus it needs to be
> hosted
> > > on a
> > > > > > maven
> > > > > > >> > repo
> > > > > > >> > > > > >> (Apache
> > > > > > >> > > > > >> >>> > > License
> > > > > > >> > > > > >> >>> > > >> >> 2.0). Is there a repo for utilities
> > that
> > > > > Ignite
> > > > > > >> uses
> > > > > > >> > > for
> > > > > > >> > > > > the
> > > > > > >> > > > > >> >>> > > >> test/build?
> > > > > > >> > > > > >> >>> > > >> >> I have also created a ticket [3]
> > > > > > >> > > > > >> >>> > > >> >>
> > > > > > >> > > > > >> >>> > > >> >> Kind regards
> > > > > > >> > > > > >> >>> > > >> >>
> > > > > > >> > > > > >> >>> > > >> >> [1]
> > > > > > >> > > > > >> >>> >
> > > > > > >> > >
> > > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > > > > > >> > > > > >> >>> > > >> >> [2]
> > > > > > >> > > > > >> >>>
> > > > > > >> https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> > > > > > >> > > > > >> >>> > > >> >> [3]
> > > > > > >> > > https://issues.apache.org/jira/browse/IGNITE-12262
> > > > > > >> > > > > >> >>> > > >> >>
> > > > > > >> > > > > >> >>> > > >> >> *Emmanouil Gkatziouras*
> > > > > > >> > > > > >> >>> > > >> >> https://egkatzioura.com/ |
> > > > > > >> > > > > >> >>> > > >> >>
> > > > > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > > >> > > > > >> >>> > > >> >> https://github.com/gkatzioura
> > > > > > >> > > > > >> >>> > > >> >>
> > > > > > >> > > > > >> >>> > > >> >>
> > > > > > >> > > > > >> >>> > > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat
> > > > Maitra <
> > > > > > >> > > > > >> >>> > saikat.maitra@gmail.com
> > > > > > >> > > > > >> >>> > > >
> > > > > > >> > > > > >> >>> > > >> >> wrote:
> > > > > > >> > > > > >> >>> > > >> >>
> > > > > > >> > > > > >> >>> > > >> >> > Hello Ilya,
> > > > > > >> > > > > >> >>> > > >> >> >
> > > > > > >> > > > > >> >>> > > >> >> > Thank you for your email. Yes,
> before
> > > we
> > > > > > remove
> > > > > > >> > any
> > > > > > >> > > > > >> >>> integration
> > > > > > >> > > > > >> >>> > we
> > > > > > >> > > > > >> >>> > > >> will
> > > > > > >> > > > > >> >>> > > >> >> > move them to another repository and
> > > > > release.
> > > > > > >> > > > > >> >>> > > >> >> >
> > > > > > >> > > > > >> >>> > > >> >> > We are thinking we will start with
> > > > Pub/Sub
> > > > > > >> > Streamer
> > > > > > >> > > > > >> >>> > implementation
> > > > > > >> > > > > >> >>> > > >> and
> > > > > > >> > > > > >> >>> > > >> >> > continue to move other integration
> as
> > > > > > >> mentioned in
> > > > > > >> > > the
> > > > > > >> > > > > >> >>> following
> > > > > > >> > > > > >> >>> > > doc.
> > > > > > >> > > > > >> >>> > > >> >> >
> > > > > > >> > > > > >> >>> > > >> >> >
> > > > > > >> > > > > >> >>> > > >> >> >
> > > > > > >> > > > > >> >>> > > >> >>
> > > > > > >> > > > > >> >>> > > >>
> > > > > > >> > > > > >> >>> > >
> > > > > > >> > > > > >> >>> >
> > > > > > >> > > > > >> >>>
> > > > > > >> > > > > >>
> > > > > > >> > > > >
> > > > > > >> > > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > > > >> > > > > >> >>> > > >> >> >
> > > > > > >> > > > > >> >>> > > >> >> > Regards,
> > > > > > >> > > > > >> >>> > > >> >> > Saikat
> > > > > > >> > > > > >> >>> > > >> >> >
> > > > > > >> > > > > >> >>> > > >> >> >
> > > > > > >> > > > > >> >>> > > >> >> > On Thu, Oct 24, 2019 at 11:00 AM
> Ilya
> > > > > > >> Kasnacheev <
> > > > > > >> > > > > >> >>> > > >> >> > ilya.kasnacheev@gmail.com>
> > > > > > >> > > > > >> >>> > > >> >> > wrote:
> > > > > > >> > > > > >> >>> > > >> >> >
> > > > > > >> > > > > >> >>> > > >> >> > > Hello!
> > > > > > >> > > > > >> >>> > > >> >> > >
> > > > > > >> > > > > >> >>> > > >> >> > > My take on this: we want to
> offload
> > > > some
> > > > > of
> > > > > > >> > > > > >> integrations,
> > > > > > >> > > > > >> >>> but
> > > > > > >> > > > > >> >>> > > >> please
> > > > > > >> > > > > >> >>> > > >> >> > commit
> > > > > > >> > > > > >> >>> > > >> >> > > and release them to another
> project
> > > > > first,
> > > > > > >> and
> > > > > > >> > > only
> > > > > > >> > > > > >> then we
> > > > > > >> > > > > >> >>> > will
> > > > > > >> > > > > >> >>> > > >> >> decide
> > > > > > >> > > > > >> >>> > > >> >> > to
> > > > > > >> > > > > >> >>> > > >> >> > > remove them from our repository.
> > > > > > >> > > > > >> >>> > > >> >> > >
> > > > > > >> > > > > >> >>> > > >> >> > > Please also make sure they are
> > > > available
> > > > > > with
> > > > > > >> > same
> > > > > > >> > > > > >> artifact
> > > > > > >> > > > > >> >>> > name
> > > > > > >> > > > > >> >>> > > >> and
> > > > > > >> > > > > >> >>> > > >> >> API.
> > > > > > >> > > > > >> >>> > > >> >> > > If this has to be changed, let's
> > wait
> > > > for
> > > > > > AI
> > > > > > >> 3.0
> > > > > > >> > > > > >> >>> > > >> >> > >
> > > > > > >> > > > > >> >>> > > >> >> > > Regards,
> > > > > > >> > > > > >> >>> > > >> >> > > --
> > > > > > >> > > > > >> >>> > > >> >> > > Ilya Kasnacheev
> > > > > > >> > > > > >> >>> > > >> >> > >
> > > > > > >> > > > > >> >>> > > >> >> > >
> > > > > > >> > > > > >> >>> > > >> >> > > ср, 23 окт. 2019 г. в 03:23,
> Saikat
> > > > > Maitra
> > > > > > <
> > > > > > >> > > > > >> >>> > > >> saikat.maitra@gmail.com>:
> > > > > > >> > > > > >> >>> > > >> >> > >
> > > > > > >> > > > > >> >>> > > >> >> > > > Hello Emmanouil,
> > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > As discussed earlier, I
> discussed
> > > > with
> > > > > > >> Apache
> > > > > > >> > > > Bahir
> > > > > > >> > > > > >> >>> community
> > > > > > >> > > > > >> >>> > > and
> > > > > > >> > > > > >> >>> > > >> >> they
> > > > > > >> > > > > >> >>> > > >> >> > > are
> > > > > > >> > > > > >> >>> > > >> >> > > > interested to have Apache
> Ignite
> > > > > > >> extensions as
> > > > > > >> > > > part
> > > > > > >> > > > > of
> > > > > > >> > > > > >> >>> Apache
> > > > > > >> > > > > >> >>> > > >> Bahir
> > > > > > >> > > > > >> >>> > > >> >> > > > project.
> > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > Can you please share Pub/Sub
> > > streamer
> > > > > > >> > > > implementation
> > > > > > >> > > > > >> >>> details
> > > > > > >> > > > > >> >>> > > with
> > > > > > >> > > > > >> >>> > > >> >> > Apache
> > > > > > >> > > > > >> >>> > > >> >> > > > Bahir community and request for
> > > > > feedback.
> > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > https://bahir.apache.org/
> > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > I have also requested for
> > > contributor
> > > > > > >> access
> > > > > > >> > in
> > > > > > >> > > > Jira
> > > > > > >> > > > > >> for
> > > > > > >> > > > > >> >>> > Apache
> > > > > > >> > > > > >> >>> > > >> >> Bahir
> > > > > > >> > > > > >> >>> > > >> >> > > > project so that I can create
> > issues
> > > > and
> > > > > > >> assign
> > > > > > >> > > to
> > > > > > >> > > > > >> myself.
> > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > I can help with code reviews as
> > > well.
> > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > Here is the email thread for
> > > > reference
> > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > >> > > > > >> >>> > >
> > > > > > >> > > >
> > > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > Regards,
> > > > > > >> > > > > >> >>> > > >> >> > > > Saikat
> > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM
> > > > Saikat
> > > > > > >> Maitra
> > > > > > >> > <
> > > > > > >> > > > > >> >>> > > >> >> saikat.maitra@gmail.com
> > > > > > >> > > > > >> >>> > > >> >> > >
> > > > > > >> > > > > >> >>> > > >> >> > > > wrote:
> > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > > Hello,
> > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > > As discussed I have created
> > > > following
> > > > > > >> > > discussion
> > > > > > >> > > > > >> >>> threads on
> > > > > > >> > > > > >> >>> > > our
> > > > > > >> > > > > >> >>> > > >> >> > > migration
> > > > > > >> > > > > >> >>> > > >> >> > > > > proposals for Apache Ignite
> > > > > extensions:
> > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > >> > > > > >> >>> > > >> >> > >
> > > > > > >> > > > > >> >>> > > >> >> >
> > > > > > >> > > > > >> >>> > > >> >>
> > > > > > >> > > > > >> >>> > > >>
> > > > > > >> > > > > >> >>> > >
> > > > > > >> > > > > >> >>> >
> > > > > > >> > > > > >> >>>
> > > > > > >> > > > > >>
> > > > > > >> > > > >
> > > > > > >> > > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > >> > > > > >> >>> > > >>
> > > > > > >> > > > > >>
> > > > > > >>
> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > > I will share update as I hear
> > > more
> > > > > > >> > > information.
> > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > > Regards,
> > > > > > >> > > > > >> >>> > > >> >> > > > > Saikat
> > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > > On Fri, Oct 18, 2019 at 9:03
> PM
> > > > > Saikat
> > > > > > >> > Maitra
> > > > > > >> > > <
> > > > > > >> > > > > >> >>> > > >> >> > saikat.maitra@gmail.com
> > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > > wrote:
> > > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >> Hello Denis,
> > > > > > >> > > > > >> >>> > > >> >> > > > >>
> > > > > > >> > > > > >> >>> > > >> >> > > > >> Sure, sounds good. I will
> > > create a
> > > > > > >> separate
> > > > > > >> > > > > >> discussion
> > > > > > >> > > > > >> >>> > > thread
> > > > > > >> > > > > >> >>> > > >> to
> > > > > > >> > > > > >> >>> > > >> >> get
> > > > > > >> > > > > >> >>> > > >> >> > > > >> community feedback on
> whether
> > we
> > > > > > >> > > > > >> >>> > > >> >> > > > >> should join the Bahir or
> > > kick-off
> > > > > > >> "Ignite
> > > > > > >> > > > > >> Extensions"
> > > > > > >> > > > > >> >>> > > project.
> > > > > > >> > > > > >> >>> > > >> >> > > > >>
> > > > > > >> > > > > >> >>> > > >> >> > > > >> Regards,
> > > > > > >> > > > > >> >>> > > >> >> > > > >> Saikat
> > > > > > >> > > > > >> >>> > > >> >> > > > >>
> > > > > > >> > > > > >> >>> > > >> >> > > > >> On Thu, Oct 17, 2019 at 2:21
> > PM
> > > > > Denis
> > > > > > >> > Magda <
> > > > > > >> > > > > >> >>> > > >> dmagda@apache.org>
> > > > > > >> > > > > >> >>> > > >> >> > > wrote:
> > > > > > >> > > > > >> >>> > > >> >> > > > >>
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> Folks,
> > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> The concept of Apache Bahir
> > is
> > > > > > >> precisely
> > > > > > >> > > what
> > > > > > >> > > > we
> > > > > > >> > > > > >> >>> need!
> > > > > > >> > > > > >> >>> > > >> Saikat,
> > > > > > >> > > > > >> >>> > > >> >> > thanks
> > > > > > >> > > > > >> >>> > > >> >> > > > for
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> doing the research. Why I
> > > believe
> > > > > > the *
> > > > > > >> > > idea*
> > > > > > >> > > > > >> fits us
> > > > > > >> > > > > >> >>> > well:
> > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > > >> > > > > >> >>> > > >> >> > > > >>>    - All integrations can
> be
> > > > stored
> > > > > > in
> > > > > > >> > > > separate
> > > > > > >> > > > > >> >>> Github
> > > > > > >> > > > > >> >>> > > >> >> repositories
> > > > > > >> > > > > >> >>> > > >> >> > > and
> > > > > > >> > > > > >> >>> > > >> >> > > > >>>    have their dev
> lifecycles.
> > > > We've
> > > > > > not
> > > > > > >> > > > obliged
> > > > > > >> > > > > to
> > > > > > >> > > > > >> >>> couple
> > > > > > >> > > > > >> >>> > > all
> > > > > > >> > > > > >> >>> > > >> >> the
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> integrations
> > > > > > >> > > > > >> >>> > > >> >> > > > >>>    in a single repo. For
> > > > instance,
> > > > > > >> Spark
> > > > > > >> > can
> > > > > > >> > > > be
> > > > > > >> > > > > >> >>> located
> > > > > > >> > > > > >> >>> > in
> > > > > > >> > > > > >> >>> > > a
> > > > > > >> > > > > >> >>> > > >> >> > > dedicated
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> repo
> > > > > > >> > > > > >> >>> > > >> >> > > > >>>    while streaming
> > integrations
> > > > > might
> > > > > > >> be
> > > > > > >> > in
> > > > > > >> > > a
> > > > > > >> > > > > >> single
> > > > > > >> > > > > >> >>> one.
> > > > > > >> > > > > >> >>> > > >> It's
> > > > > > >> > > > > >> >>> > > >> >> up
> > > > > > >> > > > > >> >>> > > >> >> > to
> > > > > > >> > > > > >> >>> > > >> >> > > > us.
> > > > > > >> > > > > >> >>> > > >> >> > > > >>>    - All the repositories
> and
> > > > > > >> integrations
> > > > > > >> > > > > belong
> > > > > > >> > > > > >> to
> > > > > > >> > > > > >> >>> ASF.
> > > > > > >> > > > > >> >>> > > >> We're
> > > > > > >> > > > > >> >>> > > >> >> not
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> dumping
> > > > > > >> > > > > >> >>> > > >> >> > > > >>>    them on Github but
> rather
> > > keep
> > > > > > >> > supporting
> > > > > > >> > > > and
> > > > > > >> > > > > >> >>> > developing
> > > > > > >> > > > > >> >>> > > >> in
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> accordance with
> > > > > > >> > > > > >> >>> > > >> >> > > > >>>    ASF vision and
> practices.
> > > > > > >> > > > > >> >>> > > >> >> > > > >>>    - There is a way to
> reward
> > > > > > >> contributors
> > > > > > >> > > via
> > > > > > >> > > > > >> >>> > > committership
> > > > > > >> > > > > >> >>> > > >> and
> > > > > > >> > > > > >> >>> > > >> >> > PMC
> > > > > > >> > > > > >> >>> > > >> >> > > > >>>    membership. You won't
> get
> > > this
> > > > > if
> > > > > > a
> > > > > > >> > > project
> > > > > > >> > > > > is
> > > > > > >> > > > > >> >>> just
> > > > > > >> > > > > >> >>> > one
> > > > > > >> > > > > >> >>> > > of
> > > > > > >> > > > > >> >>> > > >> >> the
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> millions of
> > > > > > >> > > > > >> >>> > > >> >> > > > >>>    Github projects.
> > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> Saikat, as Ignite PMC
> member,
> > > > could
> > > > > > you
> > > > > > >> > > please
> > > > > > >> > > > > >> >>> kick-off a
> > > > > > >> > > > > >> >>> > > >> >> separate
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> dev-list
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> discussion to involve more
> > > > > community
> > > > > > >> > members
> > > > > > >> > > > and
> > > > > > >> > > > > >> >>> > referring
> > > > > > >> > > > > >> >>> > > to
> > > > > > >> > > > > >> >>> > > >> >> this
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> thread?
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> I think the primary
> question
> > > the
> > > > > > >> community
> > > > > > >> > > > needs
> > > > > > >> > > > > >> to
> > > > > > >> > > > > >> >>> > answer
> > > > > > >> > > > > >> >>> > > >> >> whether
> > > > > > >> > > > > >> >>> > > >> >> > we
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> should join the Bahir or
> > > kick-off
> > > > > > >> "Ignite
> > > > > > >> > > > > >> Extensions"
> > > > > > >> > > > > >> >>> > > >> project?
> > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> -
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> Denis
> > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> On Thu, Oct 17, 2019 at
> 2:54
> > AM
> > > > > > Alexey
> > > > > > >> > > > Zinoviev
> > > > > > >> > > > > <
> > > > > > >> > > > > >> >>> > > >> >> > > > zaleslaw.sin@gmail.com>
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> wrote:
> > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > Maybe we could move all
> our
> > > > > > Streaming
> > > > > > >> > > > > >> Integrations
> > > > > > >> > > > > >> >>> > there,
> > > > > > >> > > > > >> >>> > > >> but
> > > > > > >> > > > > >> >>> > > >> >> > what
> > > > > > >> > > > > >> >>> > > >> >> > > is
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> about
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > maintaining and committer
> > > > > > >> permissions to
> > > > > > >> > > the
> > > > > > >> > > > > new
> > > > > > >> > > > > >> >>> > > >> repositories?
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > I see the list of the
> > > > committers
> > > > > > and
> > > > > > >> PMC
> > > > > > >> > > > > members
> > > > > > >> > > > > >> >>> there
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > > > >> > > https://bahir.apache.org/community-members/
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > Could somebody from
> Ignite
> > > > > > community
> > > > > > >> be
> > > > > > >> > > > added
> > > > > > >> > > > > to
> > > > > > >> > > > > >> >>> this
> > > > > > >> > > > > >> >>> > > list
> > > > > > >> > > > > >> >>> > > >> as
> > > > > > >> > > > > >> >>> > > >> >> a
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > PMC/committer to maintain
> > > > Ignite
> > > > > > >> > > > integrations?
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > If yes, I'd happy to join
> > > this
> > > > > > >> project
> > > > > > >> > and
> > > > > > >> > > > > >> >>> collaborate
> > > > > > >> > > > > >> >>> > > with
> > > > > > >> > > > > >> >>> > > >> >> these
> > > > > > >> > > > > >> >>> > > >> >> > > > guys
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > together
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > If no, and we should
> start
> > > from
> > > > > the
> > > > > > >> zero
> > > > > > >> > > > level
> > > > > > >> > > > > >> with
> > > > > > >> > > > > >> >>> > > >> external
> > > > > > >> > > > > >> >>> > > >> >> PRs
> > > > > > >> > > > > >> >>> > > >> >> > -
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> hmmm,
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > it's better to have our
> own
> > > > > > external
> > > > > > >> > > > > repository
> > > > > > >> > > > > >> >>> with
> > > > > > >> > > > > >> >>> > > >> >> ApacheBahirr
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> ideology.
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > Also, I agree, that all
> > > > > connectors
> > > > > > >> could
> > > > > > >> > > be
> > > > > > >> > > > > >> moved
> > > > > > >> > > > > >> >>> there
> > > > > > >> > > > > >> >>> > > (in
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> ApacheBahirr
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > like repository), but not
> > all
> > > > > > modules
> > > > > > >> > from
> > > > > > >> > > > the
> > > > > > >> > > > > >> page
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > >> > > > > >> >>> > > >> >> > >
> > > > > > >> > > > > >> >>> > > >> >> >
> > > > > > >> > > > > >> >>> > > >> >>
> > > > > > >> > > > > >> >>> > > >>
> > > > > > >> > > > > >> >>> > >
> > > > > > >> > > > > >> >>> >
> > > > > > >> > > > > >> >>>
> > > > > > >> > > > > >>
> > > > > > >> > > > >
> > > > > > >> > > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > because
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > they use different parts
> of
> > > > core
> > > > > > >> modules
> > > > > > >> > > and
> > > > > > >> > > > > >> could
> > > > > > >> > > > > >> >>> be
> > > > > > >> > > > > >> >>> > > >> >> developed
> > > > > > >> > > > > >> >>> > > >> >> > > with
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > different velocity.
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > In reality, before
> creation
> > > of
> > > > > new
> > > > > > >> > > > > repositories
> > > > > > >> > > > > >> we
> > > > > > >> > > > > >> >>> need
> > > > > > >> > > > > >> >>> > > >> wide
> > > > > > >> > > > > >> >>> > > >> >> > > > discussion
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > based on the document
> > > mentioned
> > > > > > >> above.
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > P.S. Of course, we could
> > > > > experiment
> > > > > > >> in
> > > > > > >> > the
> > > > > > >> > > > > next
> > > > > > >> > > > > >> >>> release
> > > > > > >> > > > > >> >>> > > 2.8
> > > > > > >> > > > > >> >>> > > >> >> with
> > > > > > >> > > > > >> >>> > > >> >> > > one
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> or two
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > integrations like twitter
> > or
> > > > > ZeroMQ
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > Also, I'd like Denis
> Magda
> > > idea
> > > > > for
> > > > > > >> the
> > > > > > >> > > > > separate
> > > > > > >> > > > > >> >>> > > >> repositories
> > > > > > >> > > > > >> >>> > > >> >> in
> > > > > > >> > > > > >> >>> > > >> >> > > > Apache
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > Github to maintain them
> by
> > > > > anybody.
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > чт, 17 окт. 2019 г. в
> > 05:02,
> > > > > Saikat
> > > > > > >> > > Maitra <
> > > > > > >> > > > > >> >>> > > >> >> > > saikat.maitra@gmail.com
> > > > > > >> > > > > >> >>> > > >> >> > > > >:
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > Hi Denis, Emmanouil
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > Thank you for your
> > email. I
> > > > > think
> > > > > > >> > > > creating a
> > > > > > >> > > > > >> >>> separate
> > > > > > >> > > > > >> >>> > > >> >> > integration
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> project
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > in github and also
> > > proposing
> > > > it
> > > > > > as
> > > > > > >> an
> > > > > > >> > > > apache
> > > > > > >> > > > > >> >>> > incubator
> > > > > > >> > > > > >> >>> > > >> >> project
> > > > > > >> > > > > >> >>> > > >> >> > > will
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> be a
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > good move. The other
> > > separate
> > > > > > >> > > integration
> > > > > > >> > > > > >> modules
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > >> > > > > >> >>> > > >> >> > >
> > > > > > >> > > > > >> >>> > > >> >> >
> > > > > > >> > > > > >> >>> > > >> >>
> > > > > > >> > > > > >> >>> > > >>
> > > > > > >> > > > > >> >>> > >
> > > > > > >> > > > > >> >>> >
> > > > > > >> > > > > >> >>>
> > > > > > >> > > > > >>
> > > > > > >> > > > >
> > > > > > >> > > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > can
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > be moved to this new
> > apache
> > > > > > >> incubator
> > > > > > >> > > > > project.
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > There are similar
> > > integration
> > > > > > >> > available
> > > > > > >> > > > for
> > > > > > >> > > > > >> >>> Flink and
> > > > > > >> > > > > >> >>> > > >> Spark
> > > > > > >> > > > > >> >>> > > >> >> in
> > > > > > >> > > > > >> >>> > > >> >> > > > Apache
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > Bahir
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > https://bahir.apache.org/
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > Do you think we should
> > > reach
> > > > > out
> > > > > > to
> > > > > > >> > > Apache
> > > > > > >> > > > > >> Bahir
> > > > > > >> > > > > >> >>> > > >> community
> > > > > > >> > > > > >> >>> > > >> >> > with a
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > proposal
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > or we should plan to
> > > create a
> > > > > > >> separate
> > > > > > >> > > > > Apache
> > > > > > >> > > > > >> >>> > Incubator
> > > > > > >> > > > > >> >>> > > >> >> > project?
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > Regards,
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > Saikat
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > On Wed, Oct 16, 2019 at
> > > 6:21
> > > > AM
> > > > > > >> > > Emmanouil
> > > > > > >> > > > > >> >>> > Gkatziouras <
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > gkatzioura@gmail.com>
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > wrote:
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > Hi all!
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > Based on the
> > discussions
> > > > most
> > > > > > >> > probably
> > > > > > >> > > > > this
> > > > > > >> > > > > >> is
> > > > > > >> > > > > >> >>> > going
> > > > > > >> > > > > >> >>> > > to
> > > > > > >> > > > > >> >>> > > >> >> be on
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> another
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > GitHub repo.
> Therefore
> > I
> > > > will
> > > > > > >> > prepare
> > > > > > >> > > a
> > > > > > >> > > > > >> >>> standalone
> > > > > > >> > > > > >> >>> > > >> project
> > > > > > >> > > > > >> >>> > > >> >> > with
> > > > > > >> > > > > >> >>> > > >> >> > > > the
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > Pub/Sub
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > module and once the
> > > > > repository
> > > > > > is
> > > > > > >> > > > created
> > > > > > >> > > > > a
> > > > > > >> > > > > >> >>> pull
> > > > > > >> > > > > >> >>> > > >> request
> > > > > > >> > > > > >> >>> > > >> >> > shall
> > > > > > >> > > > > >> >>> > > >> >> > > be
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > issued.
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > If there is anything
> > > else I
> > > > > can
> > > > > > >> do
> > > > > > >> > in
> > > > > > >> > > > > order
> > > > > > >> > > > > >> to
> > > > > > >> > > > > >> >>> > > >> facilitate
> > > > > > >> > > > > >> >>> > > >> >> > this
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> please
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > let
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > me know.
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > If a ticket is
> created
> > > for
> > > > > this
> > > > > > >> > > issue, I
> > > > > > >> > > > > >> shall
> > > > > > >> > > > > >> >>> be
> > > > > > >> > > > > >> >>> > > able
> > > > > > >> > > > > >> >>> > > >> to
> > > > > > >> > > > > >> >>> > > >> >> > > create
> > > > > > >> > > > > >> >>> > > >> >> > > > a
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> pull
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > request and thus a
> > build
> > > > will
> > > > > > >> take
> > > > > > >> > > place
> > > > > > >> > > > > on
> > > > > > >> > > > > >> >>> > ignite's
> > > > > > >> > > > > >> >>> > > >> CI.
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > If everyone is
> aligned
> > > with
> > > > > > >> adding a
> > > > > > >> > > > > Pub/Sub
> > > > > > >> > > > > >> >>> > > >> >> implementation
> > > > > > >> > > > > >> >>> > > >> >> > > may I
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > create
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > a
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > ticket on JIRA?
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > My next question has
> to
> > > do
> > > > on
> > > > > > how
> > > > > > >> > > Ignite
> > > > > > >> > > > > >> >>> configures
> > > > > > >> > > > > >> >>> > > the
> > > > > > >> > > > > >> >>> > > >> >> build
> > > > > > >> > > > > >> >>> > > >> >> > > > jobs
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> in
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > Team
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > City. The unit tests
> > for
> > > my
> > > > > > >> > > > implementation
> > > > > > >> > > > > >> >>> need a
> > > > > > >> > > > > >> >>> > > local
> > > > > > >> > > > > >> >>> > > >> >> > Pub/Sub
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> server
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > up
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > and running (they
> > provide
> > > > one
> > > > > > for
> > > > > > >> > > > testing
> > > > > > >> > > > > >> >>> > purposes).
> > > > > > >> > > > > >> >>> > > >> Will
> > > > > > >> > > > > >> >>> > > >> >> > this
> > > > > > >> > > > > >> >>> > > >> >> > > > take
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > effect
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > in a form of a build
> > > script
> > > > > > >> inside
> > > > > > >> > my
> > > > > > >> > > > > >> >>> > implementation
> > > > > > >> > > > > >> >>> > > or
> > > > > > >> > > > > >> >>> > > >> >> is it
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> something
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > that should be
> > configured
> > > > on
> > > > > > Team
> > > > > > >> > > City?
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > Kind regards,
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > Emmanouil
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > *Emmanouil
> Gkatziouras*
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > https://egkatzioura.com/
> > > |
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > > >> > > > > >> >>>
> > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > https://github.com/gkatzioura
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > On Tue, 15 Oct 2019
> at
> > > > 23:27,
> > > > > > >> Denis
> > > > > > >> > > > Magda
> > > > > > >> > > > > <
> > > > > > >> > > > > >> >>> > > >> >> dmagda@apache.org
> > > > > > >> > > > > >> >>> > > >> >> > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> wrote:
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > Emmanouil, Saikat,
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > After contemplating
> > on
> > > > > this,
> > > > > > >> let
> > > > > > >> > me
> > > > > > >> > > > > >> propose
> > > > > > >> > > > > >> >>> > another
> > > > > > >> > > > > >> >>> > > >> way
> > > > > > >> > > > > >> >>> > > >> >> > with
> > > > > > >> > > > > >> >>> > > >> >> > > > this
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > particular
> > integration
> > > > that
> > > > > > is
> > > > > > >> > > aligned
> > > > > > >> > > > > >> with
> > > > > > >> > > > > >> >>> our
> > > > > > >> > > > > >> >>> > > >> >> > > modularization
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > endeavor.
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > The modularization
> > [1]
> > > > > > defines
> > > > > > >> the
> > > > > > >> > > > > Ignite
> > > > > > >> > > > > >> >>> core,
> > > > > > >> > > > > >> >>> > > >> Ignite
> > > > > > >> > > > > >> >>> > > >> >> > > modules,
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> and
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > independent
> > > integrations.
> > > > > The
> > > > > > >> > first
> > > > > > >> > > > two
> > > > > > >> > > > > >> are
> > > > > > >> > > > > >> >>> to be
> > > > > > >> > > > > >> >>> > > >> >> located
> > > > > > >> > > > > >> >>> > > >> >> > in
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> Ignite
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > repositories and to
> > be
> > > > > > >> > > tested/updated
> > > > > > >> > > > by
> > > > > > >> > > > > >> the
> > > > > > >> > > > > >> >>> > > >> community
> > > > > > >> > > > > >> >>> > > >> >> for
> > > > > > >> > > > > >> >>> > > >> >> > > > every
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > release.
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > The modules are a
> > > subset
> > > > of
> > > > > > >> > existing
> > > > > > >> > > > > >> >>> integrations
> > > > > > >> > > > > >> >>> > > >> that
> > > > > > >> > > > > >> >>> > > >> >> are
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> adopted
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > widely
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > and require
> community
> > > > > > >> attention.
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > As for the
> > independent
> > > > > > >> > integrations
> > > > > > >> > > > [1],
> > > > > > >> > > > > >> >>> those
> > > > > > >> > > > > >> >>> > are
> > > > > > >> > > > > >> >>> > > >> >> supposed
> > > > > > >> > > > > >> >>> > > >> >> > > to
> > > > > > >> > > > > >> >>> > > >> >> > > > be
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > located
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > in a separate
> GitHub
> > > > repo.
> > > > > I
> > > > > > >> would
> > > > > > >> > > > > advise
> > > > > > >> > > > > >> us
> > > > > > >> > > > > >> >>> to
> > > > > > >> > > > > >> >>> > > start
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> implementing
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > this
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > concept with the
> > > Pub/Sub
> > > > > > >> > > integration.
> > > > > > >> > > > > Even
> > > > > > >> > > > > >> >>> though
> > > > > > >> > > > > >> >>> > > it
> > > > > > >> > > > > >> >>> > > >> >> will
> > > > > > >> > > > > >> >>> > > >> >> > be
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> located
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > in a
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > separate Github
> repo,
> > > it
> > > > > will
> > > > > > >> be
> > > > > > >> > > > listed
> > > > > > >> > > > > >> among
> > > > > > >> > > > > >> >>> > > >> officially
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> available
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > integrations on the
> > > > Ignite
> > > > > > >> website
> > > > > > >> > > > > >> featuring
> > > > > > >> > > > > >> >>> > > >> Emmanouil
> > > > > > >> > > > > >> >>> > > >> >> as a
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > contributor.
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > What do you think?
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > [1]
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > > >> > > > > >> >>> > > >> >> > >
> > > > > > >> > > > > >> >>> > > >> >> >
> > > > > > >> > > > > >> >>> > > >> >>
> > > > > > >> > > > > >> >>> > > >>
> > > > > > >> > > > > >> >>> > >
> > > > > > >> > > > > >> >>> >
> > > > > > >> > > > > >> >>>
> > > > > > >> > > > > >>
> > > > > > >> > > > >
> > > > > > >> > > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > -
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > Denis
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > On Mon, Oct 14,
> 2019
> > at
> > > > > 9:47
> > > > > > PM
> > > > > > >> > > Saikat
> > > > > > >> > > > > >> >>> Maitra <
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > saikat.maitra@gmail.com
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > wrote:
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > Hi Emmanouil,
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > The changes looks
> > > good
> > > > to
> > > > > > >> me. I
> > > > > > >> > > > wanted
> > > > > > >> > > > > >> to
> > > > > > >> > > > > >> >>> check
> > > > > > >> > > > > >> >>> > > if
> > > > > > >> > > > > >> >>> > > >> you
> > > > > > >> > > > > >> >>> > > >> >> > are
> > > > > > >> > > > > >> >>> > > >> >> > > > also
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > able
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > to
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > validate that the
> > > once
> > > > > data
> > > > > > >> is
> > > > > > >> > > added
> > > > > > >> > > > > to
> > > > > > >> > > > > >> >>> Ignite
> > > > > > >> > > > > >> >>> > > >> cluster
> > > > > > >> > > > > >> >>> > > >> >> > you
> > > > > > >> > > > > >> >>> > > >> >> > > > are
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> also
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > able
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > to
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > access it using
> > > another
> > > > > > >> Ignite
> > > > > > >> > > > client
> > > > > > >> > > > > or
> > > > > > >> > > > > >> >>> using
> > > > > > >> > > > > >> >>> > > rest
> > > > > > >> > > > > >> >>> > > >> >> > > > endpoints.
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > We use Teamcity
> for
> > > CI
> > > > > > >> process,
> > > > > > >> > > the
> > > > > > >> > > > > >> >>> details are
> > > > > > >> > > > > >> >>> > > as
> > > > > > >> > > > > >> >>> > > >> >> > follows
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > Trigger
> validation
> > of
> > > > > those
> > > > > > >> test
> > > > > > >> > > > > suites
> > > > > > >> > > > > >> >>> that
> > > > > > >> > > > > >> >>> > have
> > > > > > >> > > > > >> >>> > > >> been
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> affected by
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > your
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > changes on
> > TeamCity <
> > > > > > >> > > > > >> >>> > > http://ci.ignite.apache.org/>:
> > > > > > >> > > > > >> >>> > > >> <
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > http://204.14.53.153/
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Locate a
> test
> > > > suite
> > > > > > you
> > > > > > >> > have
> > > > > > >> > > to
> > > > > > >> > > > > >> check,
> > > > > > >> > > > > >> >>> > press
> > > > > > >> > > > > >> >>> > > >> >> button
> > > > > > >> > > > > >> >>> > > >> >> > > > named
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > "..."
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > that
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    is located on
> > the
> > > > left
> > > > > > of
> > > > > > >> > "Run"
> > > > > > >> > > > > >> button.
> > > > > > >> > > > > >> >>> "Run
> > > > > > >> > > > > >> >>> > > >> custom
> > > > > > >> > > > > >> >>> > > >> >> > > build"
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > dialog
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > will
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    appear;
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Go to
> > "Changes"
> > > > tab
> > > > > > and
> > > > > > >> > > choose
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > "pull/<pull-request-number>/head"
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > in
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    "Build branch"
> > > > > dropdown
> > > > > > >> list;
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Press "Run
> > > build"
> > > > > > button
> > > > > > >> > and
> > > > > > >> > > > > >> monitor
> > > > > > >> > > > > >> >>> tests
> > > > > > >> > > > > >> >>> > > >> >> results.
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > @Roman
> > > > rshtykh@yahoo.com
> > > > > -
> > > > > > >> can
> > > > > > >> > > you
> > > > > > >> > > > > also
> > > > > > >> > > > > >> >>> please
> > > > > > >> > > > > >> >>> > > >> take a
> > > > > > >> > > > > >> >>> > > >> >> > > look?
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > Regards,
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > Saikat
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > On Sun, Oct 13,
> > 2019
> > > at
> > > > > > 6:20
> > > > > > >> PM
> > > > > > >> > > > > >> Emmanouil
> > > > > > >> > > > > >> >>> > > >> Gkatziouras
> > > > > > >> > > > > >> >>> > > >> >> <
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > gkatzioura@gmail.com
> > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > wrote:
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Hi @Saikat
> Maitra
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > I have my
> > > > > implementation
> > > > > > >> on an
> > > > > > >> > > > > Ignite
> > > > > > >> > > > > >> >>> fork on
> > > > > > >> > > > > >> >>> > > my
> > > > > > >> > > > > >> >>> > > >> >> > personal
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> GitHub
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > account
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > [1]
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > It is based on
> > the
> > > > > Kafka
> > > > > > >> > > > > >> implementation
> > > > > > >> > > > > >> >>> which
> > > > > > >> > > > > >> >>> > > is
> > > > > > >> > > > > >> >>> > > >> of
> > > > > > >> > > > > >> >>> > > >> >> a
> > > > > > >> > > > > >> >>> > > >> >> > > > polling
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > nature.
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > GCP does not
> > > provide
> > > > a
> > > > > > >> service
> > > > > > >> > > > like
> > > > > > >> > > > > >> >>> Kafka.
> > > > > > >> > > > > >> >>> > > >> Instead
> > > > > > >> > > > > >> >>> > > >> >> it
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> provides us
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > with
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Pub/Sub which
> is
> > > just
> > > > > > like
> > > > > > >> > > Kafka.
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Therefore using
> > the
> > > > > Kafka
> > > > > > >> > > streamer
> > > > > > >> > > > > as
> > > > > > >> > > > > >> a
> > > > > > >> > > > > >> >>> > > guideline
> > > > > > >> > > > > >> >>> > > >> >> made
> > > > > > >> > > > > >> >>> > > >> >> > > more
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> sense
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > to
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > me.
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Apart from the
> > > > > > >> implementation
> > > > > > >> > I
> > > > > > >> > > > have
> > > > > > >> > > > > >> also
> > > > > > >> > > > > >> >>> > > added a
> > > > > > >> > > > > >> >>> > > >> >> unit
> > > > > > >> > > > > >> >>> > > >> >> > > > test.
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > It definitely
> > needs
> > > > > some
> > > > > > >> > > polishing
> > > > > > >> > > > > >> and a
> > > > > > >> > > > > >> >>> > double
> > > > > > >> > > > > >> >>> > > >> >> check
> > > > > > >> > > > > >> >>> > > >> >> > on
> > > > > > >> > > > > >> >>> > > >> >> > > my
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> side
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > to
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > make
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > the review
> > smooth.
> > > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > For the tests a
> > > local
> > > > > > >> Pub/Sub
> > > > > > >> > > [2]
> > > > > > >> > > > > >> server
> > > > > > >> > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Emmanouil Gkatziouras <gk...@gmail.com>]

Hi Saikat!

Thank you on this one! I tried it and it worked as expected [1]

Kind regards
Emmanouil

[1]
https://ci.ignite.apache.org/viewLog.html?buildId=4888357&tab=buildLog&_focus=2046#_state=2046

*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura


On Thu, 26 Dec 2019 at 19:50, Saikat Maitra <sa...@gmail.com> wrote:

> Hi Emmanouil,
>
> I have been able to resolve the build failures for missing modules.
>
>
> https://ci.ignite.apache.org/viewLog.html?buildId=4883394&buildTypeId=IgniteExtensions_Build
>
> Will you please take the latest changes from my PR and run the build for
> your PR.
>
> https://github.com/apache/ignite-extensions/pull/1/files
>
> The new build configurations should let your module build also pass.
>
> The issue that was causing failure was that I had put ignite.zip as
> artifact dependency and it was downloading the artifacts including pom file
> inside the build checkout folder of ignite-extensions and as a result the
> pom.xml of ignite-extensions was getting overwritten by ignite's pom.xml.
> This was resulting in missing flink-ext module error as it was not even
> present in the ignite main pom.xml.
>
> To resolve the issue I just had to download and extract artifact of
> ignite.zip in a separate directory.
>
> Regards,
> Saikat
>
>
> On Sat, Nov 30, 2019 at 5:53 PM Emmanouil Gkatziouras <
> gkatzioura@gmail.com>
> wrote:
>
> > Hi Saikat!
> >
> > Thank you for your assistance on that!
> >
> > Kind regards
> >
> > *Emmanouil Gkatziouras*
> > https://egkatzioura.com/ |
> > https://www.linkedin.com/in/gkatziourasemmanouil/
> > https://github.com/gkatzioura
> >
> >
> > On Sat, 30 Nov 2019 at 23:51, Saikat Maitra <sa...@gmail.com>
> > wrote:
> >
> > > Hi Emmanouil,
> > >
> > > I looked into the build logs and I observed that since I added
> artifacts
> > > dependencies to "Build apache Ignite"[1] it was able to pull the
> required
> > > dependencies and was able to run the existing tests. It is however not
> > > identifying new modules like I changed from flink to flink-ext or
> > pub-sub.
> > >
> > > [1]
> > >
> > >
> >
> https://ci.ignite.apache.org/admin/editDependencies.html?id=buildType:IgniteExtensions_Build
> > >
> > > I will look into the issue and debug further.
> > >
> > > Regards,
> > > Saikat
> > >
> > >
> > > On Sat, Nov 30, 2019 at 5:35 PM Emmanouil Gkatziouras <
> > > gkatzioura@gmail.com>
> > > wrote:
> > >
> > > > Hi Saikat!
> > > >
> > > > From the logs[1] it seems that TC cannot find the project. However I
> > did
> > > > include the project on the parent pom (modules sections) [2]
> > > > I tried to reproduce locally unfortunately I am not aware of the full
> > > > context on TC. It seems as if a different parent pom is being picked
> > up.
> > > >
> > > > Kind regards
> > > >
> > > > [1]
> > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4804893&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=1704
> > > > [2]
> > > >
> > > >
> > >
> >
> https://github.com/gkatzioura/ignite-extensions/blob/d6a8beee7feff28f59d9a12be692016305564d25/pom.xml#L46
> > > >
> > > > *Emmanouil Gkatziouras*
> > > > https://egkatzioura.com/ |
> > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > https://github.com/gkatzioura
> > > >
> > > >
> > > > On Sat, 30 Nov 2019 at 22:20, Saikat Maitra <saikat.maitra@gmail.com
> >
> > > > wrote:
> > > >
> > > > > Hello,
> > > > >
> > > > > I have made changes in Ignite Extensions build configurations.
> > > > >
> > > > > It is now running the tests as expected.
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4804480&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> > > > >
> > > > > Regards,
> > > > > Saikat
> > > > >
> > > > > On Thu, Nov 28, 2019 at 1:29 PM Saikat Maitra <
> > saikat.maitra@gmail.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > We do run build with -DskipTests and then configure selective
> test
> > > > suites
> > > > > > to run.
> > > > > >
> > > > > > I will check why these test suites are not getting executed.
> > > > > >
> > > > > > Regards
> > > > > > Saikat
> > > > > >
> > > > > > On Wed, 27 Nov 2019 at 3:05 PM, Emmanouil Gkatziouras <
> > > > > > gkatzioura@gmail.com> wrote:
> > > > > >
> > > > > >> Hi all,
> > > > > >>
> > > > > >> I did add a test suite and run the build with the corresponding
> > > > > arguments,
> > > > > >> however the tests did not run [1].
> > > > > >> I checked the Flink's build logs and the same message is
> displayed
> > > > there
> > > > > >> too `No tests to run` [2].
> > > > > >>
> > > > > >> [1]
> > > > > >>
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4798203&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=248
> > > > > >> [2]
> > > > > >>
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4789199&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=1256
> > > > > >>
> > > > > >> Kind regards
> > > > > >> *Emmanouil Gkatziouras*
> > > > > >> https://egkatzioura.com/ |
> > > > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > >> https://github.com/gkatzioura
> > > > > >>
> > > > > >>
> > > > > >> On Wed, 27 Nov 2019 at 18:58, Saikat Maitra <
> > > saikat.maitra@gmail.com>
> > > > > >> wrote:
> > > > > >>
> > > > > >> > Hi,
> > > > > >> >
> > > > > >> > We will need to add test suites for tests to be executed in
> > build.
> > > > We
> > > > > >> pass
> > > > > >> > them as TestSuites param. I had added Flink sink and source
> > > > > testsuites.
> > > > > >> >
> > > > > >> > Can you please review and confirm.
> > > > > >> >
> > > > > >> > Regards
> > > > > >> > Saikat
> > > > > >> >
> > > > > >> > On Wed, 27 Nov 2019 at 6:00 AM, Emmanouil Gkatziouras <
> > > > > >> > gkatzioura@gmail.com>
> > > > > >> > wrote:
> > > > > >> >
> > > > > >> > > Hi all!
> > > > > >> > >
> > > > > >> > > I was successful building and running my pull requests. It
> > seems
> > > > > that
> > > > > >> the
> > > > > >> > > tests do not run both for Flink and Pub/Sub [1]
> > > > > >> > > If there is something I cannot do to make them discoverable
> > > please
> > > > > >> let me
> > > > > >> > > know.
> > > > > >> > >
> > > > > >> > > [1]
> > > > > >> > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4796722&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=283
> > > > > >> > >
> > > > > >> > > *Emmanouil Gkatziouras*
> > > > > >> > > https://egkatzioura.com/ |
> > > > > >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > >> > > https://github.com/gkatzioura
> > > > > >> > >
> > > > > >> > >
> > > > > >> > > On Wed, 27 Nov 2019 at 04:57, Saikat Maitra <
> > > > > saikat.maitra@gmail.com>
> > > > > >> > > wrote:
> > > > > >> > >
> > > > > >> > > > Hi Emmanouil,
> > > > > >> > > >
> > > > > >> > > > I have added you as contributor in Ignite Extensions
> > project.
> > > > Can
> > > > > >> you
> > > > > >> > > > please check and confirm if you are able to see the
> project
> > > and
> > > > > >> execute
> > > > > >> > > > build on your pull request.
> > > > > >> > > >
> > > > > >> > > > Regards,
> > > > > >> > > > Saikat
> > > > > >> > > >
> > > > > >> > > > On Tue, Nov 26, 2019 at 12:49 PM Emmanouil Gkatziouras <
> > > > > >> > > > gkatzioura@gmail.com>
> > > > > >> > > > wrote:
> > > > > >> > > >
> > > > > >> > > > > Hi all!
> > > > > >> > > > >
> > > > > >> > > > > If someone could give me read access to the Job of
> Ignite
> > > > > >> Extensions
> > > > > >> > on
> > > > > >> > > > > Team City will greatly help me (username gkatzioura).
> > > > > >> > > > > I suppose the builds get triggered automatically on a
> pull
> > > > > >> request.
> > > > > >> > > > >
> > > > > >> > > > > Kind regards,
> > > > > >> > > > > Emmanouil
> > > > > >> > > > >
> > > > > >> > > > > *Emmanouil Gkatziouras*
> > > > > >> > > > > https://egkatzioura.com/ |
> > > > > >> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > >> > > > > https://github.com/gkatzioura
> > > > > >> > > > >
> > > > > >> > > > >
> > > > > >> > > > > On Sun, 24 Nov 2019 at 21:11, Emmanouil Gkatziouras <
> > > > > >> > > > gkatzioura@gmail.com>
> > > > > >> > > > > wrote:
> > > > > >> > > > >
> > > > > >> > > > > > Hi Saikat!
> > > > > >> > > > > >
> > > > > >> > > > > > I just rebased with the flink branch. Unfortunately I
> do
> > > not
> > > > > >> have
> > > > > >> > > read
> > > > > >> > > > > > access to the team city link you provided and thus
> > > evaluate
> > > > > the
> > > > > >> > tests
> > > > > >> > > > for
> > > > > >> > > > > > my pull request.
> > > > > >> > > > > > I guess it has to do with ignite extensions being new.
> > > > > >> > > > > > My username is gkatzioura and email the one I am using
> > on
> > > > this
> > > > > >> > email
> > > > > >> > > > > >
> > > > > >> > > > > > Thank you for your time.
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > > > *Emmanouil Gkatziouras*
> > > > > >> > > > > > https://egkatzioura.com/ |
> > > > > >> > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > >> > > > > > https://github.com/gkatzioura
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > > > On Sun, 24 Nov 2019 at 18:03, Saikat Maitra <
> > > > > >> > saikat.maitra@gmail.com
> > > > > >> > > >
> > > > > >> > > > > > wrote:
> > > > > >> > > > > >
> > > > > >> > > > > >> Hi Emmanouil,
> > > > > >> > > > > >>
> > > > > >> > > > > >> The latest build on teamcity has passed on Flink pull
> > > > > request.
> > > > > >> > > > > >>
> > > > > >> > > > > >>
> > > > > >> > > > > >>
> > > > > >> > > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4788928&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> > > > > >> > > > > >>
> > > > > >> > > > > >> you should be able to take the changes I made in my
> PR
> > > and
> > > > > run
> > > > > >> > build
> > > > > >> > > > on
> > > > > >> > > > > >> your pull request.
> > > > > >> > > > > >>
> > > > > >> > > > > >> Please let me know if you have any questions.
> > > > > >> > > > > >>
> > > > > >> > > > > >> Regards,
> > > > > >> > > > > >> Saikat
> > > > > >> > > > > >>
> > > > > >> > > > > >> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra <
> > > > > >> > > > saikat.maitra@gmail.com
> > > > > >> > > > > >
> > > > > >> > > > > >> wrote:
> > > > > >> > > > > >>
> > > > > >> > > > > >> > Hi Emmanouil,
> > > > > >> > > > > >> >
> > > > > >> > > > > >> > I have fixed the Flink module testsuites and have
> > setup
> > > > new
> > > > > >> > > project
> > > > > >> > > > in
> > > > > >> > > > > >> > teamcity.
> > > > > >> > > > > >> >
> > > > > >> > > > > >> >
> > > > > >> > > > > >> >
> > > > > >> > > > > >>
> > > > > >> > > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview
> > > > > >> > > > > >> >
> > > > > >> > > > > >> > I am looking into setting up the build.
> > > > > >> > > > > >> >
> > > > > >> > > > > >> > Regards,
> > > > > >> > > > > >> > Saikat
> > > > > >> > > > > >> >
> > > > > >> > > > > >> > On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra <
> > > > > >> > > > > saikat.maitra@gmail.com>
> > > > > >> > > > > >> > wrote:
> > > > > >> > > > > >> >
> > > > > >> > > > > >> >> Hi Emmanouil,
> > > > > >> > > > > >> >>
> > > > > >> > > > > >> >> Thank you for your email. Yes, the plan is once
> the
> > > > Flink
> > > > > >> PR is
> > > > > >> > > > > merged
> > > > > >> > > > > >> it
> > > > > >> > > > > >> >> will provide the Licence, parent POMs etc and you
> > can
> > > > > rebase
> > > > > >> > from
> > > > > >> > > > > >> master
> > > > > >> > > > > >> >> branch and apply the changes on top of it.
> > > > > >> > > > > >> >>
> > > > > >> > > > > >> >> I am getting some issues with test failures in
> local
> > > > with
> > > > > >> > > > > >> >> GridTestProperties as the test.properties is not
> > > present
> > > > > in
> > > > > >> > this
> > > > > >> > > > new
> > > > > >> > > > > >> >> project but should be available from dependencies.
> > > > > >> > > > > >> >>
> > > > > >> > > > > >> >> Once I address this issue, I will go ahead and
> merge
> > > the
> > > > > >> > changes
> > > > > >> > > > and
> > > > > >> > > > > >> then
> > > > > >> > > > > >> >> we can take it from there.
> > > > > >> > > > > >> >>
> > > > > >> > > > > >> >> Regards,
> > > > > >> > > > > >> >> Saikat
> > > > > >> > > > > >> >>
> > > > > >> > > > > >> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil
> > Gkatziouras
> > > <
> > > > > >> > > > > >> >> gkatzioura@gmail.com> wrote:
> > > > > >> > > > > >> >>
> > > > > >> > > > > >> >>> Hi all,
> > > > > >> > > > > >> >>>
> > > > > >> > > > > >> >>> I made my first pull request [1]. Since this
> > project
> > > is
> > > > > >> brand
> > > > > >> > > new
> > > > > >> > > > > (no
> > > > > >> > > > > >> >>> parent poms, licensing), I reckoned it was best
> to
> > > use
> > > > > >> > Saikat's
> > > > > >> > > > > >> branch on
> > > > > >> > > > > >> >>> Flink.
> > > > > >> > > > > >> >>> I suppose the Flink branch will be merged first.
> If
> > > not
> > > > > >> please
> > > > > >> > > > give
> > > > > >> > > > > me
> > > > > >> > > > > >> >>> guidelines on how I should proceed next.
> > > > > >> > > > > >> >>>
> > > > > >> > > > > >> >>> Kind regards
> > > > > >> > > > > >> >>> Emmanouil
> > > > > >> > > > > >> >>>
> > > > > >> > > > > >> >>> [1]
> > > https://github.com/apache/ignite-extensions/pull/2
> > > > > >> > > > > >> >>>
> > > > > >> > > > > >> >>> *Emmanouil Gkatziouras*
> > > > > >> > > > > >> >>> https://egkatzioura.com/ |
> > > > > >> > > > > >> >>>
> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > >> > > > > >> >>> https://github.com/gkatzioura
> > > > > >> > > > > >> >>>
> > > > > >> > > > > >> >>>
> > > > > >> > > > > >> >>> On Fri, 22 Nov 2019 at 20:55, Denis Magda <
> > > > > >> dmagda@apache.org>
> > > > > >> > > > > wrote:
> > > > > >> > > > > >> >>>
> > > > > >> > > > > >> >>> > Awesome, ping us whenever you're ready!
> > > > > >> > > > > >> >>> >
> > > > > >> > > > > >> >>> > -
> > > > > >> > > > > >> >>> > Denis
> > > > > >> > > > > >> >>> >
> > > > > >> > > > > >> >>> >
> > > > > >> > > > > >> >>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil
> > > > Gkatziouras
> > > > > <
> > > > > >> > > > > >> >>> > gkatzioura@gmail.com>
> > > > > >> > > > > >> >>> > wrote:
> > > > > >> > > > > >> >>> >
> > > > > >> > > > > >> >>> > > Hi all!
> > > > > >> > > > > >> >>> > >
> > > > > >> > > > > >> >>> > > I am sorry for being late on that. I was
> trying
> > > to
> > > > > >> > refactor
> > > > > >> > > > the
> > > > > >> > > > > >> test
> > > > > >> > > > > >> >>> in
> > > > > >> > > > > >> >>> > > order not to be in need of any external tools
> > or
> > > > > >> spinning
> > > > > >> > > up a
> > > > > >> > > > > >> >>> server.
> > > > > >> > > > > >> >>> > > I did forked the new repo and indeed my
> changes
> > > > > there,
> > > > > >> so
> > > > > >> > a
> > > > > >> > > > pull
> > > > > >> > > > > >> >>> request
> > > > > >> > > > > >> >>> > is
> > > > > >> > > > > >> >>> > > a matter of time!
> > > > > >> > > > > >> >>> > >
> > > > > >> > > > > >> >>> > > Kind regards
> > > > > >> > > > > >> >>> > >
> > > > > >> > > > > >> >>> > > *Emmanouil Gkatziouras*
> > > > > >> > > > > >> >>> > > https://egkatzioura.com/ |
> > > > > >> > > > > >> >>> > >
> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > >> > > > > >> >>> > > https://github.com/gkatzioura
> > > > > >> > > > > >> >>> > >
> > > > > >> > > > > >> >>> > >
> > > > > >> > > > > >> >>> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda <
> > > > > >> > > dmagda@apache.org>
> > > > > >> > > > > >> wrote:
> > > > > >> > > > > >> >>> > >
> > > > > >> > > > > >> >>> > > > Hi Emmanouil,
> > > > > >> > > > > >> >>> > > >
> > > > > >> > > > > >> >>> > > > Do you have any questions or need any
> support
> > > > from
> > > > > >> the
> > > > > >> > > > > >> community?
> > > > > >> > > > > >> >>> > > >
> > > > > >> > > > > >> >>> > > > -
> > > > > >> > > > > >> >>> > > > Denis
> > > > > >> > > > > >> >>> > > >
> > > > > >> > > > > >> >>> > > >
> > > > > >> > > > > >> >>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat
> > Maitra <
> > > > > >> > > > > >> >>> saikat.maitra@gmail.com
> > > > > >> > > > > >> >>> > >
> > > > > >> > > > > >> >>> > > > wrote:
> > > > > >> > > > > >> >>> > > >
> > > > > >> > > > > >> >>> > > >> Hi Emmanouil,
> > > > > >> > > > > >> >>> > > >>
> > > > > >> > > > > >> >>> > > >> Can you please take a looks at dev utils,
> if
> > > > this
> > > > > is
> > > > > >> > > > > something
> > > > > >> > > > > >> >>> you are
> > > > > >> > > > > >> >>> > > >> looking for
> > > > > >> > > > > >> >>> > > >>
> > > > > >> > > > >
> > > > https://github.com/apache/ignite/tree/master/modules/dev-utils
> > > > > >> > > > > >> ?
> > > > > >> > > > > >> >>> > > >>
> > > > > >> > > > > >> >>> > > >> IMO, if you can release Pub/Sub server in
> > > maven
> > > > > and
> > > > > >> > then
> > > > > >> > > > use
> > > > > >> > > > > >> it as
> > > > > >> > > > > >> >>> > > >> dependency, that would be great.
> > > > > >> > > > > >> >>> > > >>
> > > > > >> > > > > >> >>> > > >> Regards,
> > > > > >> > > > > >> >>> > > >> Saikat
> > > > > >> > > > > >> >>> > > >>
> > > > > >> > > > > >> >>> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat
> > Maitra
> > > <
> > > > > >> > > > > >> >>> > saikat.maitra@gmail.com>
> > > > > >> > > > > >> >>> > > >> wrote:
> > > > > >> > > > > >> >>> > > >>
> > > > > >> > > > > >> >>> > > >> > Hi Emmanouil,
> > > > > >> > > > > >> >>> > > >> >
> > > > > >> > > > > >> >>> > > >> > The master branch has been initialized.
> I
> > > have
> > > > > >> > raised a
> > > > > >> > > > PR
> > > > > >> > > > > to
> > > > > >> > > > > >> >>> > migrate
> > > > > >> > > > > >> >>> > > >> > Flink module to ignite-extensions.
> > > > > >> > > > > >> >>> > > >> >
> > > > > >> > > > > >> >>> > > >> > PR
> > > > > >> > https://github.com/apache/ignite-extensions/pull/1
> > > > > >> > > > > >> >>> > > >> > Jira :
> > > > > >> > > > https://issues.apache.org/jira/browse/IGNITE-12356
> > > > > >> > > > > >> >>> > > >> >
> > > > > >> > > > > >> >>> > > >> > Regards,
> > > > > >> > > > > >> >>> > > >> > Saikat
> > > > > >> > > > > >> >>> > > >> >
> > > > > >> > > > > >> >>> > > >> >
> > > > > >> > > > > >> >>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM
> Emmanouil
> > > > > >> > Gkatziouras <
> > > > > >> > > > > >> >>> > > >> > gkatzioura@gmail.com> wrote:
> > > > > >> > > > > >> >>> > > >> >
> > > > > >> > > > > >> >>> > > >> >> Hi all!
> > > > > >> > > > > >> >>> > > >> >>
> > > > > >> > > > > >> >>> > > >> >> This is an update. The integration is
> > > tested
> > > > > with
> > > > > >> > > mocks
> > > > > >> > > > > >> based
> > > > > >> > > > > >> >>> on
> > > > > >> > > > > >> >>> > the
> > > > > >> > > > > >> >>> > > >> >> original Ignite repo [1].
> > > > > >> > > > > >> >>> > > >> >> I am currently waiting for the first
> > > commits
> > > > on
> > > > > >> the
> > > > > >> > > new
> > > > > >> > > > > >> >>> integration
> > > > > >> > > > > >> >>> > > >> >> project
> > > > > >> > > > > >> >>> > > >> >> to take place in order to create my
> pull
> > > > > request
> > > > > >> > > > > accordingly
> > > > > >> > > > > >> >>> to the
> > > > > >> > > > > >> >>> > > >> >> initial
> > > > > >> > > > > >> >>> > > >> >> commits.
> > > > > >> > > > > >> >>> > > >> >> In order to simulate the Pub/Sub server
> > in
> > > a
> > > > > >> test I
> > > > > >> > > > used a
> > > > > >> > > > > >> test
> > > > > >> > > > > >> >>> > > server
> > > > > >> > > > > >> >>> > > >> >> implementation [2]. Unfortunately they
> > > don't
> > > > > have
> > > > > >> > this
> > > > > >> > > > > util
> > > > > >> > > > > >> >>> > uploaded
> > > > > >> > > > > >> >>> > > >> in a
> > > > > >> > > > > >> >>> > > >> >> maven repo, thus it needs to be hosted
> > on a
> > > > > maven
> > > > > >> > repo
> > > > > >> > > > > >> (Apache
> > > > > >> > > > > >> >>> > > License
> > > > > >> > > > > >> >>> > > >> >> 2.0). Is there a repo for utilities
> that
> > > > Ignite
> > > > > >> uses
> > > > > >> > > for
> > > > > >> > > > > the
> > > > > >> > > > > >> >>> > > >> test/build?
> > > > > >> > > > > >> >>> > > >> >> I have also created a ticket [3]
> > > > > >> > > > > >> >>> > > >> >>
> > > > > >> > > > > >> >>> > > >> >> Kind regards
> > > > > >> > > > > >> >>> > > >> >>
> > > > > >> > > > > >> >>> > > >> >> [1]
> > > > > >> > > > > >> >>> >
> > > > > >> > >
> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > > > > >> > > > > >> >>> > > >> >> [2]
> > > > > >> > > > > >> >>>
> > > > > >> https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> > > > > >> > > > > >> >>> > > >> >> [3]
> > > > > >> > > https://issues.apache.org/jira/browse/IGNITE-12262
> > > > > >> > > > > >> >>> > > >> >>
> > > > > >> > > > > >> >>> > > >> >> *Emmanouil Gkatziouras*
> > > > > >> > > > > >> >>> > > >> >> https://egkatzioura.com/ |
> > > > > >> > > > > >> >>> > > >> >>
> > > > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > >> > > > > >> >>> > > >> >> https://github.com/gkatzioura
> > > > > >> > > > > >> >>> > > >> >>
> > > > > >> > > > > >> >>> > > >> >>
> > > > > >> > > > > >> >>> > > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat
> > > Maitra <
> > > > > >> > > > > >> >>> > saikat.maitra@gmail.com
> > > > > >> > > > > >> >>> > > >
> > > > > >> > > > > >> >>> > > >> >> wrote:
> > > > > >> > > > > >> >>> > > >> >>
> > > > > >> > > > > >> >>> > > >> >> > Hello Ilya,
> > > > > >> > > > > >> >>> > > >> >> >
> > > > > >> > > > > >> >>> > > >> >> > Thank you for your email. Yes, before
> > we
> > > > > remove
> > > > > >> > any
> > > > > >> > > > > >> >>> integration
> > > > > >> > > > > >> >>> > we
> > > > > >> > > > > >> >>> > > >> will
> > > > > >> > > > > >> >>> > > >> >> > move them to another repository and
> > > > release.
> > > > > >> > > > > >> >>> > > >> >> >
> > > > > >> > > > > >> >>> > > >> >> > We are thinking we will start with
> > > Pub/Sub
> > > > > >> > Streamer
> > > > > >> > > > > >> >>> > implementation
> > > > > >> > > > > >> >>> > > >> and
> > > > > >> > > > > >> >>> > > >> >> > continue to move other integration as
> > > > > >> mentioned in
> > > > > >> > > the
> > > > > >> > > > > >> >>> following
> > > > > >> > > > > >> >>> > > doc.
> > > > > >> > > > > >> >>> > > >> >> >
> > > > > >> > > > > >> >>> > > >> >> >
> > > > > >> > > > > >> >>> > > >> >> >
> > > > > >> > > > > >> >>> > > >> >>
> > > > > >> > > > > >> >>> > > >>
> > > > > >> > > > > >> >>> > >
> > > > > >> > > > > >> >>> >
> > > > > >> > > > > >> >>>
> > > > > >> > > > > >>
> > > > > >> > > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > > >> > > > > >> >>> > > >> >> >
> > > > > >> > > > > >> >>> > > >> >> > Regards,
> > > > > >> > > > > >> >>> > > >> >> > Saikat
> > > > > >> > > > > >> >>> > > >> >> >
> > > > > >> > > > > >> >>> > > >> >> >
> > > > > >> > > > > >> >>> > > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya
> > > > > >> Kasnacheev <
> > > > > >> > > > > >> >>> > > >> >> > ilya.kasnacheev@gmail.com>
> > > > > >> > > > > >> >>> > > >> >> > wrote:
> > > > > >> > > > > >> >>> > > >> >> >
> > > > > >> > > > > >> >>> > > >> >> > > Hello!
> > > > > >> > > > > >> >>> > > >> >> > >
> > > > > >> > > > > >> >>> > > >> >> > > My take on this: we want to offload
> > > some
> > > > of
> > > > > >> > > > > >> integrations,
> > > > > >> > > > > >> >>> but
> > > > > >> > > > > >> >>> > > >> please
> > > > > >> > > > > >> >>> > > >> >> > commit
> > > > > >> > > > > >> >>> > > >> >> > > and release them to another project
> > > > first,
> > > > > >> and
> > > > > >> > > only
> > > > > >> > > > > >> then we
> > > > > >> > > > > >> >>> > will
> > > > > >> > > > > >> >>> > > >> >> decide
> > > > > >> > > > > >> >>> > > >> >> > to
> > > > > >> > > > > >> >>> > > >> >> > > remove them from our repository.
> > > > > >> > > > > >> >>> > > >> >> > >
> > > > > >> > > > > >> >>> > > >> >> > > Please also make sure they are
> > > available
> > > > > with
> > > > > >> > same
> > > > > >> > > > > >> artifact
> > > > > >> > > > > >> >>> > name
> > > > > >> > > > > >> >>> > > >> and
> > > > > >> > > > > >> >>> > > >> >> API.
> > > > > >> > > > > >> >>> > > >> >> > > If this has to be changed, let's
> wait
> > > for
> > > > > AI
> > > > > >> 3.0
> > > > > >> > > > > >> >>> > > >> >> > >
> > > > > >> > > > > >> >>> > > >> >> > > Regards,
> > > > > >> > > > > >> >>> > > >> >> > > --
> > > > > >> > > > > >> >>> > > >> >> > > Ilya Kasnacheev
> > > > > >> > > > > >> >>> > > >> >> > >
> > > > > >> > > > > >> >>> > > >> >> > >
> > > > > >> > > > > >> >>> > > >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat
> > > > Maitra
> > > > > <
> > > > > >> > > > > >> >>> > > >> saikat.maitra@gmail.com>:
> > > > > >> > > > > >> >>> > > >> >> > >
> > > > > >> > > > > >> >>> > > >> >> > > > Hello Emmanouil,
> > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > >> > > > > >> >>> > > >> >> > > > As discussed earlier, I discussed
> > > with
> > > > > >> Apache
> > > > > >> > > > Bahir
> > > > > >> > > > > >> >>> community
> > > > > >> > > > > >> >>> > > and
> > > > > >> > > > > >> >>> > > >> >> they
> > > > > >> > > > > >> >>> > > >> >> > > are
> > > > > >> > > > > >> >>> > > >> >> > > > interested to have Apache Ignite
> > > > > >> extensions as
> > > > > >> > > > part
> > > > > >> > > > > of
> > > > > >> > > > > >> >>> Apache
> > > > > >> > > > > >> >>> > > >> Bahir
> > > > > >> > > > > >> >>> > > >> >> > > > project.
> > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > >> > > > > >> >>> > > >> >> > > > Can you please share Pub/Sub
> > streamer
> > > > > >> > > > implementation
> > > > > >> > > > > >> >>> details
> > > > > >> > > > > >> >>> > > with
> > > > > >> > > > > >> >>> > > >> >> > Apache
> > > > > >> > > > > >> >>> > > >> >> > > > Bahir community and request for
> > > > feedback.
> > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > >> > > > > >> >>> > > >> >> > > > https://bahir.apache.org/
> > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > >> > > > > >> >>> > > >> >> > > > I have also requested for
> > contributor
> > > > > >> access
> > > > > >> > in
> > > > > >> > > > Jira
> > > > > >> > > > > >> for
> > > > > >> > > > > >> >>> > Apache
> > > > > >> > > > > >> >>> > > >> >> Bahir
> > > > > >> > > > > >> >>> > > >> >> > > > project so that I can create
> issues
> > > and
> > > > > >> assign
> > > > > >> > > to
> > > > > >> > > > > >> myself.
> > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > >> > > > > >> >>> > > >> >> > > > I can help with code reviews as
> > well.
> > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > >> > > > > >> >>> > > >> >> > > > Here is the email thread for
> > > reference
> > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > >> > > > > >> >>> > >
> > > > > >> > > >
> > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > >> > > > > >> >>> > > >> >> > > > Regards,
> > > > > >> > > > > >> >>> > > >> >> > > > Saikat
> > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > >> > > > > >> >>> > > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM
> > > Saikat
> > > > > >> Maitra
> > > > > >> > <
> > > > > >> > > > > >> >>> > > >> >> saikat.maitra@gmail.com
> > > > > >> > > > > >> >>> > > >> >> > >
> > > > > >> > > > > >> >>> > > >> >> > > > wrote:
> > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > >> > > > > >> >>> > > >> >> > > > > Hello,
> > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > > As discussed I have created
> > > following
> > > > > >> > > discussion
> > > > > >> > > > > >> >>> threads on
> > > > > >> > > > > >> >>> > > our
> > > > > >> > > > > >> >>> > > >> >> > > migration
> > > > > >> > > > > >> >>> > > >> >> > > > > proposals for Apache Ignite
> > > > extensions:
> > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > >> > > > > >> >>> > > >> >> > >
> > > > > >> > > > > >> >>> > > >> >> >
> > > > > >> > > > > >> >>> > > >> >>
> > > > > >> > > > > >> >>> > > >>
> > > > > >> > > > > >> >>> > >
> > > > > >> > > > > >> >>> >
> > > > > >> > > > > >> >>>
> > > > > >> > > > > >>
> > > > > >> > > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > >> > > > > >> >>> > > >>
> > > > > >> > > > > >>
> > > > > >> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > > I will share update as I hear
> > more
> > > > > >> > > information.
> > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > > Regards,
> > > > > >> > > > > >> >>> > > >> >> > > > > Saikat
> > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM
> > > > Saikat
> > > > > >> > Maitra
> > > > > >> > > <
> > > > > >> > > > > >> >>> > > >> >> > saikat.maitra@gmail.com
> > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > >> > > > > >> >>> > > >> >> > > > > wrote:
> > > > > >> > > > > >> >>> > > >> >> > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >> Hello Denis,
> > > > > >> > > > > >> >>> > > >> >> > > > >>
> > > > > >> > > > > >> >>> > > >> >> > > > >> Sure, sounds good. I will
> > create a
> > > > > >> separate
> > > > > >> > > > > >> discussion
> > > > > >> > > > > >> >>> > > thread
> > > > > >> > > > > >> >>> > > >> to
> > > > > >> > > > > >> >>> > > >> >> get
> > > > > >> > > > > >> >>> > > >> >> > > > >> community feedback on whether
> we
> > > > > >> > > > > >> >>> > > >> >> > > > >> should join the Bahir or
> > kick-off
> > > > > >> "Ignite
> > > > > >> > > > > >> Extensions"
> > > > > >> > > > > >> >>> > > project.
> > > > > >> > > > > >> >>> > > >> >> > > > >>
> > > > > >> > > > > >> >>> > > >> >> > > > >> Regards,
> > > > > >> > > > > >> >>> > > >> >> > > > >> Saikat
> > > > > >> > > > > >> >>> > > >> >> > > > >>
> > > > > >> > > > > >> >>> > > >> >> > > > >> On Thu, Oct 17, 2019 at 2:21
> PM
> > > > Denis
> > > > > >> > Magda <
> > > > > >> > > > > >> >>> > > >> dmagda@apache.org>
> > > > > >> > > > > >> >>> > > >> >> > > wrote:
> > > > > >> > > > > >> >>> > > >> >> > > > >>
> > > > > >> > > > > >> >>> > > >> >> > > > >>> Folks,
> > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > >> > > > > >> >>> > > >> >> > > > >>> The concept of Apache Bahir
> is
> > > > > >> precisely
> > > > > >> > > what
> > > > > >> > > > we
> > > > > >> > > > > >> >>> need!
> > > > > >> > > > > >> >>> > > >> Saikat,
> > > > > >> > > > > >> >>> > > >> >> > thanks
> > > > > >> > > > > >> >>> > > >> >> > > > for
> > > > > >> > > > > >> >>> > > >> >> > > > >>> doing the research. Why I
> > believe
> > > > > the *
> > > > > >> > > idea*
> > > > > >> > > > > >> fits us
> > > > > >> > > > > >> >>> > well:
> > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > >> > > > > >> >>> > > >> >> > > > >>>    - All integrations can be
> > > stored
> > > > > in
> > > > > >> > > > separate
> > > > > >> > > > > >> >>> Github
> > > > > >> > > > > >> >>> > > >> >> repositories
> > > > > >> > > > > >> >>> > > >> >> > > and
> > > > > >> > > > > >> >>> > > >> >> > > > >>>    have their dev lifecycles.
> > > We've
> > > > > not
> > > > > >> > > > obliged
> > > > > >> > > > > to
> > > > > >> > > > > >> >>> couple
> > > > > >> > > > > >> >>> > > all
> > > > > >> > > > > >> >>> > > >> >> the
> > > > > >> > > > > >> >>> > > >> >> > > > >>> integrations
> > > > > >> > > > > >> >>> > > >> >> > > > >>>    in a single repo. For
> > > instance,
> > > > > >> Spark
> > > > > >> > can
> > > > > >> > > > be
> > > > > >> > > > > >> >>> located
> > > > > >> > > > > >> >>> > in
> > > > > >> > > > > >> >>> > > a
> > > > > >> > > > > >> >>> > > >> >> > > dedicated
> > > > > >> > > > > >> >>> > > >> >> > > > >>> repo
> > > > > >> > > > > >> >>> > > >> >> > > > >>>    while streaming
> integrations
> > > > might
> > > > > >> be
> > > > > >> > in
> > > > > >> > > a
> > > > > >> > > > > >> single
> > > > > >> > > > > >> >>> one.
> > > > > >> > > > > >> >>> > > >> It's
> > > > > >> > > > > >> >>> > > >> >> up
> > > > > >> > > > > >> >>> > > >> >> > to
> > > > > >> > > > > >> >>> > > >> >> > > > us.
> > > > > >> > > > > >> >>> > > >> >> > > > >>>    - All the repositories and
> > > > > >> integrations
> > > > > >> > > > > belong
> > > > > >> > > > > >> to
> > > > > >> > > > > >> >>> ASF.
> > > > > >> > > > > >> >>> > > >> We're
> > > > > >> > > > > >> >>> > > >> >> not
> > > > > >> > > > > >> >>> > > >> >> > > > >>> dumping
> > > > > >> > > > > >> >>> > > >> >> > > > >>>    them on Github but rather
> > keep
> > > > > >> > supporting
> > > > > >> > > > and
> > > > > >> > > > > >> >>> > developing
> > > > > >> > > > > >> >>> > > >> in
> > > > > >> > > > > >> >>> > > >> >> > > > >>> accordance with
> > > > > >> > > > > >> >>> > > >> >> > > > >>>    ASF vision and practices.
> > > > > >> > > > > >> >>> > > >> >> > > > >>>    - There is a way to reward
> > > > > >> contributors
> > > > > >> > > via
> > > > > >> > > > > >> >>> > > committership
> > > > > >> > > > > >> >>> > > >> and
> > > > > >> > > > > >> >>> > > >> >> > PMC
> > > > > >> > > > > >> >>> > > >> >> > > > >>>    membership. You won't get
> > this
> > > > if
> > > > > a
> > > > > >> > > project
> > > > > >> > > > > is
> > > > > >> > > > > >> >>> just
> > > > > >> > > > > >> >>> > one
> > > > > >> > > > > >> >>> > > of
> > > > > >> > > > > >> >>> > > >> >> the
> > > > > >> > > > > >> >>> > > >> >> > > > >>> millions of
> > > > > >> > > > > >> >>> > > >> >> > > > >>>    Github projects.
> > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > >> > > > > >> >>> > > >> >> > > > >>> Saikat, as Ignite PMC member,
> > > could
> > > > > you
> > > > > >> > > please
> > > > > >> > > > > >> >>> kick-off a
> > > > > >> > > > > >> >>> > > >> >> separate
> > > > > >> > > > > >> >>> > > >> >> > > > >>> dev-list
> > > > > >> > > > > >> >>> > > >> >> > > > >>> discussion to involve more
> > > > community
> > > > > >> > members
> > > > > >> > > > and
> > > > > >> > > > > >> >>> > referring
> > > > > >> > > > > >> >>> > > to
> > > > > >> > > > > >> >>> > > >> >> this
> > > > > >> > > > > >> >>> > > >> >> > > > >>> thread?
> > > > > >> > > > > >> >>> > > >> >> > > > >>> I think the primary question
> > the
> > > > > >> community
> > > > > >> > > > needs
> > > > > >> > > > > >> to
> > > > > >> > > > > >> >>> > answer
> > > > > >> > > > > >> >>> > > >> >> whether
> > > > > >> > > > > >> >>> > > >> >> > we
> > > > > >> > > > > >> >>> > > >> >> > > > >>> should join the Bahir or
> > kick-off
> > > > > >> "Ignite
> > > > > >> > > > > >> Extensions"
> > > > > >> > > > > >> >>> > > >> project?
> > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > >> > > > > >> >>> > > >> >> > > > >>> -
> > > > > >> > > > > >> >>> > > >> >> > > > >>> Denis
> > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > >> > > > > >> >>> > > >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54
> AM
> > > > > Alexey
> > > > > >> > > > Zinoviev
> > > > > >> > > > > <
> > > > > >> > > > > >> >>> > > >> >> > > > zaleslaw.sin@gmail.com>
> > > > > >> > > > > >> >>> > > >> >> > > > >>> wrote:
> > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > Maybe we could move all our
> > > > > Streaming
> > > > > >> > > > > >> Integrations
> > > > > >> > > > > >> >>> > there,
> > > > > >> > > > > >> >>> > > >> but
> > > > > >> > > > > >> >>> > > >> >> > what
> > > > > >> > > > > >> >>> > > >> >> > > is
> > > > > >> > > > > >> >>> > > >> >> > > > >>> about
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > maintaining and committer
> > > > > >> permissions to
> > > > > >> > > the
> > > > > >> > > > > new
> > > > > >> > > > > >> >>> > > >> repositories?
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > I see the list of the
> > > committers
> > > > > and
> > > > > >> PMC
> > > > > >> > > > > members
> > > > > >> > > > > >> >>> there
> > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > > >> > > https://bahir.apache.org/community-members/
> > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > Could somebody from Ignite
> > > > > community
> > > > > >> be
> > > > > >> > > > added
> > > > > >> > > > > to
> > > > > >> > > > > >> >>> this
> > > > > >> > > > > >> >>> > > list
> > > > > >> > > > > >> >>> > > >> as
> > > > > >> > > > > >> >>> > > >> >> a
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > PMC/committer to maintain
> > > Ignite
> > > > > >> > > > integrations?
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > If yes, I'd happy to join
> > this
> > > > > >> project
> > > > > >> > and
> > > > > >> > > > > >> >>> collaborate
> > > > > >> > > > > >> >>> > > with
> > > > > >> > > > > >> >>> > > >> >> these
> > > > > >> > > > > >> >>> > > >> >> > > > guys
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > together
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > If no, and we should start
> > from
> > > > the
> > > > > >> zero
> > > > > >> > > > level
> > > > > >> > > > > >> with
> > > > > >> > > > > >> >>> > > >> external
> > > > > >> > > > > >> >>> > > >> >> PRs
> > > > > >> > > > > >> >>> > > >> >> > -
> > > > > >> > > > > >> >>> > > >> >> > > > >>> hmmm,
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > it's better to have our own
> > > > > external
> > > > > >> > > > > repository
> > > > > >> > > > > >> >>> with
> > > > > >> > > > > >> >>> > > >> >> ApacheBahirr
> > > > > >> > > > > >> >>> > > >> >> > > > >>> ideology.
> > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > Also, I agree, that all
> > > > connectors
> > > > > >> could
> > > > > >> > > be
> > > > > >> > > > > >> moved
> > > > > >> > > > > >> >>> there
> > > > > >> > > > > >> >>> > > (in
> > > > > >> > > > > >> >>> > > >> >> > > > >>> ApacheBahirr
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > like repository), but not
> all
> > > > > modules
> > > > > >> > from
> > > > > >> > > > the
> > > > > >> > > > > >> page
> > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > >> > > > > >> >>> > > >> >> > >
> > > > > >> > > > > >> >>> > > >> >> >
> > > > > >> > > > > >> >>> > > >> >>
> > > > > >> > > > > >> >>> > > >>
> > > > > >> > > > > >> >>> > >
> > > > > >> > > > > >> >>> >
> > > > > >> > > > > >> >>>
> > > > > >> > > > > >>
> > > > > >> > > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > because
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > they use different parts of
> > > core
> > > > > >> modules
> > > > > >> > > and
> > > > > >> > > > > >> could
> > > > > >> > > > > >> >>> be
> > > > > >> > > > > >> >>> > > >> >> developed
> > > > > >> > > > > >> >>> > > >> >> > > with
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > different velocity.
> > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > In reality, before creation
> > of
> > > > new
> > > > > >> > > > > repositories
> > > > > >> > > > > >> we
> > > > > >> > > > > >> >>> need
> > > > > >> > > > > >> >>> > > >> wide
> > > > > >> > > > > >> >>> > > >> >> > > > discussion
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > based on the document
> > mentioned
> > > > > >> above.
> > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > P.S. Of course, we could
> > > > experiment
> > > > > >> in
> > > > > >> > the
> > > > > >> > > > > next
> > > > > >> > > > > >> >>> release
> > > > > >> > > > > >> >>> > > 2.8
> > > > > >> > > > > >> >>> > > >> >> with
> > > > > >> > > > > >> >>> > > >> >> > > one
> > > > > >> > > > > >> >>> > > >> >> > > > >>> or two
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > integrations like twitter
> or
> > > > ZeroMQ
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > Also, I'd like Denis Magda
> > idea
> > > > for
> > > > > >> the
> > > > > >> > > > > separate
> > > > > >> > > > > >> >>> > > >> repositories
> > > > > >> > > > > >> >>> > > >> >> in
> > > > > >> > > > > >> >>> > > >> >> > > > Apache
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > Github to maintain them by
> > > > anybody.
> > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > чт, 17 окт. 2019 г. в
> 05:02,
> > > > Saikat
> > > > > >> > > Maitra <
> > > > > >> > > > > >> >>> > > >> >> > > saikat.maitra@gmail.com
> > > > > >> > > > > >> >>> > > >> >> > > > >:
> > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > Hi Denis, Emmanouil
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > Thank you for your
> email. I
> > > > think
> > > > > >> > > > creating a
> > > > > >> > > > > >> >>> separate
> > > > > >> > > > > >> >>> > > >> >> > integration
> > > > > >> > > > > >> >>> > > >> >> > > > >>> project
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > in github and also
> > proposing
> > > it
> > > > > as
> > > > > >> an
> > > > > >> > > > apache
> > > > > >> > > > > >> >>> > incubator
> > > > > >> > > > > >> >>> > > >> >> project
> > > > > >> > > > > >> >>> > > >> >> > > will
> > > > > >> > > > > >> >>> > > >> >> > > > >>> be a
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > good move. The other
> > separate
> > > > > >> > > integration
> > > > > >> > > > > >> modules
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > >> > > > > >> >>> > > >> >> > >
> > > > > >> > > > > >> >>> > > >> >> >
> > > > > >> > > > > >> >>> > > >> >>
> > > > > >> > > > > >> >>> > > >>
> > > > > >> > > > > >> >>> > >
> > > > > >> > > > > >> >>> >
> > > > > >> > > > > >> >>>
> > > > > >> > > > > >>
> > > > > >> > > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > can
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > be moved to this new
> apache
> > > > > >> incubator
> > > > > >> > > > > project.
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > There are similar
> > integration
> > > > > >> > available
> > > > > >> > > > for
> > > > > >> > > > > >> >>> Flink and
> > > > > >> > > > > >> >>> > > >> Spark
> > > > > >> > > > > >> >>> > > >> >> in
> > > > > >> > > > > >> >>> > > >> >> > > > Apache
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > Bahir
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > >
> https://bahir.apache.org/
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > Do you think we should
> > reach
> > > > out
> > > > > to
> > > > > >> > > Apache
> > > > > >> > > > > >> Bahir
> > > > > >> > > > > >> >>> > > >> community
> > > > > >> > > > > >> >>> > > >> >> > with a
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > proposal
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > or we should plan to
> > create a
> > > > > >> separate
> > > > > >> > > > > Apache
> > > > > >> > > > > >> >>> > Incubator
> > > > > >> > > > > >> >>> > > >> >> > project?
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > Regards,
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > Saikat
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > On Wed, Oct 16, 2019 at
> > 6:21
> > > AM
> > > > > >> > > Emmanouil
> > > > > >> > > > > >> >>> > Gkatziouras <
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > gkatzioura@gmail.com>
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > wrote:
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > Hi all!
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > Based on the
> discussions
> > > most
> > > > > >> > probably
> > > > > >> > > > > this
> > > > > >> > > > > >> is
> > > > > >> > > > > >> >>> > going
> > > > > >> > > > > >> >>> > > to
> > > > > >> > > > > >> >>> > > >> >> be on
> > > > > >> > > > > >> >>> > > >> >> > > > >>> another
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > GitHub repo. Therefore
> I
> > > will
> > > > > >> > prepare
> > > > > >> > > a
> > > > > >> > > > > >> >>> standalone
> > > > > >> > > > > >> >>> > > >> project
> > > > > >> > > > > >> >>> > > >> >> > with
> > > > > >> > > > > >> >>> > > >> >> > > > the
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > Pub/Sub
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > module and once the
> > > > repository
> > > > > is
> > > > > >> > > > created
> > > > > >> > > > > a
> > > > > >> > > > > >> >>> pull
> > > > > >> > > > > >> >>> > > >> request
> > > > > >> > > > > >> >>> > > >> >> > shall
> > > > > >> > > > > >> >>> > > >> >> > > be
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > issued.
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > If there is anything
> > else I
> > > > can
> > > > > >> do
> > > > > >> > in
> > > > > >> > > > > order
> > > > > >> > > > > >> to
> > > > > >> > > > > >> >>> > > >> facilitate
> > > > > >> > > > > >> >>> > > >> >> > this
> > > > > >> > > > > >> >>> > > >> >> > > > >>> please
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > let
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > me know.
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > If a ticket is created
> > for
> > > > this
> > > > > >> > > issue, I
> > > > > >> > > > > >> shall
> > > > > >> > > > > >> >>> be
> > > > > >> > > > > >> >>> > > able
> > > > > >> > > > > >> >>> > > >> to
> > > > > >> > > > > >> >>> > > >> >> > > create
> > > > > >> > > > > >> >>> > > >> >> > > > a
> > > > > >> > > > > >> >>> > > >> >> > > > >>> pull
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > request and thus a
> build
> > > will
> > > > > >> take
> > > > > >> > > place
> > > > > >> > > > > on
> > > > > >> > > > > >> >>> > ignite's
> > > > > >> > > > > >> >>> > > >> CI.
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > If everyone is aligned
> > with
> > > > > >> adding a
> > > > > >> > > > > Pub/Sub
> > > > > >> > > > > >> >>> > > >> >> implementation
> > > > > >> > > > > >> >>> > > >> >> > > may I
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > create
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > a
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > ticket on JIRA?
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > My next question has to
> > do
> > > on
> > > > > how
> > > > > >> > > Ignite
> > > > > >> > > > > >> >>> configures
> > > > > >> > > > > >> >>> > > the
> > > > > >> > > > > >> >>> > > >> >> build
> > > > > >> > > > > >> >>> > > >> >> > > > jobs
> > > > > >> > > > > >> >>> > > >> >> > > > >>> in
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > Team
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > City. The unit tests
> for
> > my
> > > > > >> > > > implementation
> > > > > >> > > > > >> >>> need a
> > > > > >> > > > > >> >>> > > local
> > > > > >> > > > > >> >>> > > >> >> > Pub/Sub
> > > > > >> > > > > >> >>> > > >> >> > > > >>> server
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > up
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > and running (they
> provide
> > > one
> > > > > for
> > > > > >> > > > testing
> > > > > >> > > > > >> >>> > purposes).
> > > > > >> > > > > >> >>> > > >> Will
> > > > > >> > > > > >> >>> > > >> >> > this
> > > > > >> > > > > >> >>> > > >> >> > > > take
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > effect
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > in a form of a build
> > script
> > > > > >> inside
> > > > > >> > my
> > > > > >> > > > > >> >>> > implementation
> > > > > >> > > > > >> >>> > > or
> > > > > >> > > > > >> >>> > > >> >> is it
> > > > > >> > > > > >> >>> > > >> >> > > > >>> something
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > that should be
> configured
> > > on
> > > > > Team
> > > > > >> > > City?
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > Kind regards,
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > Emmanouil
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > *Emmanouil Gkatziouras*
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> https://egkatzioura.com/
> > |
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> > > > > >> >>>
> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > https://github.com/gkatzioura
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > On Tue, 15 Oct 2019 at
> > > 23:27,
> > > > > >> Denis
> > > > > >> > > > Magda
> > > > > >> > > > > <
> > > > > >> > > > > >> >>> > > >> >> dmagda@apache.org
> > > > > >> > > > > >> >>> > > >> >> > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> wrote:
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > Emmanouil, Saikat,
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > After contemplating
> on
> > > > this,
> > > > > >> let
> > > > > >> > me
> > > > > >> > > > > >> propose
> > > > > >> > > > > >> >>> > another
> > > > > >> > > > > >> >>> > > >> way
> > > > > >> > > > > >> >>> > > >> >> > with
> > > > > >> > > > > >> >>> > > >> >> > > > this
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > particular
> integration
> > > that
> > > > > is
> > > > > >> > > aligned
> > > > > >> > > > > >> with
> > > > > >> > > > > >> >>> our
> > > > > >> > > > > >> >>> > > >> >> > > modularization
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > endeavor.
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > The modularization
> [1]
> > > > > defines
> > > > > >> the
> > > > > >> > > > > Ignite
> > > > > >> > > > > >> >>> core,
> > > > > >> > > > > >> >>> > > >> Ignite
> > > > > >> > > > > >> >>> > > >> >> > > modules,
> > > > > >> > > > > >> >>> > > >> >> > > > >>> and
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > independent
> > integrations.
> > > > The
> > > > > >> > first
> > > > > >> > > > two
> > > > > >> > > > > >> are
> > > > > >> > > > > >> >>> to be
> > > > > >> > > > > >> >>> > > >> >> located
> > > > > >> > > > > >> >>> > > >> >> > in
> > > > > >> > > > > >> >>> > > >> >> > > > >>> Ignite
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > repositories and to
> be
> > > > > >> > > tested/updated
> > > > > >> > > > by
> > > > > >> > > > > >> the
> > > > > >> > > > > >> >>> > > >> community
> > > > > >> > > > > >> >>> > > >> >> for
> > > > > >> > > > > >> >>> > > >> >> > > > every
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > release.
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > The modules are a
> > subset
> > > of
> > > > > >> > existing
> > > > > >> > > > > >> >>> integrations
> > > > > >> > > > > >> >>> > > >> that
> > > > > >> > > > > >> >>> > > >> >> are
> > > > > >> > > > > >> >>> > > >> >> > > > >>> adopted
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > widely
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > and require community
> > > > > >> attention.
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > As for the
> independent
> > > > > >> > integrations
> > > > > >> > > > [1],
> > > > > >> > > > > >> >>> those
> > > > > >> > > > > >> >>> > are
> > > > > >> > > > > >> >>> > > >> >> supposed
> > > > > >> > > > > >> >>> > > >> >> > > to
> > > > > >> > > > > >> >>> > > >> >> > > > be
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > located
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > in a separate GitHub
> > > repo.
> > > > I
> > > > > >> would
> > > > > >> > > > > advise
> > > > > >> > > > > >> us
> > > > > >> > > > > >> >>> to
> > > > > >> > > > > >> >>> > > start
> > > > > >> > > > > >> >>> > > >> >> > > > >>> implementing
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > this
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > concept with the
> > Pub/Sub
> > > > > >> > > integration.
> > > > > >> > > > > Even
> > > > > >> > > > > >> >>> though
> > > > > >> > > > > >> >>> > > it
> > > > > >> > > > > >> >>> > > >> >> will
> > > > > >> > > > > >> >>> > > >> >> > be
> > > > > >> > > > > >> >>> > > >> >> > > > >>> located
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > in a
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > separate Github repo,
> > it
> > > > will
> > > > > >> be
> > > > > >> > > > listed
> > > > > >> > > > > >> among
> > > > > >> > > > > >> >>> > > >> officially
> > > > > >> > > > > >> >>> > > >> >> > > > >>> available
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > integrations on the
> > > Ignite
> > > > > >> website
> > > > > >> > > > > >> featuring
> > > > > >> > > > > >> >>> > > >> Emmanouil
> > > > > >> > > > > >> >>> > > >> >> as a
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > contributor.
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > What do you think?
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > [1]
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > > >> > > > > >> >>> > > >> >> > > >
> > > > > >> > > > > >> >>> > > >> >> > >
> > > > > >> > > > > >> >>> > > >> >> >
> > > > > >> > > > > >> >>> > > >> >>
> > > > > >> > > > > >> >>> > > >>
> > > > > >> > > > > >> >>> > >
> > > > > >> > > > > >> >>> >
> > > > > >> > > > > >> >>>
> > > > > >> > > > > >>
> > > > > >> > > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > -
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > Denis
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > On Mon, Oct 14, 2019
> at
> > > > 9:47
> > > > > PM
> > > > > >> > > Saikat
> > > > > >> > > > > >> >>> Maitra <
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > saikat.maitra@gmail.com
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > wrote:
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > Hi Emmanouil,
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > The changes looks
> > good
> > > to
> > > > > >> me. I
> > > > > >> > > > wanted
> > > > > >> > > > > >> to
> > > > > >> > > > > >> >>> check
> > > > > >> > > > > >> >>> > > if
> > > > > >> > > > > >> >>> > > >> you
> > > > > >> > > > > >> >>> > > >> >> > are
> > > > > >> > > > > >> >>> > > >> >> > > > also
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > able
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > to
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > validate that the
> > once
> > > > data
> > > > > >> is
> > > > > >> > > added
> > > > > >> > > > > to
> > > > > >> > > > > >> >>> Ignite
> > > > > >> > > > > >> >>> > > >> cluster
> > > > > >> > > > > >> >>> > > >> >> > you
> > > > > >> > > > > >> >>> > > >> >> > > > are
> > > > > >> > > > > >> >>> > > >> >> > > > >>> also
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > able
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > to
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > access it using
> > another
> > > > > >> Ignite
> > > > > >> > > > client
> > > > > >> > > > > or
> > > > > >> > > > > >> >>> using
> > > > > >> > > > > >> >>> > > rest
> > > > > >> > > > > >> >>> > > >> >> > > > endpoints.
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > We use Teamcity for
> > CI
> > > > > >> process,
> > > > > >> > > the
> > > > > >> > > > > >> >>> details are
> > > > > >> > > > > >> >>> > > as
> > > > > >> > > > > >> >>> > > >> >> > follows
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > Trigger validation
> of
> > > > those
> > > > > >> test
> > > > > >> > > > > suites
> > > > > >> > > > > >> >>> that
> > > > > >> > > > > >> >>> > have
> > > > > >> > > > > >> >>> > > >> been
> > > > > >> > > > > >> >>> > > >> >> > > > >>> affected by
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > your
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > changes on
> TeamCity <
> > > > > >> > > > > >> >>> > > http://ci.ignite.apache.org/>:
> > > > > >> > > > > >> >>> > > >> <
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> http://204.14.53.153/
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Locate a test
> > > suite
> > > > > you
> > > > > >> > have
> > > > > >> > > to
> > > > > >> > > > > >> check,
> > > > > >> > > > > >> >>> > press
> > > > > >> > > > > >> >>> > > >> >> button
> > > > > >> > > > > >> >>> > > >> >> > > > named
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > "..."
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > that
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    is located on
> the
> > > left
> > > > > of
> > > > > >> > "Run"
> > > > > >> > > > > >> button.
> > > > > >> > > > > >> >>> "Run
> > > > > >> > > > > >> >>> > > >> custom
> > > > > >> > > > > >> >>> > > >> >> > > build"
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > dialog
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > will
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    appear;
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Go to
> "Changes"
> > > tab
> > > > > and
> > > > > >> > > choose
> > > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > "pull/<pull-request-number>/head"
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > in
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    "Build branch"
> > > > dropdown
> > > > > >> list;
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Press "Run
> > build"
> > > > > button
> > > > > >> > and
> > > > > >> > > > > >> monitor
> > > > > >> > > > > >> >>> tests
> > > > > >> > > > > >> >>> > > >> >> results.
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > @Roman
> > > rshtykh@yahoo.com
> > > > -
> > > > > >> can
> > > > > >> > > you
> > > > > >> > > > > also
> > > > > >> > > > > >> >>> please
> > > > > >> > > > > >> >>> > > >> take a
> > > > > >> > > > > >> >>> > > >> >> > > look?
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > Regards,
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > Saikat
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > On Sun, Oct 13,
> 2019
> > at
> > > > > 6:20
> > > > > >> PM
> > > > > >> > > > > >> Emmanouil
> > > > > >> > > > > >> >>> > > >> Gkatziouras
> > > > > >> > > > > >> >>> > > >> >> <
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> gkatzioura@gmail.com
> > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > wrote:
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Hi @Saikat Maitra
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > I have my
> > > > implementation
> > > > > >> on an
> > > > > >> > > > > Ignite
> > > > > >> > > > > >> >>> fork on
> > > > > >> > > > > >> >>> > > my
> > > > > >> > > > > >> >>> > > >> >> > personal
> > > > > >> > > > > >> >>> > > >> >> > > > >>> GitHub
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > account
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > [1]
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > It is based on
> the
> > > > Kafka
> > > > > >> > > > > >> implementation
> > > > > >> > > > > >> >>> which
> > > > > >> > > > > >> >>> > > is
> > > > > >> > > > > >> >>> > > >> of
> > > > > >> > > > > >> >>> > > >> >> a
> > > > > >> > > > > >> >>> > > >> >> > > > polling
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > nature.
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > GCP does not
> > provide
> > > a
> > > > > >> service
> > > > > >> > > > like
> > > > > >> > > > > >> >>> Kafka.
> > > > > >> > > > > >> >>> > > >> Instead
> > > > > >> > > > > >> >>> > > >> >> it
> > > > > >> > > > > >> >>> > > >> >> > > > >>> provides us
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > with
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Pub/Sub which is
> > just
> > > > > like
> > > > > >> > > Kafka.
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Therefore using
> the
> > > > Kafka
> > > > > >> > > streamer
> > > > > >> > > > > as
> > > > > >> > > > > >> a
> > > > > >> > > > > >> >>> > > guideline
> > > > > >> > > > > >> >>> > > >> >> made
> > > > > >> > > > > >> >>> > > >> >> > > more
> > > > > >> > > > > >> >>> > > >> >> > > > >>> sense
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > to
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > me.
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Apart from the
> > > > > >> implementation
> > > > > >> > I
> > > > > >> > > > have
> > > > > >> > > > > >> also
> > > > > >> > > > > >> >>> > > added a
> > > > > >> > > > > >> >>> > > >> >> unit
> > > > > >> > > > > >> >>> > > >> >> > > > test.
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > It definitely
> needs
> > > > some
> > > > > >> > > polishing
> > > > > >> > > > > >> and a
> > > > > >> > > > > >> >>> > double
> > > > > >> > > > > >> >>> > > >> >> check
> > > > > >> > > > > >> >>> > > >> >> > on
> > > > > >> > > > > >> >>> > > >> >> > > my
> > > > > >> > > > > >> >>> > > >> >> > > > >>> side
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > to
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > make
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > the review
> smooth.
> > > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > For the tests a
> > local
> > > > > >> Pub/Sub
> > > > > >> > > [2]
> > > > > >> > > > > >> server
> > > > > >> > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra <sa...@gmail.com>]

Hi Emmanouil,

I have been able to resolve the build failures for missing modules.

https://ci.ignite.apache.org/viewLog.html?buildId=4883394&buildTypeId=IgniteExtensions_Build

Will you please take the latest changes from my PR and run the build for
your PR.

https://github.com/apache/ignite-extensions/pull/1/files

The new build configurations should let your module build also pass.

The issue that was causing failure was that I had put ignite.zip as
artifact dependency and it was downloading the artifacts including pom file
inside the build checkout folder of ignite-extensions and as a result the
pom.xml of ignite-extensions was getting overwritten by ignite's pom.xml.
This was resulting in missing flink-ext module error as it was not even
present in the ignite main pom.xml.

To resolve the issue I just had to download and extract artifact of
ignite.zip in a separate directory.

Regards,
Saikat


On Sat, Nov 30, 2019 at 5:53 PM Emmanouil Gkatziouras <gk...@gmail.com>
wrote:

> Hi Saikat!
>
> Thank you for your assistance on that!
>
> Kind regards
>
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>
>
> On Sat, 30 Nov 2019 at 23:51, Saikat Maitra <sa...@gmail.com>
> wrote:
>
> > Hi Emmanouil,
> >
> > I looked into the build logs and I observed that since I added artifacts
> > dependencies to "Build apache Ignite"[1] it was able to pull the required
> > dependencies and was able to run the existing tests. It is however not
> > identifying new modules like I changed from flink to flink-ext or
> pub-sub.
> >
> > [1]
> >
> >
> https://ci.ignite.apache.org/admin/editDependencies.html?id=buildType:IgniteExtensions_Build
> >
> > I will look into the issue and debug further.
> >
> > Regards,
> > Saikat
> >
> >
> > On Sat, Nov 30, 2019 at 5:35 PM Emmanouil Gkatziouras <
> > gkatzioura@gmail.com>
> > wrote:
> >
> > > Hi Saikat!
> > >
> > > From the logs[1] it seems that TC cannot find the project. However I
> did
> > > include the project on the parent pom (modules sections) [2]
> > > I tried to reproduce locally unfortunately I am not aware of the full
> > > context on TC. It seems as if a different parent pom is being picked
> up.
> > >
> > > Kind regards
> > >
> > > [1]
> > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4804893&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=1704
> > > [2]
> > >
> > >
> >
> https://github.com/gkatzioura/ignite-extensions/blob/d6a8beee7feff28f59d9a12be692016305564d25/pom.xml#L46
> > >
> > > *Emmanouil Gkatziouras*
> > > https://egkatzioura.com/ |
> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > https://github.com/gkatzioura
> > >
> > >
> > > On Sat, 30 Nov 2019 at 22:20, Saikat Maitra <sa...@gmail.com>
> > > wrote:
> > >
> > > > Hello,
> > > >
> > > > I have made changes in Ignite Extensions build configurations.
> > > >
> > > > It is now running the tests as expected.
> > > >
> > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4804480&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> > > >
> > > > Regards,
> > > > Saikat
> > > >
> > > > On Thu, Nov 28, 2019 at 1:29 PM Saikat Maitra <
> saikat.maitra@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > We do run build with -DskipTests and then configure selective test
> > > suites
> > > > > to run.
> > > > >
> > > > > I will check why these test suites are not getting executed.
> > > > >
> > > > > Regards
> > > > > Saikat
> > > > >
> > > > > On Wed, 27 Nov 2019 at 3:05 PM, Emmanouil Gkatziouras <
> > > > > gkatzioura@gmail.com> wrote:
> > > > >
> > > > >> Hi all,
> > > > >>
> > > > >> I did add a test suite and run the build with the corresponding
> > > > arguments,
> > > > >> however the tests did not run [1].
> > > > >> I checked the Flink's build logs and the same message is displayed
> > > there
> > > > >> too `No tests to run` [2].
> > > > >>
> > > > >> [1]
> > > > >>
> > > > >>
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4798203&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=248
> > > > >> [2]
> > > > >>
> > > > >>
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4789199&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=1256
> > > > >>
> > > > >> Kind regards
> > > > >> *Emmanouil Gkatziouras*
> > > > >> https://egkatzioura.com/ |
> > > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > >> https://github.com/gkatzioura
> > > > >>
> > > > >>
> > > > >> On Wed, 27 Nov 2019 at 18:58, Saikat Maitra <
> > saikat.maitra@gmail.com>
> > > > >> wrote:
> > > > >>
> > > > >> > Hi,
> > > > >> >
> > > > >> > We will need to add test suites for tests to be executed in
> build.
> > > We
> > > > >> pass
> > > > >> > them as TestSuites param. I had added Flink sink and source
> > > > testsuites.
> > > > >> >
> > > > >> > Can you please review and confirm.
> > > > >> >
> > > > >> > Regards
> > > > >> > Saikat
> > > > >> >
> > > > >> > On Wed, 27 Nov 2019 at 6:00 AM, Emmanouil Gkatziouras <
> > > > >> > gkatzioura@gmail.com>
> > > > >> > wrote:
> > > > >> >
> > > > >> > > Hi all!
> > > > >> > >
> > > > >> > > I was successful building and running my pull requests. It
> seems
> > > > that
> > > > >> the
> > > > >> > > tests do not run both for Flink and Pub/Sub [1]
> > > > >> > > If there is something I cannot do to make them discoverable
> > please
> > > > >> let me
> > > > >> > > know.
> > > > >> > >
> > > > >> > > [1]
> > > > >> > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4796722&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=283
> > > > >> > >
> > > > >> > > *Emmanouil Gkatziouras*
> > > > >> > > https://egkatzioura.com/ |
> > > > >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > >> > > https://github.com/gkatzioura
> > > > >> > >
> > > > >> > >
> > > > >> > > On Wed, 27 Nov 2019 at 04:57, Saikat Maitra <
> > > > saikat.maitra@gmail.com>
> > > > >> > > wrote:
> > > > >> > >
> > > > >> > > > Hi Emmanouil,
> > > > >> > > >
> > > > >> > > > I have added you as contributor in Ignite Extensions
> project.
> > > Can
> > > > >> you
> > > > >> > > > please check and confirm if you are able to see the project
> > and
> > > > >> execute
> > > > >> > > > build on your pull request.
> > > > >> > > >
> > > > >> > > > Regards,
> > > > >> > > > Saikat
> > > > >> > > >
> > > > >> > > > On Tue, Nov 26, 2019 at 12:49 PM Emmanouil Gkatziouras <
> > > > >> > > > gkatzioura@gmail.com>
> > > > >> > > > wrote:
> > > > >> > > >
> > > > >> > > > > Hi all!
> > > > >> > > > >
> > > > >> > > > > If someone could give me read access to the Job of Ignite
> > > > >> Extensions
> > > > >> > on
> > > > >> > > > > Team City will greatly help me (username gkatzioura).
> > > > >> > > > > I suppose the builds get triggered automatically on a pull
> > > > >> request.
> > > > >> > > > >
> > > > >> > > > > Kind regards,
> > > > >> > > > > Emmanouil
> > > > >> > > > >
> > > > >> > > > > *Emmanouil Gkatziouras*
> > > > >> > > > > https://egkatzioura.com/ |
> > > > >> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > >> > > > > https://github.com/gkatzioura
> > > > >> > > > >
> > > > >> > > > >
> > > > >> > > > > On Sun, 24 Nov 2019 at 21:11, Emmanouil Gkatziouras <
> > > > >> > > > gkatzioura@gmail.com>
> > > > >> > > > > wrote:
> > > > >> > > > >
> > > > >> > > > > > Hi Saikat!
> > > > >> > > > > >
> > > > >> > > > > > I just rebased with the flink branch. Unfortunately I do
> > not
> > > > >> have
> > > > >> > > read
> > > > >> > > > > > access to the team city link you provided and thus
> > evaluate
> > > > the
> > > > >> > tests
> > > > >> > > > for
> > > > >> > > > > > my pull request.
> > > > >> > > > > > I guess it has to do with ignite extensions being new.
> > > > >> > > > > > My username is gkatzioura and email the one I am using
> on
> > > this
> > > > >> > email
> > > > >> > > > > >
> > > > >> > > > > > Thank you for your time.
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > > *Emmanouil Gkatziouras*
> > > > >> > > > > > https://egkatzioura.com/ |
> > > > >> > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > >> > > > > > https://github.com/gkatzioura
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > > On Sun, 24 Nov 2019 at 18:03, Saikat Maitra <
> > > > >> > saikat.maitra@gmail.com
> > > > >> > > >
> > > > >> > > > > > wrote:
> > > > >> > > > > >
> > > > >> > > > > >> Hi Emmanouil,
> > > > >> > > > > >>
> > > > >> > > > > >> The latest build on teamcity has passed on Flink pull
> > > > request.
> > > > >> > > > > >>
> > > > >> > > > > >>
> > > > >> > > > > >>
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4788928&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> > > > >> > > > > >>
> > > > >> > > > > >> you should be able to take the changes I made in my PR
> > and
> > > > run
> > > > >> > build
> > > > >> > > > on
> > > > >> > > > > >> your pull request.
> > > > >> > > > > >>
> > > > >> > > > > >> Please let me know if you have any questions.
> > > > >> > > > > >>
> > > > >> > > > > >> Regards,
> > > > >> > > > > >> Saikat
> > > > >> > > > > >>
> > > > >> > > > > >> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra <
> > > > >> > > > saikat.maitra@gmail.com
> > > > >> > > > > >
> > > > >> > > > > >> wrote:
> > > > >> > > > > >>
> > > > >> > > > > >> > Hi Emmanouil,
> > > > >> > > > > >> >
> > > > >> > > > > >> > I have fixed the Flink module testsuites and have
> setup
> > > new
> > > > >> > > project
> > > > >> > > > in
> > > > >> > > > > >> > teamcity.
> > > > >> > > > > >> >
> > > > >> > > > > >> >
> > > > >> > > > > >> >
> > > > >> > > > > >>
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview
> > > > >> > > > > >> >
> > > > >> > > > > >> > I am looking into setting up the build.
> > > > >> > > > > >> >
> > > > >> > > > > >> > Regards,
> > > > >> > > > > >> > Saikat
> > > > >> > > > > >> >
> > > > >> > > > > >> > On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra <
> > > > >> > > > > saikat.maitra@gmail.com>
> > > > >> > > > > >> > wrote:
> > > > >> > > > > >> >
> > > > >> > > > > >> >> Hi Emmanouil,
> > > > >> > > > > >> >>
> > > > >> > > > > >> >> Thank you for your email. Yes, the plan is once the
> > > Flink
> > > > >> PR is
> > > > >> > > > > merged
> > > > >> > > > > >> it
> > > > >> > > > > >> >> will provide the Licence, parent POMs etc and you
> can
> > > > rebase
> > > > >> > from
> > > > >> > > > > >> master
> > > > >> > > > > >> >> branch and apply the changes on top of it.
> > > > >> > > > > >> >>
> > > > >> > > > > >> >> I am getting some issues with test failures in local
> > > with
> > > > >> > > > > >> >> GridTestProperties as the test.properties is not
> > present
> > > > in
> > > > >> > this
> > > > >> > > > new
> > > > >> > > > > >> >> project but should be available from dependencies.
> > > > >> > > > > >> >>
> > > > >> > > > > >> >> Once I address this issue, I will go ahead and merge
> > the
> > > > >> > changes
> > > > >> > > > and
> > > > >> > > > > >> then
> > > > >> > > > > >> >> we can take it from there.
> > > > >> > > > > >> >>
> > > > >> > > > > >> >> Regards,
> > > > >> > > > > >> >> Saikat
> > > > >> > > > > >> >>
> > > > >> > > > > >> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil
> Gkatziouras
> > <
> > > > >> > > > > >> >> gkatzioura@gmail.com> wrote:
> > > > >> > > > > >> >>
> > > > >> > > > > >> >>> Hi all,
> > > > >> > > > > >> >>>
> > > > >> > > > > >> >>> I made my first pull request [1]. Since this
> project
> > is
> > > > >> brand
> > > > >> > > new
> > > > >> > > > > (no
> > > > >> > > > > >> >>> parent poms, licensing), I reckoned it was best to
> > use
> > > > >> > Saikat's
> > > > >> > > > > >> branch on
> > > > >> > > > > >> >>> Flink.
> > > > >> > > > > >> >>> I suppose the Flink branch will be merged first. If
> > not
> > > > >> please
> > > > >> > > > give
> > > > >> > > > > me
> > > > >> > > > > >> >>> guidelines on how I should proceed next.
> > > > >> > > > > >> >>>
> > > > >> > > > > >> >>> Kind regards
> > > > >> > > > > >> >>> Emmanouil
> > > > >> > > > > >> >>>
> > > > >> > > > > >> >>> [1]
> > https://github.com/apache/ignite-extensions/pull/2
> > > > >> > > > > >> >>>
> > > > >> > > > > >> >>> *Emmanouil Gkatziouras*
> > > > >> > > > > >> >>> https://egkatzioura.com/ |
> > > > >> > > > > >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > >> > > > > >> >>> https://github.com/gkatzioura
> > > > >> > > > > >> >>>
> > > > >> > > > > >> >>>
> > > > >> > > > > >> >>> On Fri, 22 Nov 2019 at 20:55, Denis Magda <
> > > > >> dmagda@apache.org>
> > > > >> > > > > wrote:
> > > > >> > > > > >> >>>
> > > > >> > > > > >> >>> > Awesome, ping us whenever you're ready!
> > > > >> > > > > >> >>> >
> > > > >> > > > > >> >>> > -
> > > > >> > > > > >> >>> > Denis
> > > > >> > > > > >> >>> >
> > > > >> > > > > >> >>> >
> > > > >> > > > > >> >>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil
> > > Gkatziouras
> > > > <
> > > > >> > > > > >> >>> > gkatzioura@gmail.com>
> > > > >> > > > > >> >>> > wrote:
> > > > >> > > > > >> >>> >
> > > > >> > > > > >> >>> > > Hi all!
> > > > >> > > > > >> >>> > >
> > > > >> > > > > >> >>> > > I am sorry for being late on that. I was trying
> > to
> > > > >> > refactor
> > > > >> > > > the
> > > > >> > > > > >> test
> > > > >> > > > > >> >>> in
> > > > >> > > > > >> >>> > > order not to be in need of any external tools
> or
> > > > >> spinning
> > > > >> > > up a
> > > > >> > > > > >> >>> server.
> > > > >> > > > > >> >>> > > I did forked the new repo and indeed my changes
> > > > there,
> > > > >> so
> > > > >> > a
> > > > >> > > > pull
> > > > >> > > > > >> >>> request
> > > > >> > > > > >> >>> > is
> > > > >> > > > > >> >>> > > a matter of time!
> > > > >> > > > > >> >>> > >
> > > > >> > > > > >> >>> > > Kind regards
> > > > >> > > > > >> >>> > >
> > > > >> > > > > >> >>> > > *Emmanouil Gkatziouras*
> > > > >> > > > > >> >>> > > https://egkatzioura.com/ |
> > > > >> > > > > >> >>> > >
> > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > >> > > > > >> >>> > > https://github.com/gkatzioura
> > > > >> > > > > >> >>> > >
> > > > >> > > > > >> >>> > >
> > > > >> > > > > >> >>> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda <
> > > > >> > > dmagda@apache.org>
> > > > >> > > > > >> wrote:
> > > > >> > > > > >> >>> > >
> > > > >> > > > > >> >>> > > > Hi Emmanouil,
> > > > >> > > > > >> >>> > > >
> > > > >> > > > > >> >>> > > > Do you have any questions or need any support
> > > from
> > > > >> the
> > > > >> > > > > >> community?
> > > > >> > > > > >> >>> > > >
> > > > >> > > > > >> >>> > > > -
> > > > >> > > > > >> >>> > > > Denis
> > > > >> > > > > >> >>> > > >
> > > > >> > > > > >> >>> > > >
> > > > >> > > > > >> >>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat
> Maitra <
> > > > >> > > > > >> >>> saikat.maitra@gmail.com
> > > > >> > > > > >> >>> > >
> > > > >> > > > > >> >>> > > > wrote:
> > > > >> > > > > >> >>> > > >
> > > > >> > > > > >> >>> > > >> Hi Emmanouil,
> > > > >> > > > > >> >>> > > >>
> > > > >> > > > > >> >>> > > >> Can you please take a looks at dev utils, if
> > > this
> > > > is
> > > > >> > > > > something
> > > > >> > > > > >> >>> you are
> > > > >> > > > > >> >>> > > >> looking for
> > > > >> > > > > >> >>> > > >>
> > > > >> > > > >
> > > https://github.com/apache/ignite/tree/master/modules/dev-utils
> > > > >> > > > > >> ?
> > > > >> > > > > >> >>> > > >>
> > > > >> > > > > >> >>> > > >> IMO, if you can release Pub/Sub server in
> > maven
> > > > and
> > > > >> > then
> > > > >> > > > use
> > > > >> > > > > >> it as
> > > > >> > > > > >> >>> > > >> dependency, that would be great.
> > > > >> > > > > >> >>> > > >>
> > > > >> > > > > >> >>> > > >> Regards,
> > > > >> > > > > >> >>> > > >> Saikat
> > > > >> > > > > >> >>> > > >>
> > > > >> > > > > >> >>> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat
> Maitra
> > <
> > > > >> > > > > >> >>> > saikat.maitra@gmail.com>
> > > > >> > > > > >> >>> > > >> wrote:
> > > > >> > > > > >> >>> > > >>
> > > > >> > > > > >> >>> > > >> > Hi Emmanouil,
> > > > >> > > > > >> >>> > > >> >
> > > > >> > > > > >> >>> > > >> > The master branch has been initialized. I
> > have
> > > > >> > raised a
> > > > >> > > > PR
> > > > >> > > > > to
> > > > >> > > > > >> >>> > migrate
> > > > >> > > > > >> >>> > > >> > Flink module to ignite-extensions.
> > > > >> > > > > >> >>> > > >> >
> > > > >> > > > > >> >>> > > >> > PR
> > > > >> > https://github.com/apache/ignite-extensions/pull/1
> > > > >> > > > > >> >>> > > >> > Jira :
> > > > >> > > > https://issues.apache.org/jira/browse/IGNITE-12356
> > > > >> > > > > >> >>> > > >> >
> > > > >> > > > > >> >>> > > >> > Regards,
> > > > >> > > > > >> >>> > > >> > Saikat
> > > > >> > > > > >> >>> > > >> >
> > > > >> > > > > >> >>> > > >> >
> > > > >> > > > > >> >>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil
> > > > >> > Gkatziouras <
> > > > >> > > > > >> >>> > > >> > gkatzioura@gmail.com> wrote:
> > > > >> > > > > >> >>> > > >> >
> > > > >> > > > > >> >>> > > >> >> Hi all!
> > > > >> > > > > >> >>> > > >> >>
> > > > >> > > > > >> >>> > > >> >> This is an update. The integration is
> > tested
> > > > with
> > > > >> > > mocks
> > > > >> > > > > >> based
> > > > >> > > > > >> >>> on
> > > > >> > > > > >> >>> > the
> > > > >> > > > > >> >>> > > >> >> original Ignite repo [1].
> > > > >> > > > > >> >>> > > >> >> I am currently waiting for the first
> > commits
> > > on
> > > > >> the
> > > > >> > > new
> > > > >> > > > > >> >>> integration
> > > > >> > > > > >> >>> > > >> >> project
> > > > >> > > > > >> >>> > > >> >> to take place in order to create my pull
> > > > request
> > > > >> > > > > accordingly
> > > > >> > > > > >> >>> to the
> > > > >> > > > > >> >>> > > >> >> initial
> > > > >> > > > > >> >>> > > >> >> commits.
> > > > >> > > > > >> >>> > > >> >> In order to simulate the Pub/Sub server
> in
> > a
> > > > >> test I
> > > > >> > > > used a
> > > > >> > > > > >> test
> > > > >> > > > > >> >>> > > server
> > > > >> > > > > >> >>> > > >> >> implementation [2]. Unfortunately they
> > don't
> > > > have
> > > > >> > this
> > > > >> > > > > util
> > > > >> > > > > >> >>> > uploaded
> > > > >> > > > > >> >>> > > >> in a
> > > > >> > > > > >> >>> > > >> >> maven repo, thus it needs to be hosted
> on a
> > > > maven
> > > > >> > repo
> > > > >> > > > > >> (Apache
> > > > >> > > > > >> >>> > > License
> > > > >> > > > > >> >>> > > >> >> 2.0). Is there a repo for utilities that
> > > Ignite
> > > > >> uses
> > > > >> > > for
> > > > >> > > > > the
> > > > >> > > > > >> >>> > > >> test/build?
> > > > >> > > > > >> >>> > > >> >> I have also created a ticket [3]
> > > > >> > > > > >> >>> > > >> >>
> > > > >> > > > > >> >>> > > >> >> Kind regards
> > > > >> > > > > >> >>> > > >> >>
> > > > >> > > > > >> >>> > > >> >> [1]
> > > > >> > > > > >> >>> >
> > > > >> > >
> https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > > > >> > > > > >> >>> > > >> >> [2]
> > > > >> > > > > >> >>>
> > > > >> https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> > > > >> > > > > >> >>> > > >> >> [3]
> > > > >> > > https://issues.apache.org/jira/browse/IGNITE-12262
> > > > >> > > > > >> >>> > > >> >>
> > > > >> > > > > >> >>> > > >> >> *Emmanouil Gkatziouras*
> > > > >> > > > > >> >>> > > >> >> https://egkatzioura.com/ |
> > > > >> > > > > >> >>> > > >> >>
> > > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > >> > > > > >> >>> > > >> >> https://github.com/gkatzioura
> > > > >> > > > > >> >>> > > >> >>
> > > > >> > > > > >> >>> > > >> >>
> > > > >> > > > > >> >>> > > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat
> > Maitra <
> > > > >> > > > > >> >>> > saikat.maitra@gmail.com
> > > > >> > > > > >> >>> > > >
> > > > >> > > > > >> >>> > > >> >> wrote:
> > > > >> > > > > >> >>> > > >> >>
> > > > >> > > > > >> >>> > > >> >> > Hello Ilya,
> > > > >> > > > > >> >>> > > >> >> >
> > > > >> > > > > >> >>> > > >> >> > Thank you for your email. Yes, before
> we
> > > > remove
> > > > >> > any
> > > > >> > > > > >> >>> integration
> > > > >> > > > > >> >>> > we
> > > > >> > > > > >> >>> > > >> will
> > > > >> > > > > >> >>> > > >> >> > move them to another repository and
> > > release.
> > > > >> > > > > >> >>> > > >> >> >
> > > > >> > > > > >> >>> > > >> >> > We are thinking we will start with
> > Pub/Sub
> > > > >> > Streamer
> > > > >> > > > > >> >>> > implementation
> > > > >> > > > > >> >>> > > >> and
> > > > >> > > > > >> >>> > > >> >> > continue to move other integration as
> > > > >> mentioned in
> > > > >> > > the
> > > > >> > > > > >> >>> following
> > > > >> > > > > >> >>> > > doc.
> > > > >> > > > > >> >>> > > >> >> >
> > > > >> > > > > >> >>> > > >> >> >
> > > > >> > > > > >> >>> > > >> >> >
> > > > >> > > > > >> >>> > > >> >>
> > > > >> > > > > >> >>> > > >>
> > > > >> > > > > >> >>> > >
> > > > >> > > > > >> >>> >
> > > > >> > > > > >> >>>
> > > > >> > > > > >>
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > >> > > > > >> >>> > > >> >> >
> > > > >> > > > > >> >>> > > >> >> > Regards,
> > > > >> > > > > >> >>> > > >> >> > Saikat
> > > > >> > > > > >> >>> > > >> >> >
> > > > >> > > > > >> >>> > > >> >> >
> > > > >> > > > > >> >>> > > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya
> > > > >> Kasnacheev <
> > > > >> > > > > >> >>> > > >> >> > ilya.kasnacheev@gmail.com>
> > > > >> > > > > >> >>> > > >> >> > wrote:
> > > > >> > > > > >> >>> > > >> >> >
> > > > >> > > > > >> >>> > > >> >> > > Hello!
> > > > >> > > > > >> >>> > > >> >> > >
> > > > >> > > > > >> >>> > > >> >> > > My take on this: we want to offload
> > some
> > > of
> > > > >> > > > > >> integrations,
> > > > >> > > > > >> >>> but
> > > > >> > > > > >> >>> > > >> please
> > > > >> > > > > >> >>> > > >> >> > commit
> > > > >> > > > > >> >>> > > >> >> > > and release them to another project
> > > first,
> > > > >> and
> > > > >> > > only
> > > > >> > > > > >> then we
> > > > >> > > > > >> >>> > will
> > > > >> > > > > >> >>> > > >> >> decide
> > > > >> > > > > >> >>> > > >> >> > to
> > > > >> > > > > >> >>> > > >> >> > > remove them from our repository.
> > > > >> > > > > >> >>> > > >> >> > >
> > > > >> > > > > >> >>> > > >> >> > > Please also make sure they are
> > available
> > > > with
> > > > >> > same
> > > > >> > > > > >> artifact
> > > > >> > > > > >> >>> > name
> > > > >> > > > > >> >>> > > >> and
> > > > >> > > > > >> >>> > > >> >> API.
> > > > >> > > > > >> >>> > > >> >> > > If this has to be changed, let's wait
> > for
> > > > AI
> > > > >> 3.0
> > > > >> > > > > >> >>> > > >> >> > >
> > > > >> > > > > >> >>> > > >> >> > > Regards,
> > > > >> > > > > >> >>> > > >> >> > > --
> > > > >> > > > > >> >>> > > >> >> > > Ilya Kasnacheev
> > > > >> > > > > >> >>> > > >> >> > >
> > > > >> > > > > >> >>> > > >> >> > >
> > > > >> > > > > >> >>> > > >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat
> > > Maitra
> > > > <
> > > > >> > > > > >> >>> > > >> saikat.maitra@gmail.com>:
> > > > >> > > > > >> >>> > > >> >> > >
> > > > >> > > > > >> >>> > > >> >> > > > Hello Emmanouil,
> > > > >> > > > > >> >>> > > >> >> > > >
> > > > >> > > > > >> >>> > > >> >> > > > As discussed earlier, I discussed
> > with
> > > > >> Apache
> > > > >> > > > Bahir
> > > > >> > > > > >> >>> community
> > > > >> > > > > >> >>> > > and
> > > > >> > > > > >> >>> > > >> >> they
> > > > >> > > > > >> >>> > > >> >> > > are
> > > > >> > > > > >> >>> > > >> >> > > > interested to have Apache Ignite
> > > > >> extensions as
> > > > >> > > > part
> > > > >> > > > > of
> > > > >> > > > > >> >>> Apache
> > > > >> > > > > >> >>> > > >> Bahir
> > > > >> > > > > >> >>> > > >> >> > > > project.
> > > > >> > > > > >> >>> > > >> >> > > >
> > > > >> > > > > >> >>> > > >> >> > > > Can you please share Pub/Sub
> streamer
> > > > >> > > > implementation
> > > > >> > > > > >> >>> details
> > > > >> > > > > >> >>> > > with
> > > > >> > > > > >> >>> > > >> >> > Apache
> > > > >> > > > > >> >>> > > >> >> > > > Bahir community and request for
> > > feedback.
> > > > >> > > > > >> >>> > > >> >> > > >
> > > > >> > > > > >> >>> > > >> >> > > > https://bahir.apache.org/
> > > > >> > > > > >> >>> > > >> >> > > >
> > > > >> > > > > >> >>> > > >> >> > > > I have also requested for
> contributor
> > > > >> access
> > > > >> > in
> > > > >> > > > Jira
> > > > >> > > > > >> for
> > > > >> > > > > >> >>> > Apache
> > > > >> > > > > >> >>> > > >> >> Bahir
> > > > >> > > > > >> >>> > > >> >> > > > project so that I can create issues
> > and
> > > > >> assign
> > > > >> > > to
> > > > >> > > > > >> myself.
> > > > >> > > > > >> >>> > > >> >> > > >
> > > > >> > > > > >> >>> > > >> >> > > > I can help with code reviews as
> well.
> > > > >> > > > > >> >>> > > >> >> > > >
> > > > >> > > > > >> >>> > > >> >> > > > Here is the email thread for
> > reference
> > > > >> > > > > >> >>> > > >> >> > > >
> > > > >> > > > > >> >>> > >
> > > > >> > > >
> > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > > > >> > > > > >> >>> > > >> >> > > >
> > > > >> > > > > >> >>> > > >> >> > > > Regards,
> > > > >> > > > > >> >>> > > >> >> > > > Saikat
> > > > >> > > > > >> >>> > > >> >> > > >
> > > > >> > > > > >> >>> > > >> >> > > >
> > > > >> > > > > >> >>> > > >> >> > > >
> > > > >> > > > > >> >>> > > >> >> > > >
> > > > >> > > > > >> >>> > > >> >> > > >
> > > > >> > > > > >> >>> > > >> >> > > >
> > > > >> > > > > >> >>> > > >> >> > > >
> > > > >> > > > > >> >>> > > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM
> > Saikat
> > > > >> Maitra
> > > > >> > <
> > > > >> > > > > >> >>> > > >> >> saikat.maitra@gmail.com
> > > > >> > > > > >> >>> > > >> >> > >
> > > > >> > > > > >> >>> > > >> >> > > > wrote:
> > > > >> > > > > >> >>> > > >> >> > > >
> > > > >> > > > > >> >>> > > >> >> > > > > Hello,
> > > > >> > > > > >> >>> > > >> >> > > > >
> > > > >> > > > > >> >>> > > >> >> > > > > As discussed I have created
> > following
> > > > >> > > discussion
> > > > >> > > > > >> >>> threads on
> > > > >> > > > > >> >>> > > our
> > > > >> > > > > >> >>> > > >> >> > > migration
> > > > >> > > > > >> >>> > > >> >> > > > > proposals for Apache Ignite
> > > extensions:
> > > > >> > > > > >> >>> > > >> >> > > > >
> > > > >> > > > > >> >>> > > >> >> > > > >
> > > > >> > > > > >> >>> > > >> >> > > > >
> > > > >> > > > > >> >>> > > >> >> > > >
> > > > >> > > > > >> >>> > > >> >> > >
> > > > >> > > > > >> >>> > > >> >> >
> > > > >> > > > > >> >>> > > >> >>
> > > > >> > > > > >> >>> > > >>
> > > > >> > > > > >> >>> > >
> > > > >> > > > > >> >>> >
> > > > >> > > > > >> >>>
> > > > >> > > > > >>
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> > > > >> > > > > >> >>> > > >> >> > > > >
> > > > >> > > > > >> >>> > > >> >> > > > >
> > > > >> > > > > >> >>> > > >>
> > > > >> > > > > >>
> > > > >> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > > > >> > > > > >> >>> > > >> >> > > > >
> > > > >> > > > > >> >>> > > >> >> > > > > I will share update as I hear
> more
> > > > >> > > information.
> > > > >> > > > > >> >>> > > >> >> > > > >
> > > > >> > > > > >> >>> > > >> >> > > > > Regards,
> > > > >> > > > > >> >>> > > >> >> > > > > Saikat
> > > > >> > > > > >> >>> > > >> >> > > > >
> > > > >> > > > > >> >>> > > >> >> > > > >
> > > > >> > > > > >> >>> > > >> >> > > > >
> > > > >> > > > > >> >>> > > >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM
> > > Saikat
> > > > >> > Maitra
> > > > >> > > <
> > > > >> > > > > >> >>> > > >> >> > saikat.maitra@gmail.com
> > > > >> > > > > >> >>> > > >> >> > > >
> > > > >> > > > > >> >>> > > >> >> > > > > wrote:
> > > > >> > > > > >> >>> > > >> >> > > > >
> > > > >> > > > > >> >>> > > >> >> > > > >> Hello Denis,
> > > > >> > > > > >> >>> > > >> >> > > > >>
> > > > >> > > > > >> >>> > > >> >> > > > >> Sure, sounds good. I will
> create a
> > > > >> separate
> > > > >> > > > > >> discussion
> > > > >> > > > > >> >>> > > thread
> > > > >> > > > > >> >>> > > >> to
> > > > >> > > > > >> >>> > > >> >> get
> > > > >> > > > > >> >>> > > >> >> > > > >> community feedback on whether we
> > > > >> > > > > >> >>> > > >> >> > > > >> should join the Bahir or
> kick-off
> > > > >> "Ignite
> > > > >> > > > > >> Extensions"
> > > > >> > > > > >> >>> > > project.
> > > > >> > > > > >> >>> > > >> >> > > > >>
> > > > >> > > > > >> >>> > > >> >> > > > >> Regards,
> > > > >> > > > > >> >>> > > >> >> > > > >> Saikat
> > > > >> > > > > >> >>> > > >> >> > > > >>
> > > > >> > > > > >> >>> > > >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM
> > > Denis
> > > > >> > Magda <
> > > > >> > > > > >> >>> > > >> dmagda@apache.org>
> > > > >> > > > > >> >>> > > >> >> > > wrote:
> > > > >> > > > > >> >>> > > >> >> > > > >>
> > > > >> > > > > >> >>> > > >> >> > > > >>> Folks,
> > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > >> > > > > >> >>> > > >> >> > > > >>> The concept of Apache Bahir is
> > > > >> precisely
> > > > >> > > what
> > > > >> > > > we
> > > > >> > > > > >> >>> need!
> > > > >> > > > > >> >>> > > >> Saikat,
> > > > >> > > > > >> >>> > > >> >> > thanks
> > > > >> > > > > >> >>> > > >> >> > > > for
> > > > >> > > > > >> >>> > > >> >> > > > >>> doing the research. Why I
> believe
> > > > the *
> > > > >> > > idea*
> > > > >> > > > > >> fits us
> > > > >> > > > > >> >>> > well:
> > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > >> > > > > >> >>> > > >> >> > > > >>>    - All integrations can be
> > stored
> > > > in
> > > > >> > > > separate
> > > > >> > > > > >> >>> Github
> > > > >> > > > > >> >>> > > >> >> repositories
> > > > >> > > > > >> >>> > > >> >> > > and
> > > > >> > > > > >> >>> > > >> >> > > > >>>    have their dev lifecycles.
> > We've
> > > > not
> > > > >> > > > obliged
> > > > >> > > > > to
> > > > >> > > > > >> >>> couple
> > > > >> > > > > >> >>> > > all
> > > > >> > > > > >> >>> > > >> >> the
> > > > >> > > > > >> >>> > > >> >> > > > >>> integrations
> > > > >> > > > > >> >>> > > >> >> > > > >>>    in a single repo. For
> > instance,
> > > > >> Spark
> > > > >> > can
> > > > >> > > > be
> > > > >> > > > > >> >>> located
> > > > >> > > > > >> >>> > in
> > > > >> > > > > >> >>> > > a
> > > > >> > > > > >> >>> > > >> >> > > dedicated
> > > > >> > > > > >> >>> > > >> >> > > > >>> repo
> > > > >> > > > > >> >>> > > >> >> > > > >>>    while streaming integrations
> > > might
> > > > >> be
> > > > >> > in
> > > > >> > > a
> > > > >> > > > > >> single
> > > > >> > > > > >> >>> one.
> > > > >> > > > > >> >>> > > >> It's
> > > > >> > > > > >> >>> > > >> >> up
> > > > >> > > > > >> >>> > > >> >> > to
> > > > >> > > > > >> >>> > > >> >> > > > us.
> > > > >> > > > > >> >>> > > >> >> > > > >>>    - All the repositories and
> > > > >> integrations
> > > > >> > > > > belong
> > > > >> > > > > >> to
> > > > >> > > > > >> >>> ASF.
> > > > >> > > > > >> >>> > > >> We're
> > > > >> > > > > >> >>> > > >> >> not
> > > > >> > > > > >> >>> > > >> >> > > > >>> dumping
> > > > >> > > > > >> >>> > > >> >> > > > >>>    them on Github but rather
> keep
> > > > >> > supporting
> > > > >> > > > and
> > > > >> > > > > >> >>> > developing
> > > > >> > > > > >> >>> > > >> in
> > > > >> > > > > >> >>> > > >> >> > > > >>> accordance with
> > > > >> > > > > >> >>> > > >> >> > > > >>>    ASF vision and practices.
> > > > >> > > > > >> >>> > > >> >> > > > >>>    - There is a way to reward
> > > > >> contributors
> > > > >> > > via
> > > > >> > > > > >> >>> > > committership
> > > > >> > > > > >> >>> > > >> and
> > > > >> > > > > >> >>> > > >> >> > PMC
> > > > >> > > > > >> >>> > > >> >> > > > >>>    membership. You won't get
> this
> > > if
> > > > a
> > > > >> > > project
> > > > >> > > > > is
> > > > >> > > > > >> >>> just
> > > > >> > > > > >> >>> > one
> > > > >> > > > > >> >>> > > of
> > > > >> > > > > >> >>> > > >> >> the
> > > > >> > > > > >> >>> > > >> >> > > > >>> millions of
> > > > >> > > > > >> >>> > > >> >> > > > >>>    Github projects.
> > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > >> > > > > >> >>> > > >> >> > > > >>> Saikat, as Ignite PMC member,
> > could
> > > > you
> > > > >> > > please
> > > > >> > > > > >> >>> kick-off a
> > > > >> > > > > >> >>> > > >> >> separate
> > > > >> > > > > >> >>> > > >> >> > > > >>> dev-list
> > > > >> > > > > >> >>> > > >> >> > > > >>> discussion to involve more
> > > community
> > > > >> > members
> > > > >> > > > and
> > > > >> > > > > >> >>> > referring
> > > > >> > > > > >> >>> > > to
> > > > >> > > > > >> >>> > > >> >> this
> > > > >> > > > > >> >>> > > >> >> > > > >>> thread?
> > > > >> > > > > >> >>> > > >> >> > > > >>> I think the primary question
> the
> > > > >> community
> > > > >> > > > needs
> > > > >> > > > > >> to
> > > > >> > > > > >> >>> > answer
> > > > >> > > > > >> >>> > > >> >> whether
> > > > >> > > > > >> >>> > > >> >> > we
> > > > >> > > > > >> >>> > > >> >> > > > >>> should join the Bahir or
> kick-off
> > > > >> "Ignite
> > > > >> > > > > >> Extensions"
> > > > >> > > > > >> >>> > > >> project?
> > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > >> > > > > >> >>> > > >> >> > > > >>> -
> > > > >> > > > > >> >>> > > >> >> > > > >>> Denis
> > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > >> > > > > >> >>> > > >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM
> > > > Alexey
> > > > >> > > > Zinoviev
> > > > >> > > > > <
> > > > >> > > > > >> >>> > > >> >> > > > zaleslaw.sin@gmail.com>
> > > > >> > > > > >> >>> > > >> >> > > > >>> wrote:
> > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > >> > > > > >> >>> > > >> >> > > > >>> > Maybe we could move all our
> > > > Streaming
> > > > >> > > > > >> Integrations
> > > > >> > > > > >> >>> > there,
> > > > >> > > > > >> >>> > > >> but
> > > > >> > > > > >> >>> > > >> >> > what
> > > > >> > > > > >> >>> > > >> >> > > is
> > > > >> > > > > >> >>> > > >> >> > > > >>> about
> > > > >> > > > > >> >>> > > >> >> > > > >>> > maintaining and committer
> > > > >> permissions to
> > > > >> > > the
> > > > >> > > > > new
> > > > >> > > > > >> >>> > > >> repositories?
> > > > >> > > > > >> >>> > > >> >> > > > >>> > I see the list of the
> > committers
> > > > and
> > > > >> PMC
> > > > >> > > > > members
> > > > >> > > > > >> >>> there
> > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > >> > > https://bahir.apache.org/community-members/
> > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > Could somebody from Ignite
> > > > community
> > > > >> be
> > > > >> > > > added
> > > > >> > > > > to
> > > > >> > > > > >> >>> this
> > > > >> > > > > >> >>> > > list
> > > > >> > > > > >> >>> > > >> as
> > > > >> > > > > >> >>> > > >> >> a
> > > > >> > > > > >> >>> > > >> >> > > > >>> > PMC/committer to maintain
> > Ignite
> > > > >> > > > integrations?
> > > > >> > > > > >> >>> > > >> >> > > > >>> > If yes, I'd happy to join
> this
> > > > >> project
> > > > >> > and
> > > > >> > > > > >> >>> collaborate
> > > > >> > > > > >> >>> > > with
> > > > >> > > > > >> >>> > > >> >> these
> > > > >> > > > > >> >>> > > >> >> > > > guys
> > > > >> > > > > >> >>> > > >> >> > > > >>> > together
> > > > >> > > > > >> >>> > > >> >> > > > >>> > If no, and we should start
> from
> > > the
> > > > >> zero
> > > > >> > > > level
> > > > >> > > > > >> with
> > > > >> > > > > >> >>> > > >> external
> > > > >> > > > > >> >>> > > >> >> PRs
> > > > >> > > > > >> >>> > > >> >> > -
> > > > >> > > > > >> >>> > > >> >> > > > >>> hmmm,
> > > > >> > > > > >> >>> > > >> >> > > > >>> > it's better to have our own
> > > > external
> > > > >> > > > > repository
> > > > >> > > > > >> >>> with
> > > > >> > > > > >> >>> > > >> >> ApacheBahirr
> > > > >> > > > > >> >>> > > >> >> > > > >>> ideology.
> > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > Also, I agree, that all
> > > connectors
> > > > >> could
> > > > >> > > be
> > > > >> > > > > >> moved
> > > > >> > > > > >> >>> there
> > > > >> > > > > >> >>> > > (in
> > > > >> > > > > >> >>> > > >> >> > > > >>> ApacheBahirr
> > > > >> > > > > >> >>> > > >> >> > > > >>> > like repository), but not all
> > > > modules
> > > > >> > from
> > > > >> > > > the
> > > > >> > > > > >> page
> > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > >> > > > > >> >>> > > >> >> > > >
> > > > >> > > > > >> >>> > > >> >> > >
> > > > >> > > > > >> >>> > > >> >> >
> > > > >> > > > > >> >>> > > >> >>
> > > > >> > > > > >> >>> > > >>
> > > > >> > > > > >> >>> > >
> > > > >> > > > > >> >>> >
> > > > >> > > > > >> >>>
> > > > >> > > > > >>
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > >> > > > > >> >>> > > >> >> > > > >>> > because
> > > > >> > > > > >> >>> > > >> >> > > > >>> > they use different parts of
> > core
> > > > >> modules
> > > > >> > > and
> > > > >> > > > > >> could
> > > > >> > > > > >> >>> be
> > > > >> > > > > >> >>> > > >> >> developed
> > > > >> > > > > >> >>> > > >> >> > > with
> > > > >> > > > > >> >>> > > >> >> > > > >>> > different velocity.
> > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > In reality, before creation
> of
> > > new
> > > > >> > > > > repositories
> > > > >> > > > > >> we
> > > > >> > > > > >> >>> need
> > > > >> > > > > >> >>> > > >> wide
> > > > >> > > > > >> >>> > > >> >> > > > discussion
> > > > >> > > > > >> >>> > > >> >> > > > >>> > based on the document
> mentioned
> > > > >> above.
> > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > P.S. Of course, we could
> > > experiment
> > > > >> in
> > > > >> > the
> > > > >> > > > > next
> > > > >> > > > > >> >>> release
> > > > >> > > > > >> >>> > > 2.8
> > > > >> > > > > >> >>> > > >> >> with
> > > > >> > > > > >> >>> > > >> >> > > one
> > > > >> > > > > >> >>> > > >> >> > > > >>> or two
> > > > >> > > > > >> >>> > > >> >> > > > >>> > integrations like twitter or
> > > ZeroMQ
> > > > >> > > > > >> >>> > > >> >> > > > >>> > Also, I'd like Denis Magda
> idea
> > > for
> > > > >> the
> > > > >> > > > > separate
> > > > >> > > > > >> >>> > > >> repositories
> > > > >> > > > > >> >>> > > >> >> in
> > > > >> > > > > >> >>> > > >> >> > > > Apache
> > > > >> > > > > >> >>> > > >> >> > > > >>> > Github to maintain them by
> > > anybody.
> > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02,
> > > Saikat
> > > > >> > > Maitra <
> > > > >> > > > > >> >>> > > >> >> > > saikat.maitra@gmail.com
> > > > >> > > > > >> >>> > > >> >> > > > >:
> > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > Hi Denis, Emmanouil
> > > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > Thank you for your email. I
> > > think
> > > > >> > > > creating a
> > > > >> > > > > >> >>> separate
> > > > >> > > > > >> >>> > > >> >> > integration
> > > > >> > > > > >> >>> > > >> >> > > > >>> project
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > in github and also
> proposing
> > it
> > > > as
> > > > >> an
> > > > >> > > > apache
> > > > >> > > > > >> >>> > incubator
> > > > >> > > > > >> >>> > > >> >> project
> > > > >> > > > > >> >>> > > >> >> > > will
> > > > >> > > > > >> >>> > > >> >> > > > >>> be a
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > good move. The other
> separate
> > > > >> > > integration
> > > > >> > > > > >> modules
> > > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > >> > > > > >> >>> > > >> >> > > >
> > > > >> > > > > >> >>> > > >> >> > >
> > > > >> > > > > >> >>> > > >> >> >
> > > > >> > > > > >> >>> > > >> >>
> > > > >> > > > > >> >>> > > >>
> > > > >> > > > > >> >>> > >
> > > > >> > > > > >> >>> >
> > > > >> > > > > >> >>>
> > > > >> > > > > >>
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > can
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > be moved to this new apache
> > > > >> incubator
> > > > >> > > > > project.
> > > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > There are similar
> integration
> > > > >> > available
> > > > >> > > > for
> > > > >> > > > > >> >>> Flink and
> > > > >> > > > > >> >>> > > >> Spark
> > > > >> > > > > >> >>> > > >> >> in
> > > > >> > > > > >> >>> > > >> >> > > > Apache
> > > > >> > > > > >> >>> > > >> >> > > > >>> > Bahir
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > https://bahir.apache.org/
> > > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > Do you think we should
> reach
> > > out
> > > > to
> > > > >> > > Apache
> > > > >> > > > > >> Bahir
> > > > >> > > > > >> >>> > > >> community
> > > > >> > > > > >> >>> > > >> >> > with a
> > > > >> > > > > >> >>> > > >> >> > > > >>> > proposal
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > or we should plan to
> create a
> > > > >> separate
> > > > >> > > > > Apache
> > > > >> > > > > >> >>> > Incubator
> > > > >> > > > > >> >>> > > >> >> > project?
> > > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > Regards,
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > Saikat
> > > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > On Wed, Oct 16, 2019 at
> 6:21
> > AM
> > > > >> > > Emmanouil
> > > > >> > > > > >> >>> > Gkatziouras <
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > gkatzioura@gmail.com>
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > wrote:
> > > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > Hi all!
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > Based on the discussions
> > most
> > > > >> > probably
> > > > >> > > > > this
> > > > >> > > > > >> is
> > > > >> > > > > >> >>> > going
> > > > >> > > > > >> >>> > > to
> > > > >> > > > > >> >>> > > >> >> be on
> > > > >> > > > > >> >>> > > >> >> > > > >>> another
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > GitHub repo. Therefore I
> > will
> > > > >> > prepare
> > > > >> > > a
> > > > >> > > > > >> >>> standalone
> > > > >> > > > > >> >>> > > >> project
> > > > >> > > > > >> >>> > > >> >> > with
> > > > >> > > > > >> >>> > > >> >> > > > the
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > Pub/Sub
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > module and once the
> > > repository
> > > > is
> > > > >> > > > created
> > > > >> > > > > a
> > > > >> > > > > >> >>> pull
> > > > >> > > > > >> >>> > > >> request
> > > > >> > > > > >> >>> > > >> >> > shall
> > > > >> > > > > >> >>> > > >> >> > > be
> > > > >> > > > > >> >>> > > >> >> > > > >>> > issued.
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > If there is anything
> else I
> > > can
> > > > >> do
> > > > >> > in
> > > > >> > > > > order
> > > > >> > > > > >> to
> > > > >> > > > > >> >>> > > >> facilitate
> > > > >> > > > > >> >>> > > >> >> > this
> > > > >> > > > > >> >>> > > >> >> > > > >>> please
> > > > >> > > > > >> >>> > > >> >> > > > >>> > let
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > me know.
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > If a ticket is created
> for
> > > this
> > > > >> > > issue, I
> > > > >> > > > > >> shall
> > > > >> > > > > >> >>> be
> > > > >> > > > > >> >>> > > able
> > > > >> > > > > >> >>> > > >> to
> > > > >> > > > > >> >>> > > >> >> > > create
> > > > >> > > > > >> >>> > > >> >> > > > a
> > > > >> > > > > >> >>> > > >> >> > > > >>> pull
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > request and thus a build
> > will
> > > > >> take
> > > > >> > > place
> > > > >> > > > > on
> > > > >> > > > > >> >>> > ignite's
> > > > >> > > > > >> >>> > > >> CI.
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > If everyone is aligned
> with
> > > > >> adding a
> > > > >> > > > > Pub/Sub
> > > > >> > > > > >> >>> > > >> >> implementation
> > > > >> > > > > >> >>> > > >> >> > > may I
> > > > >> > > > > >> >>> > > >> >> > > > >>> > create
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > a
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > ticket on JIRA?
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > My next question has to
> do
> > on
> > > > how
> > > > >> > > Ignite
> > > > >> > > > > >> >>> configures
> > > > >> > > > > >> >>> > > the
> > > > >> > > > > >> >>> > > >> >> build
> > > > >> > > > > >> >>> > > >> >> > > > jobs
> > > > >> > > > > >> >>> > > >> >> > > > >>> in
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > Team
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > City. The unit tests for
> my
> > > > >> > > > implementation
> > > > >> > > > > >> >>> need a
> > > > >> > > > > >> >>> > > local
> > > > >> > > > > >> >>> > > >> >> > Pub/Sub
> > > > >> > > > > >> >>> > > >> >> > > > >>> server
> > > > >> > > > > >> >>> > > >> >> > > > >>> > up
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > and running (they provide
> > one
> > > > for
> > > > >> > > > testing
> > > > >> > > > > >> >>> > purposes).
> > > > >> > > > > >> >>> > > >> Will
> > > > >> > > > > >> >>> > > >> >> > this
> > > > >> > > > > >> >>> > > >> >> > > > take
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > effect
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > in a form of a build
> script
> > > > >> inside
> > > > >> > my
> > > > >> > > > > >> >>> > implementation
> > > > >> > > > > >> >>> > > or
> > > > >> > > > > >> >>> > > >> >> is it
> > > > >> > > > > >> >>> > > >> >> > > > >>> something
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > that should be configured
> > on
> > > > Team
> > > > >> > > City?
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > Kind regards,
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > Emmanouil
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > *Emmanouil Gkatziouras*
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > https://egkatzioura.com/
> |
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> > > > > >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > https://github.com/gkatzioura
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > On Tue, 15 Oct 2019 at
> > 23:27,
> > > > >> Denis
> > > > >> > > > Magda
> > > > >> > > > > <
> > > > >> > > > > >> >>> > > >> >> dmagda@apache.org
> > > > >> > > > > >> >>> > > >> >> > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> wrote:
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > Emmanouil, Saikat,
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > After contemplating on
> > > this,
> > > > >> let
> > > > >> > me
> > > > >> > > > > >> propose
> > > > >> > > > > >> >>> > another
> > > > >> > > > > >> >>> > > >> way
> > > > >> > > > > >> >>> > > >> >> > with
> > > > >> > > > > >> >>> > > >> >> > > > this
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > particular integration
> > that
> > > > is
> > > > >> > > aligned
> > > > >> > > > > >> with
> > > > >> > > > > >> >>> our
> > > > >> > > > > >> >>> > > >> >> > > modularization
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > endeavor.
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > The modularization [1]
> > > > defines
> > > > >> the
> > > > >> > > > > Ignite
> > > > >> > > > > >> >>> core,
> > > > >> > > > > >> >>> > > >> Ignite
> > > > >> > > > > >> >>> > > >> >> > > modules,
> > > > >> > > > > >> >>> > > >> >> > > > >>> and
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > independent
> integrations.
> > > The
> > > > >> > first
> > > > >> > > > two
> > > > >> > > > > >> are
> > > > >> > > > > >> >>> to be
> > > > >> > > > > >> >>> > > >> >> located
> > > > >> > > > > >> >>> > > >> >> > in
> > > > >> > > > > >> >>> > > >> >> > > > >>> Ignite
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > repositories and to be
> > > > >> > > tested/updated
> > > > >> > > > by
> > > > >> > > > > >> the
> > > > >> > > > > >> >>> > > >> community
> > > > >> > > > > >> >>> > > >> >> for
> > > > >> > > > > >> >>> > > >> >> > > > every
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > release.
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > The modules are a
> subset
> > of
> > > > >> > existing
> > > > >> > > > > >> >>> integrations
> > > > >> > > > > >> >>> > > >> that
> > > > >> > > > > >> >>> > > >> >> are
> > > > >> > > > > >> >>> > > >> >> > > > >>> adopted
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > widely
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > and require community
> > > > >> attention.
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > As for the independent
> > > > >> > integrations
> > > > >> > > > [1],
> > > > >> > > > > >> >>> those
> > > > >> > > > > >> >>> > are
> > > > >> > > > > >> >>> > > >> >> supposed
> > > > >> > > > > >> >>> > > >> >> > > to
> > > > >> > > > > >> >>> > > >> >> > > > be
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > located
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > in a separate GitHub
> > repo.
> > > I
> > > > >> would
> > > > >> > > > > advise
> > > > >> > > > > >> us
> > > > >> > > > > >> >>> to
> > > > >> > > > > >> >>> > > start
> > > > >> > > > > >> >>> > > >> >> > > > >>> implementing
> > > > >> > > > > >> >>> > > >> >> > > > >>> > this
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > concept with the
> Pub/Sub
> > > > >> > > integration.
> > > > >> > > > > Even
> > > > >> > > > > >> >>> though
> > > > >> > > > > >> >>> > > it
> > > > >> > > > > >> >>> > > >> >> will
> > > > >> > > > > >> >>> > > >> >> > be
> > > > >> > > > > >> >>> > > >> >> > > > >>> located
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > in a
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > separate Github repo,
> it
> > > will
> > > > >> be
> > > > >> > > > listed
> > > > >> > > > > >> among
> > > > >> > > > > >> >>> > > >> officially
> > > > >> > > > > >> >>> > > >> >> > > > >>> available
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > integrations on the
> > Ignite
> > > > >> website
> > > > >> > > > > >> featuring
> > > > >> > > > > >> >>> > > >> Emmanouil
> > > > >> > > > > >> >>> > > >> >> as a
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > contributor.
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > What do you think?
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > [1]
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > > >> > > > > >> >>> > > >> >> > > > >>>
> > > > >> > > > > >> >>> > > >> >> > > >
> > > > >> > > > > >> >>> > > >> >> > >
> > > > >> > > > > >> >>> > > >> >> >
> > > > >> > > > > >> >>> > > >> >>
> > > > >> > > > > >> >>> > > >>
> > > > >> > > > > >> >>> > >
> > > > >> > > > > >> >>> >
> > > > >> > > > > >> >>>
> > > > >> > > > > >>
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > -
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > Denis
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at
> > > 9:47
> > > > PM
> > > > >> > > Saikat
> > > > >> > > > > >> >>> Maitra <
> > > > >> > > > > >> >>> > > >> >> > > > >>> > saikat.maitra@gmail.com
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > wrote:
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > Hi Emmanouil,
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > The changes looks
> good
> > to
> > > > >> me. I
> > > > >> > > > wanted
> > > > >> > > > > >> to
> > > > >> > > > > >> >>> check
> > > > >> > > > > >> >>> > > if
> > > > >> > > > > >> >>> > > >> you
> > > > >> > > > > >> >>> > > >> >> > are
> > > > >> > > > > >> >>> > > >> >> > > > also
> > > > >> > > > > >> >>> > > >> >> > > > >>> > able
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > to
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > validate that the
> once
> > > data
> > > > >> is
> > > > >> > > added
> > > > >> > > > > to
> > > > >> > > > > >> >>> Ignite
> > > > >> > > > > >> >>> > > >> cluster
> > > > >> > > > > >> >>> > > >> >> > you
> > > > >> > > > > >> >>> > > >> >> > > > are
> > > > >> > > > > >> >>> > > >> >> > > > >>> also
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > able
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > to
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > access it using
> another
> > > > >> Ignite
> > > > >> > > > client
> > > > >> > > > > or
> > > > >> > > > > >> >>> using
> > > > >> > > > > >> >>> > > rest
> > > > >> > > > > >> >>> > > >> >> > > > endpoints.
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > We use Teamcity for
> CI
> > > > >> process,
> > > > >> > > the
> > > > >> > > > > >> >>> details are
> > > > >> > > > > >> >>> > > as
> > > > >> > > > > >> >>> > > >> >> > follows
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > Trigger validation of
> > > those
> > > > >> test
> > > > >> > > > > suites
> > > > >> > > > > >> >>> that
> > > > >> > > > > >> >>> > have
> > > > >> > > > > >> >>> > > >> been
> > > > >> > > > > >> >>> > > >> >> > > > >>> affected by
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > your
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > changes on TeamCity <
> > > > >> > > > > >> >>> > > http://ci.ignite.apache.org/>:
> > > > >> > > > > >> >>> > > >> <
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > http://204.14.53.153/
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Locate a test
> > suite
> > > > you
> > > > >> > have
> > > > >> > > to
> > > > >> > > > > >> check,
> > > > >> > > > > >> >>> > press
> > > > >> > > > > >> >>> > > >> >> button
> > > > >> > > > > >> >>> > > >> >> > > > named
> > > > >> > > > > >> >>> > > >> >> > > > >>> > "..."
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > that
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    is located on the
> > left
> > > > of
> > > > >> > "Run"
> > > > >> > > > > >> button.
> > > > >> > > > > >> >>> "Run
> > > > >> > > > > >> >>> > > >> custom
> > > > >> > > > > >> >>> > > >> >> > > build"
> > > > >> > > > > >> >>> > > >> >> > > > >>> > dialog
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > will
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    appear;
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Go to "Changes"
> > tab
> > > > and
> > > > >> > > choose
> > > > >> > > > > >> >>> > > >> >> > > > >>> >
> > "pull/<pull-request-number>/head"
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > in
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    "Build branch"
> > > dropdown
> > > > >> list;
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Press "Run
> build"
> > > > button
> > > > >> > and
> > > > >> > > > > >> monitor
> > > > >> > > > > >> >>> tests
> > > > >> > > > > >> >>> > > >> >> results.
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > @Roman
> > rshtykh@yahoo.com
> > > -
> > > > >> can
> > > > >> > > you
> > > > >> > > > > also
> > > > >> > > > > >> >>> please
> > > > >> > > > > >> >>> > > >> take a
> > > > >> > > > > >> >>> > > >> >> > > look?
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > Regards,
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > Saikat
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > On Sun, Oct 13, 2019
> at
> > > > 6:20
> > > > >> PM
> > > > >> > > > > >> Emmanouil
> > > > >> > > > > >> >>> > > >> Gkatziouras
> > > > >> > > > > >> >>> > > >> >> <
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > gkatzioura@gmail.com
> >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > wrote:
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Hi @Saikat Maitra
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > I have my
> > > implementation
> > > > >> on an
> > > > >> > > > > Ignite
> > > > >> > > > > >> >>> fork on
> > > > >> > > > > >> >>> > > my
> > > > >> > > > > >> >>> > > >> >> > personal
> > > > >> > > > > >> >>> > > >> >> > > > >>> GitHub
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > account
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > [1]
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > It is based on the
> > > Kafka
> > > > >> > > > > >> implementation
> > > > >> > > > > >> >>> which
> > > > >> > > > > >> >>> > > is
> > > > >> > > > > >> >>> > > >> of
> > > > >> > > > > >> >>> > > >> >> a
> > > > >> > > > > >> >>> > > >> >> > > > polling
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > nature.
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > GCP does not
> provide
> > a
> > > > >> service
> > > > >> > > > like
> > > > >> > > > > >> >>> Kafka.
> > > > >> > > > > >> >>> > > >> Instead
> > > > >> > > > > >> >>> > > >> >> it
> > > > >> > > > > >> >>> > > >> >> > > > >>> provides us
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > with
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Pub/Sub which is
> just
> > > > like
> > > > >> > > Kafka.
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Therefore using the
> > > Kafka
> > > > >> > > streamer
> > > > >> > > > > as
> > > > >> > > > > >> a
> > > > >> > > > > >> >>> > > guideline
> > > > >> > > > > >> >>> > > >> >> made
> > > > >> > > > > >> >>> > > >> >> > > more
> > > > >> > > > > >> >>> > > >> >> > > > >>> sense
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > to
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > me.
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Apart from the
> > > > >> implementation
> > > > >> > I
> > > > >> > > > have
> > > > >> > > > > >> also
> > > > >> > > > > >> >>> > > added a
> > > > >> > > > > >> >>> > > >> >> unit
> > > > >> > > > > >> >>> > > >> >> > > > test.
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > It definitely needs
> > > some
> > > > >> > > polishing
> > > > >> > > > > >> and a
> > > > >> > > > > >> >>> > double
> > > > >> > > > > >> >>> > > >> >> check
> > > > >> > > > > >> >>> > > >> >> > on
> > > > >> > > > > >> >>> > > >> >> > > my
> > > > >> > > > > >> >>> > > >> >> > > > >>> side
> > > > >> > > > > >> >>> > > >> >> > > > >>> > to
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > make
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > the review smooth.
> > > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > For the tests a
> local
> > > > >> Pub/Sub
> > > > >> > > [2]
> > > > >> > > > > >> server
> > > > >> > > >
> > > > >
> > > > >
> > > >
> > >
> >
>

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Emmanouil Gkatziouras <gk...@gmail.com>]

Hi Saikat!

Thank you for your assistance on that!

Kind regards

*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura


On Sat, 30 Nov 2019 at 23:51, Saikat Maitra <sa...@gmail.com> wrote:

> Hi Emmanouil,
>
> I looked into the build logs and I observed that since I added artifacts
> dependencies to "Build apache Ignite"[1] it was able to pull the required
> dependencies and was able to run the existing tests. It is however not
> identifying new modules like I changed from flink to flink-ext or pub-sub.
>
> [1]
>
> https://ci.ignite.apache.org/admin/editDependencies.html?id=buildType:IgniteExtensions_Build
>
> I will look into the issue and debug further.
>
> Regards,
> Saikat
>
>
> On Sat, Nov 30, 2019 at 5:35 PM Emmanouil Gkatziouras <
> gkatzioura@gmail.com>
> wrote:
>
> > Hi Saikat!
> >
> > From the logs[1] it seems that TC cannot find the project. However I did
> > include the project on the parent pom (modules sections) [2]
> > I tried to reproduce locally unfortunately I am not aware of the full
> > context on TC. It seems as if a different parent pom is being picked up.
> >
> > Kind regards
> >
> > [1]
> >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4804893&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=1704
> > [2]
> >
> >
> https://github.com/gkatzioura/ignite-extensions/blob/d6a8beee7feff28f59d9a12be692016305564d25/pom.xml#L46
> >
> > *Emmanouil Gkatziouras*
> > https://egkatzioura.com/ |
> > https://www.linkedin.com/in/gkatziourasemmanouil/
> > https://github.com/gkatzioura
> >
> >
> > On Sat, 30 Nov 2019 at 22:20, Saikat Maitra <sa...@gmail.com>
> > wrote:
> >
> > > Hello,
> > >
> > > I have made changes in Ignite Extensions build configurations.
> > >
> > > It is now running the tests as expected.
> > >
> > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4804480&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> > >
> > > Regards,
> > > Saikat
> > >
> > > On Thu, Nov 28, 2019 at 1:29 PM Saikat Maitra <saikat.maitra@gmail.com
> >
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > We do run build with -DskipTests and then configure selective test
> > suites
> > > > to run.
> > > >
> > > > I will check why these test suites are not getting executed.
> > > >
> > > > Regards
> > > > Saikat
> > > >
> > > > On Wed, 27 Nov 2019 at 3:05 PM, Emmanouil Gkatziouras <
> > > > gkatzioura@gmail.com> wrote:
> > > >
> > > >> Hi all,
> > > >>
> > > >> I did add a test suite and run the build with the corresponding
> > > arguments,
> > > >> however the tests did not run [1].
> > > >> I checked the Flink's build logs and the same message is displayed
> > there
> > > >> too `No tests to run` [2].
> > > >>
> > > >> [1]
> > > >>
> > > >>
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4798203&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=248
> > > >> [2]
> > > >>
> > > >>
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4789199&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=1256
> > > >>
> > > >> Kind regards
> > > >> *Emmanouil Gkatziouras*
> > > >> https://egkatzioura.com/ |
> > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> https://github.com/gkatzioura
> > > >>
> > > >>
> > > >> On Wed, 27 Nov 2019 at 18:58, Saikat Maitra <
> saikat.maitra@gmail.com>
> > > >> wrote:
> > > >>
> > > >> > Hi,
> > > >> >
> > > >> > We will need to add test suites for tests to be executed in build.
> > We
> > > >> pass
> > > >> > them as TestSuites param. I had added Flink sink and source
> > > testsuites.
> > > >> >
> > > >> > Can you please review and confirm.
> > > >> >
> > > >> > Regards
> > > >> > Saikat
> > > >> >
> > > >> > On Wed, 27 Nov 2019 at 6:00 AM, Emmanouil Gkatziouras <
> > > >> > gkatzioura@gmail.com>
> > > >> > wrote:
> > > >> >
> > > >> > > Hi all!
> > > >> > >
> > > >> > > I was successful building and running my pull requests. It seems
> > > that
> > > >> the
> > > >> > > tests do not run both for Flink and Pub/Sub [1]
> > > >> > > If there is something I cannot do to make them discoverable
> please
> > > >> let me
> > > >> > > know.
> > > >> > >
> > > >> > > [1]
> > > >> > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4796722&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=283
> > > >> > >
> > > >> > > *Emmanouil Gkatziouras*
> > > >> > > https://egkatzioura.com/ |
> > > >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> > > https://github.com/gkatzioura
> > > >> > >
> > > >> > >
> > > >> > > On Wed, 27 Nov 2019 at 04:57, Saikat Maitra <
> > > saikat.maitra@gmail.com>
> > > >> > > wrote:
> > > >> > >
> > > >> > > > Hi Emmanouil,
> > > >> > > >
> > > >> > > > I have added you as contributor in Ignite Extensions project.
> > Can
> > > >> you
> > > >> > > > please check and confirm if you are able to see the project
> and
> > > >> execute
> > > >> > > > build on your pull request.
> > > >> > > >
> > > >> > > > Regards,
> > > >> > > > Saikat
> > > >> > > >
> > > >> > > > On Tue, Nov 26, 2019 at 12:49 PM Emmanouil Gkatziouras <
> > > >> > > > gkatzioura@gmail.com>
> > > >> > > > wrote:
> > > >> > > >
> > > >> > > > > Hi all!
> > > >> > > > >
> > > >> > > > > If someone could give me read access to the Job of Ignite
> > > >> Extensions
> > > >> > on
> > > >> > > > > Team City will greatly help me (username gkatzioura).
> > > >> > > > > I suppose the builds get triggered automatically on a pull
> > > >> request.
> > > >> > > > >
> > > >> > > > > Kind regards,
> > > >> > > > > Emmanouil
> > > >> > > > >
> > > >> > > > > *Emmanouil Gkatziouras*
> > > >> > > > > https://egkatzioura.com/ |
> > > >> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> > > > > https://github.com/gkatzioura
> > > >> > > > >
> > > >> > > > >
> > > >> > > > > On Sun, 24 Nov 2019 at 21:11, Emmanouil Gkatziouras <
> > > >> > > > gkatzioura@gmail.com>
> > > >> > > > > wrote:
> > > >> > > > >
> > > >> > > > > > Hi Saikat!
> > > >> > > > > >
> > > >> > > > > > I just rebased with the flink branch. Unfortunately I do
> not
> > > >> have
> > > >> > > read
> > > >> > > > > > access to the team city link you provided and thus
> evaluate
> > > the
> > > >> > tests
> > > >> > > > for
> > > >> > > > > > my pull request.
> > > >> > > > > > I guess it has to do with ignite extensions being new.
> > > >> > > > > > My username is gkatzioura and email the one I am using on
> > this
> > > >> > email
> > > >> > > > > >
> > > >> > > > > > Thank you for your time.
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > > *Emmanouil Gkatziouras*
> > > >> > > > > > https://egkatzioura.com/ |
> > > >> > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> > > > > > https://github.com/gkatzioura
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > > On Sun, 24 Nov 2019 at 18:03, Saikat Maitra <
> > > >> > saikat.maitra@gmail.com
> > > >> > > >
> > > >> > > > > > wrote:
> > > >> > > > > >
> > > >> > > > > >> Hi Emmanouil,
> > > >> > > > > >>
> > > >> > > > > >> The latest build on teamcity has passed on Flink pull
> > > request.
> > > >> > > > > >>
> > > >> > > > > >>
> > > >> > > > > >>
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4788928&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> > > >> > > > > >>
> > > >> > > > > >> you should be able to take the changes I made in my PR
> and
> > > run
> > > >> > build
> > > >> > > > on
> > > >> > > > > >> your pull request.
> > > >> > > > > >>
> > > >> > > > > >> Please let me know if you have any questions.
> > > >> > > > > >>
> > > >> > > > > >> Regards,
> > > >> > > > > >> Saikat
> > > >> > > > > >>
> > > >> > > > > >> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra <
> > > >> > > > saikat.maitra@gmail.com
> > > >> > > > > >
> > > >> > > > > >> wrote:
> > > >> > > > > >>
> > > >> > > > > >> > Hi Emmanouil,
> > > >> > > > > >> >
> > > >> > > > > >> > I have fixed the Flink module testsuites and have setup
> > new
> > > >> > > project
> > > >> > > > in
> > > >> > > > > >> > teamcity.
> > > >> > > > > >> >
> > > >> > > > > >> >
> > > >> > > > > >> >
> > > >> > > > > >>
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview
> > > >> > > > > >> >
> > > >> > > > > >> > I am looking into setting up the build.
> > > >> > > > > >> >
> > > >> > > > > >> > Regards,
> > > >> > > > > >> > Saikat
> > > >> > > > > >> >
> > > >> > > > > >> > On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra <
> > > >> > > > > saikat.maitra@gmail.com>
> > > >> > > > > >> > wrote:
> > > >> > > > > >> >
> > > >> > > > > >> >> Hi Emmanouil,
> > > >> > > > > >> >>
> > > >> > > > > >> >> Thank you for your email. Yes, the plan is once the
> > Flink
> > > >> PR is
> > > >> > > > > merged
> > > >> > > > > >> it
> > > >> > > > > >> >> will provide the Licence, parent POMs etc and you can
> > > rebase
> > > >> > from
> > > >> > > > > >> master
> > > >> > > > > >> >> branch and apply the changes on top of it.
> > > >> > > > > >> >>
> > > >> > > > > >> >> I am getting some issues with test failures in local
> > with
> > > >> > > > > >> >> GridTestProperties as the test.properties is not
> present
> > > in
> > > >> > this
> > > >> > > > new
> > > >> > > > > >> >> project but should be available from dependencies.
> > > >> > > > > >> >>
> > > >> > > > > >> >> Once I address this issue, I will go ahead and merge
> the
> > > >> > changes
> > > >> > > > and
> > > >> > > > > >> then
> > > >> > > > > >> >> we can take it from there.
> > > >> > > > > >> >>
> > > >> > > > > >> >> Regards,
> > > >> > > > > >> >> Saikat
> > > >> > > > > >> >>
> > > >> > > > > >> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras
> <
> > > >> > > > > >> >> gkatzioura@gmail.com> wrote:
> > > >> > > > > >> >>
> > > >> > > > > >> >>> Hi all,
> > > >> > > > > >> >>>
> > > >> > > > > >> >>> I made my first pull request [1]. Since this project
> is
> > > >> brand
> > > >> > > new
> > > >> > > > > (no
> > > >> > > > > >> >>> parent poms, licensing), I reckoned it was best to
> use
> > > >> > Saikat's
> > > >> > > > > >> branch on
> > > >> > > > > >> >>> Flink.
> > > >> > > > > >> >>> I suppose the Flink branch will be merged first. If
> not
> > > >> please
> > > >> > > > give
> > > >> > > > > me
> > > >> > > > > >> >>> guidelines on how I should proceed next.
> > > >> > > > > >> >>>
> > > >> > > > > >> >>> Kind regards
> > > >> > > > > >> >>> Emmanouil
> > > >> > > > > >> >>>
> > > >> > > > > >> >>> [1]
> https://github.com/apache/ignite-extensions/pull/2
> > > >> > > > > >> >>>
> > > >> > > > > >> >>> *Emmanouil Gkatziouras*
> > > >> > > > > >> >>> https://egkatzioura.com/ |
> > > >> > > > > >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> > > > > >> >>> https://github.com/gkatzioura
> > > >> > > > > >> >>>
> > > >> > > > > >> >>>
> > > >> > > > > >> >>> On Fri, 22 Nov 2019 at 20:55, Denis Magda <
> > > >> dmagda@apache.org>
> > > >> > > > > wrote:
> > > >> > > > > >> >>>
> > > >> > > > > >> >>> > Awesome, ping us whenever you're ready!
> > > >> > > > > >> >>> >
> > > >> > > > > >> >>> > -
> > > >> > > > > >> >>> > Denis
> > > >> > > > > >> >>> >
> > > >> > > > > >> >>> >
> > > >> > > > > >> >>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil
> > Gkatziouras
> > > <
> > > >> > > > > >> >>> > gkatzioura@gmail.com>
> > > >> > > > > >> >>> > wrote:
> > > >> > > > > >> >>> >
> > > >> > > > > >> >>> > > Hi all!
> > > >> > > > > >> >>> > >
> > > >> > > > > >> >>> > > I am sorry for being late on that. I was trying
> to
> > > >> > refactor
> > > >> > > > the
> > > >> > > > > >> test
> > > >> > > > > >> >>> in
> > > >> > > > > >> >>> > > order not to be in need of any external tools or
> > > >> spinning
> > > >> > > up a
> > > >> > > > > >> >>> server.
> > > >> > > > > >> >>> > > I did forked the new repo and indeed my changes
> > > there,
> > > >> so
> > > >> > a
> > > >> > > > pull
> > > >> > > > > >> >>> request
> > > >> > > > > >> >>> > is
> > > >> > > > > >> >>> > > a matter of time!
> > > >> > > > > >> >>> > >
> > > >> > > > > >> >>> > > Kind regards
> > > >> > > > > >> >>> > >
> > > >> > > > > >> >>> > > *Emmanouil Gkatziouras*
> > > >> > > > > >> >>> > > https://egkatzioura.com/ |
> > > >> > > > > >> >>> > >
> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> > > > > >> >>> > > https://github.com/gkatzioura
> > > >> > > > > >> >>> > >
> > > >> > > > > >> >>> > >
> > > >> > > > > >> >>> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda <
> > > >> > > dmagda@apache.org>
> > > >> > > > > >> wrote:
> > > >> > > > > >> >>> > >
> > > >> > > > > >> >>> > > > Hi Emmanouil,
> > > >> > > > > >> >>> > > >
> > > >> > > > > >> >>> > > > Do you have any questions or need any support
> > from
> > > >> the
> > > >> > > > > >> community?
> > > >> > > > > >> >>> > > >
> > > >> > > > > >> >>> > > > -
> > > >> > > > > >> >>> > > > Denis
> > > >> > > > > >> >>> > > >
> > > >> > > > > >> >>> > > >
> > > >> > > > > >> >>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <
> > > >> > > > > >> >>> saikat.maitra@gmail.com
> > > >> > > > > >> >>> > >
> > > >> > > > > >> >>> > > > wrote:
> > > >> > > > > >> >>> > > >
> > > >> > > > > >> >>> > > >> Hi Emmanouil,
> > > >> > > > > >> >>> > > >>
> > > >> > > > > >> >>> > > >> Can you please take a looks at dev utils, if
> > this
> > > is
> > > >> > > > > something
> > > >> > > > > >> >>> you are
> > > >> > > > > >> >>> > > >> looking for
> > > >> > > > > >> >>> > > >>
> > > >> > > > >
> > https://github.com/apache/ignite/tree/master/modules/dev-utils
> > > >> > > > > >> ?
> > > >> > > > > >> >>> > > >>
> > > >> > > > > >> >>> > > >> IMO, if you can release Pub/Sub server in
> maven
> > > and
> > > >> > then
> > > >> > > > use
> > > >> > > > > >> it as
> > > >> > > > > >> >>> > > >> dependency, that would be great.
> > > >> > > > > >> >>> > > >>
> > > >> > > > > >> >>> > > >> Regards,
> > > >> > > > > >> >>> > > >> Saikat
> > > >> > > > > >> >>> > > >>
> > > >> > > > > >> >>> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra
> <
> > > >> > > > > >> >>> > saikat.maitra@gmail.com>
> > > >> > > > > >> >>> > > >> wrote:
> > > >> > > > > >> >>> > > >>
> > > >> > > > > >> >>> > > >> > Hi Emmanouil,
> > > >> > > > > >> >>> > > >> >
> > > >> > > > > >> >>> > > >> > The master branch has been initialized. I
> have
> > > >> > raised a
> > > >> > > > PR
> > > >> > > > > to
> > > >> > > > > >> >>> > migrate
> > > >> > > > > >> >>> > > >> > Flink module to ignite-extensions.
> > > >> > > > > >> >>> > > >> >
> > > >> > > > > >> >>> > > >> > PR
> > > >> > https://github.com/apache/ignite-extensions/pull/1
> > > >> > > > > >> >>> > > >> > Jira :
> > > >> > > > https://issues.apache.org/jira/browse/IGNITE-12356
> > > >> > > > > >> >>> > > >> >
> > > >> > > > > >> >>> > > >> > Regards,
> > > >> > > > > >> >>> > > >> > Saikat
> > > >> > > > > >> >>> > > >> >
> > > >> > > > > >> >>> > > >> >
> > > >> > > > > >> >>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil
> > > >> > Gkatziouras <
> > > >> > > > > >> >>> > > >> > gkatzioura@gmail.com> wrote:
> > > >> > > > > >> >>> > > >> >
> > > >> > > > > >> >>> > > >> >> Hi all!
> > > >> > > > > >> >>> > > >> >>
> > > >> > > > > >> >>> > > >> >> This is an update. The integration is
> tested
> > > with
> > > >> > > mocks
> > > >> > > > > >> based
> > > >> > > > > >> >>> on
> > > >> > > > > >> >>> > the
> > > >> > > > > >> >>> > > >> >> original Ignite repo [1].
> > > >> > > > > >> >>> > > >> >> I am currently waiting for the first
> commits
> > on
> > > >> the
> > > >> > > new
> > > >> > > > > >> >>> integration
> > > >> > > > > >> >>> > > >> >> project
> > > >> > > > > >> >>> > > >> >> to take place in order to create my pull
> > > request
> > > >> > > > > accordingly
> > > >> > > > > >> >>> to the
> > > >> > > > > >> >>> > > >> >> initial
> > > >> > > > > >> >>> > > >> >> commits.
> > > >> > > > > >> >>> > > >> >> In order to simulate the Pub/Sub server in
> a
> > > >> test I
> > > >> > > > used a
> > > >> > > > > >> test
> > > >> > > > > >> >>> > > server
> > > >> > > > > >> >>> > > >> >> implementation [2]. Unfortunately they
> don't
> > > have
> > > >> > this
> > > >> > > > > util
> > > >> > > > > >> >>> > uploaded
> > > >> > > > > >> >>> > > >> in a
> > > >> > > > > >> >>> > > >> >> maven repo, thus it needs to be hosted on a
> > > maven
> > > >> > repo
> > > >> > > > > >> (Apache
> > > >> > > > > >> >>> > > License
> > > >> > > > > >> >>> > > >> >> 2.0). Is there a repo for utilities that
> > Ignite
> > > >> uses
> > > >> > > for
> > > >> > > > > the
> > > >> > > > > >> >>> > > >> test/build?
> > > >> > > > > >> >>> > > >> >> I have also created a ticket [3]
> > > >> > > > > >> >>> > > >> >>
> > > >> > > > > >> >>> > > >> >> Kind regards
> > > >> > > > > >> >>> > > >> >>
> > > >> > > > > >> >>> > > >> >> [1]
> > > >> > > > > >> >>> >
> > > >> > > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > > >> > > > > >> >>> > > >> >> [2]
> > > >> > > > > >> >>>
> > > >> https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> > > >> > > > > >> >>> > > >> >> [3]
> > > >> > > https://issues.apache.org/jira/browse/IGNITE-12262
> > > >> > > > > >> >>> > > >> >>
> > > >> > > > > >> >>> > > >> >> *Emmanouil Gkatziouras*
> > > >> > > > > >> >>> > > >> >> https://egkatzioura.com/ |
> > > >> > > > > >> >>> > > >> >>
> > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> > > > > >> >>> > > >> >> https://github.com/gkatzioura
> > > >> > > > > >> >>> > > >> >>
> > > >> > > > > >> >>> > > >> >>
> > > >> > > > > >> >>> > > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat
> Maitra <
> > > >> > > > > >> >>> > saikat.maitra@gmail.com
> > > >> > > > > >> >>> > > >
> > > >> > > > > >> >>> > > >> >> wrote:
> > > >> > > > > >> >>> > > >> >>
> > > >> > > > > >> >>> > > >> >> > Hello Ilya,
> > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > >> >>> > > >> >> > Thank you for your email. Yes, before we
> > > remove
> > > >> > any
> > > >> > > > > >> >>> integration
> > > >> > > > > >> >>> > we
> > > >> > > > > >> >>> > > >> will
> > > >> > > > > >> >>> > > >> >> > move them to another repository and
> > release.
> > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > >> >>> > > >> >> > We are thinking we will start with
> Pub/Sub
> > > >> > Streamer
> > > >> > > > > >> >>> > implementation
> > > >> > > > > >> >>> > > >> and
> > > >> > > > > >> >>> > > >> >> > continue to move other integration as
> > > >> mentioned in
> > > >> > > the
> > > >> > > > > >> >>> following
> > > >> > > > > >> >>> > > doc.
> > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > >> >>> > > >> >>
> > > >> > > > > >> >>> > > >>
> > > >> > > > > >> >>> > >
> > > >> > > > > >> >>> >
> > > >> > > > > >> >>>
> > > >> > > > > >>
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > >> >>> > > >> >> > Regards,
> > > >> > > > > >> >>> > > >> >> > Saikat
> > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > >> >>> > > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya
> > > >> Kasnacheev <
> > > >> > > > > >> >>> > > >> >> > ilya.kasnacheev@gmail.com>
> > > >> > > > > >> >>> > > >> >> > wrote:
> > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > >> >>> > > >> >> > > Hello!
> > > >> > > > > >> >>> > > >> >> > >
> > > >> > > > > >> >>> > > >> >> > > My take on this: we want to offload
> some
> > of
> > > >> > > > > >> integrations,
> > > >> > > > > >> >>> but
> > > >> > > > > >> >>> > > >> please
> > > >> > > > > >> >>> > > >> >> > commit
> > > >> > > > > >> >>> > > >> >> > > and release them to another project
> > first,
> > > >> and
> > > >> > > only
> > > >> > > > > >> then we
> > > >> > > > > >> >>> > will
> > > >> > > > > >> >>> > > >> >> decide
> > > >> > > > > >> >>> > > >> >> > to
> > > >> > > > > >> >>> > > >> >> > > remove them from our repository.
> > > >> > > > > >> >>> > > >> >> > >
> > > >> > > > > >> >>> > > >> >> > > Please also make sure they are
> available
> > > with
> > > >> > same
> > > >> > > > > >> artifact
> > > >> > > > > >> >>> > name
> > > >> > > > > >> >>> > > >> and
> > > >> > > > > >> >>> > > >> >> API.
> > > >> > > > > >> >>> > > >> >> > > If this has to be changed, let's wait
> for
> > > AI
> > > >> 3.0
> > > >> > > > > >> >>> > > >> >> > >
> > > >> > > > > >> >>> > > >> >> > > Regards,
> > > >> > > > > >> >>> > > >> >> > > --
> > > >> > > > > >> >>> > > >> >> > > Ilya Kasnacheev
> > > >> > > > > >> >>> > > >> >> > >
> > > >> > > > > >> >>> > > >> >> > >
> > > >> > > > > >> >>> > > >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat
> > Maitra
> > > <
> > > >> > > > > >> >>> > > >> saikat.maitra@gmail.com>:
> > > >> > > > > >> >>> > > >> >> > >
> > > >> > > > > >> >>> > > >> >> > > > Hello Emmanouil,
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > > > As discussed earlier, I discussed
> with
> > > >> Apache
> > > >> > > > Bahir
> > > >> > > > > >> >>> community
> > > >> > > > > >> >>> > > and
> > > >> > > > > >> >>> > > >> >> they
> > > >> > > > > >> >>> > > >> >> > > are
> > > >> > > > > >> >>> > > >> >> > > > interested to have Apache Ignite
> > > >> extensions as
> > > >> > > > part
> > > >> > > > > of
> > > >> > > > > >> >>> Apache
> > > >> > > > > >> >>> > > >> Bahir
> > > >> > > > > >> >>> > > >> >> > > > project.
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > > > Can you please share Pub/Sub streamer
> > > >> > > > implementation
> > > >> > > > > >> >>> details
> > > >> > > > > >> >>> > > with
> > > >> > > > > >> >>> > > >> >> > Apache
> > > >> > > > > >> >>> > > >> >> > > > Bahir community and request for
> > feedback.
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > > > https://bahir.apache.org/
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > > > I have also requested for contributor
> > > >> access
> > > >> > in
> > > >> > > > Jira
> > > >> > > > > >> for
> > > >> > > > > >> >>> > Apache
> > > >> > > > > >> >>> > > >> >> Bahir
> > > >> > > > > >> >>> > > >> >> > > > project so that I can create issues
> and
> > > >> assign
> > > >> > > to
> > > >> > > > > >> myself.
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > > > I can help with code reviews as well.
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > > > Here is the email thread for
> reference
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > >
> > > >> > > >
> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > > > Regards,
> > > >> > > > > >> >>> > > >> >> > > > Saikat
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM
> Saikat
> > > >> Maitra
> > > >> > <
> > > >> > > > > >> >>> > > >> >> saikat.maitra@gmail.com
> > > >> > > > > >> >>> > > >> >> > >
> > > >> > > > > >> >>> > > >> >> > > > wrote:
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > > > > Hello,
> > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > >> >>> > > >> >> > > > > As discussed I have created
> following
> > > >> > > discussion
> > > >> > > > > >> >>> threads on
> > > >> > > > > >> >>> > > our
> > > >> > > > > >> >>> > > >> >> > > migration
> > > >> > > > > >> >>> > > >> >> > > > > proposals for Apache Ignite
> > extensions:
> > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > >
> > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > >> >>> > > >> >>
> > > >> > > > > >> >>> > > >>
> > > >> > > > > >> >>> > >
> > > >> > > > > >> >>> >
> > > >> > > > > >> >>>
> > > >> > > > > >>
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > >> >>> > > >>
> > > >> > > > > >>
> > > >> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > >> >>> > > >> >> > > > > I will share update as I hear more
> > > >> > > information.
> > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > >> >>> > > >> >> > > > > Regards,
> > > >> > > > > >> >>> > > >> >> > > > > Saikat
> > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > >> >>> > > >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM
> > Saikat
> > > >> > Maitra
> > > >> > > <
> > > >> > > > > >> >>> > > >> >> > saikat.maitra@gmail.com
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > > > > wrote:
> > > >> > > > > >> >>> > > >> >> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >> Hello Denis,
> > > >> > > > > >> >>> > > >> >> > > > >>
> > > >> > > > > >> >>> > > >> >> > > > >> Sure, sounds good. I will create a
> > > >> separate
> > > >> > > > > >> discussion
> > > >> > > > > >> >>> > > thread
> > > >> > > > > >> >>> > > >> to
> > > >> > > > > >> >>> > > >> >> get
> > > >> > > > > >> >>> > > >> >> > > > >> community feedback on whether we
> > > >> > > > > >> >>> > > >> >> > > > >> should join the Bahir or kick-off
> > > >> "Ignite
> > > >> > > > > >> Extensions"
> > > >> > > > > >> >>> > > project.
> > > >> > > > > >> >>> > > >> >> > > > >>
> > > >> > > > > >> >>> > > >> >> > > > >> Regards,
> > > >> > > > > >> >>> > > >> >> > > > >> Saikat
> > > >> > > > > >> >>> > > >> >> > > > >>
> > > >> > > > > >> >>> > > >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM
> > Denis
> > > >> > Magda <
> > > >> > > > > >> >>> > > >> dmagda@apache.org>
> > > >> > > > > >> >>> > > >> >> > > wrote:
> > > >> > > > > >> >>> > > >> >> > > > >>
> > > >> > > > > >> >>> > > >> >> > > > >>> Folks,
> > > >> > > > > >> >>> > > >> >> > > > >>>
> > > >> > > > > >> >>> > > >> >> > > > >>> The concept of Apache Bahir is
> > > >> precisely
> > > >> > > what
> > > >> > > > we
> > > >> > > > > >> >>> need!
> > > >> > > > > >> >>> > > >> Saikat,
> > > >> > > > > >> >>> > > >> >> > thanks
> > > >> > > > > >> >>> > > >> >> > > > for
> > > >> > > > > >> >>> > > >> >> > > > >>> doing the research. Why I believe
> > > the *
> > > >> > > idea*
> > > >> > > > > >> fits us
> > > >> > > > > >> >>> > well:
> > > >> > > > > >> >>> > > >> >> > > > >>>
> > > >> > > > > >> >>> > > >> >> > > > >>>    - All integrations can be
> stored
> > > in
> > > >> > > > separate
> > > >> > > > > >> >>> Github
> > > >> > > > > >> >>> > > >> >> repositories
> > > >> > > > > >> >>> > > >> >> > > and
> > > >> > > > > >> >>> > > >> >> > > > >>>    have their dev lifecycles.
> We've
> > > not
> > > >> > > > obliged
> > > >> > > > > to
> > > >> > > > > >> >>> couple
> > > >> > > > > >> >>> > > all
> > > >> > > > > >> >>> > > >> >> the
> > > >> > > > > >> >>> > > >> >> > > > >>> integrations
> > > >> > > > > >> >>> > > >> >> > > > >>>    in a single repo. For
> instance,
> > > >> Spark
> > > >> > can
> > > >> > > > be
> > > >> > > > > >> >>> located
> > > >> > > > > >> >>> > in
> > > >> > > > > >> >>> > > a
> > > >> > > > > >> >>> > > >> >> > > dedicated
> > > >> > > > > >> >>> > > >> >> > > > >>> repo
> > > >> > > > > >> >>> > > >> >> > > > >>>    while streaming integrations
> > might
> > > >> be
> > > >> > in
> > > >> > > a
> > > >> > > > > >> single
> > > >> > > > > >> >>> one.
> > > >> > > > > >> >>> > > >> It's
> > > >> > > > > >> >>> > > >> >> up
> > > >> > > > > >> >>> > > >> >> > to
> > > >> > > > > >> >>> > > >> >> > > > us.
> > > >> > > > > >> >>> > > >> >> > > > >>>    - All the repositories and
> > > >> integrations
> > > >> > > > > belong
> > > >> > > > > >> to
> > > >> > > > > >> >>> ASF.
> > > >> > > > > >> >>> > > >> We're
> > > >> > > > > >> >>> > > >> >> not
> > > >> > > > > >> >>> > > >> >> > > > >>> dumping
> > > >> > > > > >> >>> > > >> >> > > > >>>    them on Github but rather keep
> > > >> > supporting
> > > >> > > > and
> > > >> > > > > >> >>> > developing
> > > >> > > > > >> >>> > > >> in
> > > >> > > > > >> >>> > > >> >> > > > >>> accordance with
> > > >> > > > > >> >>> > > >> >> > > > >>>    ASF vision and practices.
> > > >> > > > > >> >>> > > >> >> > > > >>>    - There is a way to reward
> > > >> contributors
> > > >> > > via
> > > >> > > > > >> >>> > > committership
> > > >> > > > > >> >>> > > >> and
> > > >> > > > > >> >>> > > >> >> > PMC
> > > >> > > > > >> >>> > > >> >> > > > >>>    membership. You won't get this
> > if
> > > a
> > > >> > > project
> > > >> > > > > is
> > > >> > > > > >> >>> just
> > > >> > > > > >> >>> > one
> > > >> > > > > >> >>> > > of
> > > >> > > > > >> >>> > > >> >> the
> > > >> > > > > >> >>> > > >> >> > > > >>> millions of
> > > >> > > > > >> >>> > > >> >> > > > >>>    Github projects.
> > > >> > > > > >> >>> > > >> >> > > > >>>
> > > >> > > > > >> >>> > > >> >> > > > >>> Saikat, as Ignite PMC member,
> could
> > > you
> > > >> > > please
> > > >> > > > > >> >>> kick-off a
> > > >> > > > > >> >>> > > >> >> separate
> > > >> > > > > >> >>> > > >> >> > > > >>> dev-list
> > > >> > > > > >> >>> > > >> >> > > > >>> discussion to involve more
> > community
> > > >> > members
> > > >> > > > and
> > > >> > > > > >> >>> > referring
> > > >> > > > > >> >>> > > to
> > > >> > > > > >> >>> > > >> >> this
> > > >> > > > > >> >>> > > >> >> > > > >>> thread?
> > > >> > > > > >> >>> > > >> >> > > > >>> I think the primary question the
> > > >> community
> > > >> > > > needs
> > > >> > > > > >> to
> > > >> > > > > >> >>> > answer
> > > >> > > > > >> >>> > > >> >> whether
> > > >> > > > > >> >>> > > >> >> > we
> > > >> > > > > >> >>> > > >> >> > > > >>> should join the Bahir or kick-off
> > > >> "Ignite
> > > >> > > > > >> Extensions"
> > > >> > > > > >> >>> > > >> project?
> > > >> > > > > >> >>> > > >> >> > > > >>>
> > > >> > > > > >> >>> > > >> >> > > > >>> -
> > > >> > > > > >> >>> > > >> >> > > > >>> Denis
> > > >> > > > > >> >>> > > >> >> > > > >>>
> > > >> > > > > >> >>> > > >> >> > > > >>>
> > > >> > > > > >> >>> > > >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM
> > > Alexey
> > > >> > > > Zinoviev
> > > >> > > > > <
> > > >> > > > > >> >>> > > >> >> > > > zaleslaw.sin@gmail.com>
> > > >> > > > > >> >>> > > >> >> > > > >>> wrote:
> > > >> > > > > >> >>> > > >> >> > > > >>>
> > > >> > > > > >> >>> > > >> >> > > > >>> > Maybe we could move all our
> > > Streaming
> > > >> > > > > >> Integrations
> > > >> > > > > >> >>> > there,
> > > >> > > > > >> >>> > > >> but
> > > >> > > > > >> >>> > > >> >> > what
> > > >> > > > > >> >>> > > >> >> > > is
> > > >> > > > > >> >>> > > >> >> > > > >>> about
> > > >> > > > > >> >>> > > >> >> > > > >>> > maintaining and committer
> > > >> permissions to
> > > >> > > the
> > > >> > > > > new
> > > >> > > > > >> >>> > > >> repositories?
> > > >> > > > > >> >>> > > >> >> > > > >>> > I see the list of the
> committers
> > > and
> > > >> PMC
> > > >> > > > > members
> > > >> > > > > >> >>> there
> > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > >> > > https://bahir.apache.org/community-members/
> > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > >> > > > > >> >>> > > >> >> > > > >>> > Could somebody from Ignite
> > > community
> > > >> be
> > > >> > > > added
> > > >> > > > > to
> > > >> > > > > >> >>> this
> > > >> > > > > >> >>> > > list
> > > >> > > > > >> >>> > > >> as
> > > >> > > > > >> >>> > > >> >> a
> > > >> > > > > >> >>> > > >> >> > > > >>> > PMC/committer to maintain
> Ignite
> > > >> > > > integrations?
> > > >> > > > > >> >>> > > >> >> > > > >>> > If yes, I'd happy to join this
> > > >> project
> > > >> > and
> > > >> > > > > >> >>> collaborate
> > > >> > > > > >> >>> > > with
> > > >> > > > > >> >>> > > >> >> these
> > > >> > > > > >> >>> > > >> >> > > > guys
> > > >> > > > > >> >>> > > >> >> > > > >>> > together
> > > >> > > > > >> >>> > > >> >> > > > >>> > If no, and we should start from
> > the
> > > >> zero
> > > >> > > > level
> > > >> > > > > >> with
> > > >> > > > > >> >>> > > >> external
> > > >> > > > > >> >>> > > >> >> PRs
> > > >> > > > > >> >>> > > >> >> > -
> > > >> > > > > >> >>> > > >> >> > > > >>> hmmm,
> > > >> > > > > >> >>> > > >> >> > > > >>> > it's better to have our own
> > > external
> > > >> > > > > repository
> > > >> > > > > >> >>> with
> > > >> > > > > >> >>> > > >> >> ApacheBahirr
> > > >> > > > > >> >>> > > >> >> > > > >>> ideology.
> > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > >> > > > > >> >>> > > >> >> > > > >>> > Also, I agree, that all
> > connectors
> > > >> could
> > > >> > > be
> > > >> > > > > >> moved
> > > >> > > > > >> >>> there
> > > >> > > > > >> >>> > > (in
> > > >> > > > > >> >>> > > >> >> > > > >>> ApacheBahirr
> > > >> > > > > >> >>> > > >> >> > > > >>> > like repository), but not all
> > > modules
> > > >> > from
> > > >> > > > the
> > > >> > > > > >> page
> > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > >> > > > > >> >>> > > >> >> > > > >>>
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > >
> > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > >> >>> > > >> >>
> > > >> > > > > >> >>> > > >>
> > > >> > > > > >> >>> > >
> > > >> > > > > >> >>> >
> > > >> > > > > >> >>>
> > > >> > > > > >>
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > >> > > > > >> >>> > > >> >> > > > >>> > because
> > > >> > > > > >> >>> > > >> >> > > > >>> > they use different parts of
> core
> > > >> modules
> > > >> > > and
> > > >> > > > > >> could
> > > >> > > > > >> >>> be
> > > >> > > > > >> >>> > > >> >> developed
> > > >> > > > > >> >>> > > >> >> > > with
> > > >> > > > > >> >>> > > >> >> > > > >>> > different velocity.
> > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > >> > > > > >> >>> > > >> >> > > > >>> > In reality, before creation of
> > new
> > > >> > > > > repositories
> > > >> > > > > >> we
> > > >> > > > > >> >>> need
> > > >> > > > > >> >>> > > >> wide
> > > >> > > > > >> >>> > > >> >> > > > discussion
> > > >> > > > > >> >>> > > >> >> > > > >>> > based on the document mentioned
> > > >> above.
> > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > >> > > > > >> >>> > > >> >> > > > >>> > P.S. Of course, we could
> > experiment
> > > >> in
> > > >> > the
> > > >> > > > > next
> > > >> > > > > >> >>> release
> > > >> > > > > >> >>> > > 2.8
> > > >> > > > > >> >>> > > >> >> with
> > > >> > > > > >> >>> > > >> >> > > one
> > > >> > > > > >> >>> > > >> >> > > > >>> or two
> > > >> > > > > >> >>> > > >> >> > > > >>> > integrations like twitter or
> > ZeroMQ
> > > >> > > > > >> >>> > > >> >> > > > >>> > Also, I'd like Denis Magda idea
> > for
> > > >> the
> > > >> > > > > separate
> > > >> > > > > >> >>> > > >> repositories
> > > >> > > > > >> >>> > > >> >> in
> > > >> > > > > >> >>> > > >> >> > > > Apache
> > > >> > > > > >> >>> > > >> >> > > > >>> > Github to maintain them by
> > anybody.
> > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > >> > > > > >> >>> > > >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02,
> > Saikat
> > > >> > > Maitra <
> > > >> > > > > >> >>> > > >> >> > > saikat.maitra@gmail.com
> > > >> > > > > >> >>> > > >> >> > > > >:
> > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > Hi Denis, Emmanouil
> > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > Thank you for your email. I
> > think
> > > >> > > > creating a
> > > >> > > > > >> >>> separate
> > > >> > > > > >> >>> > > >> >> > integration
> > > >> > > > > >> >>> > > >> >> > > > >>> project
> > > >> > > > > >> >>> > > >> >> > > > >>> > > in github and also proposing
> it
> > > as
> > > >> an
> > > >> > > > apache
> > > >> > > > > >> >>> > incubator
> > > >> > > > > >> >>> > > >> >> project
> > > >> > > > > >> >>> > > >> >> > > will
> > > >> > > > > >> >>> > > >> >> > > > >>> be a
> > > >> > > > > >> >>> > > >> >> > > > >>> > > good move. The other separate
> > > >> > > integration
> > > >> > > > > >> modules
> > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > >> > > > > >> >>> > > >> >> > > > >>>
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > >
> > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > >> >>> > > >> >>
> > > >> > > > > >> >>> > > >>
> > > >> > > > > >> >>> > >
> > > >> > > > > >> >>> >
> > > >> > > > > >> >>>
> > > >> > > > > >>
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > >> > > > > >> >>> > > >> >> > > > >>> > > can
> > > >> > > > > >> >>> > > >> >> > > > >>> > > be moved to this new apache
> > > >> incubator
> > > >> > > > > project.
> > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > There are similar integration
> > > >> > available
> > > >> > > > for
> > > >> > > > > >> >>> Flink and
> > > >> > > > > >> >>> > > >> Spark
> > > >> > > > > >> >>> > > >> >> in
> > > >> > > > > >> >>> > > >> >> > > > Apache
> > > >> > > > > >> >>> > > >> >> > > > >>> > Bahir
> > > >> > > > > >> >>> > > >> >> > > > >>> > > https://bahir.apache.org/
> > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > Do you think we should reach
> > out
> > > to
> > > >> > > Apache
> > > >> > > > > >> Bahir
> > > >> > > > > >> >>> > > >> community
> > > >> > > > > >> >>> > > >> >> > with a
> > > >> > > > > >> >>> > > >> >> > > > >>> > proposal
> > > >> > > > > >> >>> > > >> >> > > > >>> > > or we should plan to create a
> > > >> separate
> > > >> > > > > Apache
> > > >> > > > > >> >>> > Incubator
> > > >> > > > > >> >>> > > >> >> > project?
> > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > Regards,
> > > >> > > > > >> >>> > > >> >> > > > >>> > > Saikat
> > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21
> AM
> > > >> > > Emmanouil
> > > >> > > > > >> >>> > Gkatziouras <
> > > >> > > > > >> >>> > > >> >> > > > >>> > > gkatzioura@gmail.com>
> > > >> > > > > >> >>> > > >> >> > > > >>> > > wrote:
> > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > Hi all!
> > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > Based on the discussions
> most
> > > >> > probably
> > > >> > > > > this
> > > >> > > > > >> is
> > > >> > > > > >> >>> > going
> > > >> > > > > >> >>> > > to
> > > >> > > > > >> >>> > > >> >> be on
> > > >> > > > > >> >>> > > >> >> > > > >>> another
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > GitHub repo. Therefore I
> will
> > > >> > prepare
> > > >> > > a
> > > >> > > > > >> >>> standalone
> > > >> > > > > >> >>> > > >> project
> > > >> > > > > >> >>> > > >> >> > with
> > > >> > > > > >> >>> > > >> >> > > > the
> > > >> > > > > >> >>> > > >> >> > > > >>> > > Pub/Sub
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > module and once the
> > repository
> > > is
> > > >> > > > created
> > > >> > > > > a
> > > >> > > > > >> >>> pull
> > > >> > > > > >> >>> > > >> request
> > > >> > > > > >> >>> > > >> >> > shall
> > > >> > > > > >> >>> > > >> >> > > be
> > > >> > > > > >> >>> > > >> >> > > > >>> > issued.
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > If there is anything else I
> > can
> > > >> do
> > > >> > in
> > > >> > > > > order
> > > >> > > > > >> to
> > > >> > > > > >> >>> > > >> facilitate
> > > >> > > > > >> >>> > > >> >> > this
> > > >> > > > > >> >>> > > >> >> > > > >>> please
> > > >> > > > > >> >>> > > >> >> > > > >>> > let
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > me know.
> > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > If a ticket is created for
> > this
> > > >> > > issue, I
> > > >> > > > > >> shall
> > > >> > > > > >> >>> be
> > > >> > > > > >> >>> > > able
> > > >> > > > > >> >>> > > >> to
> > > >> > > > > >> >>> > > >> >> > > create
> > > >> > > > > >> >>> > > >> >> > > > a
> > > >> > > > > >> >>> > > >> >> > > > >>> pull
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > request and thus a build
> will
> > > >> take
> > > >> > > place
> > > >> > > > > on
> > > >> > > > > >> >>> > ignite's
> > > >> > > > > >> >>> > > >> CI.
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > If everyone is aligned with
> > > >> adding a
> > > >> > > > > Pub/Sub
> > > >> > > > > >> >>> > > >> >> implementation
> > > >> > > > > >> >>> > > >> >> > > may I
> > > >> > > > > >> >>> > > >> >> > > > >>> > create
> > > >> > > > > >> >>> > > >> >> > > > >>> > > a
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > ticket on JIRA?
> > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > My next question has to do
> on
> > > how
> > > >> > > Ignite
> > > >> > > > > >> >>> configures
> > > >> > > > > >> >>> > > the
> > > >> > > > > >> >>> > > >> >> build
> > > >> > > > > >> >>> > > >> >> > > > jobs
> > > >> > > > > >> >>> > > >> >> > > > >>> in
> > > >> > > > > >> >>> > > >> >> > > > >>> > > Team
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > City. The unit tests for my
> > > >> > > > implementation
> > > >> > > > > >> >>> need a
> > > >> > > > > >> >>> > > local
> > > >> > > > > >> >>> > > >> >> > Pub/Sub
> > > >> > > > > >> >>> > > >> >> > > > >>> server
> > > >> > > > > >> >>> > > >> >> > > > >>> > up
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > and running (they provide
> one
> > > for
> > > >> > > > testing
> > > >> > > > > >> >>> > purposes).
> > > >> > > > > >> >>> > > >> Will
> > > >> > > > > >> >>> > > >> >> > this
> > > >> > > > > >> >>> > > >> >> > > > take
> > > >> > > > > >> >>> > > >> >> > > > >>> > > effect
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > in a form of a build script
> > > >> inside
> > > >> > my
> > > >> > > > > >> >>> > implementation
> > > >> > > > > >> >>> > > or
> > > >> > > > > >> >>> > > >> >> is it
> > > >> > > > > >> >>> > > >> >> > > > >>> something
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > that should be configured
> on
> > > Team
> > > >> > > City?
> > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > Kind regards,
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > Emmanouil
> > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > *Emmanouil Gkatziouras*
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > https://egkatzioura.com/ |
> > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > >> > > > > >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > https://github.com/gkatzioura
> > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > On Tue, 15 Oct 2019 at
> 23:27,
> > > >> Denis
> > > >> > > > Magda
> > > >> > > > > <
> > > >> > > > > >> >>> > > >> >> dmagda@apache.org
> > > >> > > > > >> >>> > > >> >> > >
> > > >> > > > > >> >>> > > >> >> > > > >>> wrote:
> > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > Emmanouil, Saikat,
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > After contemplating on
> > this,
> > > >> let
> > > >> > me
> > > >> > > > > >> propose
> > > >> > > > > >> >>> > another
> > > >> > > > > >> >>> > > >> way
> > > >> > > > > >> >>> > > >> >> > with
> > > >> > > > > >> >>> > > >> >> > > > this
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > particular integration
> that
> > > is
> > > >> > > aligned
> > > >> > > > > >> with
> > > >> > > > > >> >>> our
> > > >> > > > > >> >>> > > >> >> > > modularization
> > > >> > > > > >> >>> > > >> >> > > > >>> > > endeavor.
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > The modularization [1]
> > > defines
> > > >> the
> > > >> > > > > Ignite
> > > >> > > > > >> >>> core,
> > > >> > > > > >> >>> > > >> Ignite
> > > >> > > > > >> >>> > > >> >> > > modules,
> > > >> > > > > >> >>> > > >> >> > > > >>> and
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > independent integrations.
> > The
> > > >> > first
> > > >> > > > two
> > > >> > > > > >> are
> > > >> > > > > >> >>> to be
> > > >> > > > > >> >>> > > >> >> located
> > > >> > > > > >> >>> > > >> >> > in
> > > >> > > > > >> >>> > > >> >> > > > >>> Ignite
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > repositories and to be
> > > >> > > tested/updated
> > > >> > > > by
> > > >> > > > > >> the
> > > >> > > > > >> >>> > > >> community
> > > >> > > > > >> >>> > > >> >> for
> > > >> > > > > >> >>> > > >> >> > > > every
> > > >> > > > > >> >>> > > >> >> > > > >>> > > release.
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > The modules are a subset
> of
> > > >> > existing
> > > >> > > > > >> >>> integrations
> > > >> > > > > >> >>> > > >> that
> > > >> > > > > >> >>> > > >> >> are
> > > >> > > > > >> >>> > > >> >> > > > >>> adopted
> > > >> > > > > >> >>> > > >> >> > > > >>> > > widely
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > and require community
> > > >> attention.
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > As for the independent
> > > >> > integrations
> > > >> > > > [1],
> > > >> > > > > >> >>> those
> > > >> > > > > >> >>> > are
> > > >> > > > > >> >>> > > >> >> supposed
> > > >> > > > > >> >>> > > >> >> > > to
> > > >> > > > > >> >>> > > >> >> > > > be
> > > >> > > > > >> >>> > > >> >> > > > >>> > > located
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > in a separate GitHub
> repo.
> > I
> > > >> would
> > > >> > > > > advise
> > > >> > > > > >> us
> > > >> > > > > >> >>> to
> > > >> > > > > >> >>> > > start
> > > >> > > > > >> >>> > > >> >> > > > >>> implementing
> > > >> > > > > >> >>> > > >> >> > > > >>> > this
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > concept with the Pub/Sub
> > > >> > > integration.
> > > >> > > > > Even
> > > >> > > > > >> >>> though
> > > >> > > > > >> >>> > > it
> > > >> > > > > >> >>> > > >> >> will
> > > >> > > > > >> >>> > > >> >> > be
> > > >> > > > > >> >>> > > >> >> > > > >>> located
> > > >> > > > > >> >>> > > >> >> > > > >>> > > in a
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > separate Github repo, it
> > will
> > > >> be
> > > >> > > > listed
> > > >> > > > > >> among
> > > >> > > > > >> >>> > > >> officially
> > > >> > > > > >> >>> > > >> >> > > > >>> available
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > integrations on the
> Ignite
> > > >> website
> > > >> > > > > >> featuring
> > > >> > > > > >> >>> > > >> Emmanouil
> > > >> > > > > >> >>> > > >> >> as a
> > > >> > > > > >> >>> > > >> >> > > > >>> > > contributor.
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > What do you think?
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > [1]
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > >
> > > >> > > > > >> >>> > > >> >> > > > >>> >
> > > >> > > > > >> >>> > > >> >> > > > >>>
> > > >> > > > > >> >>> > > >> >> > > >
> > > >> > > > > >> >>> > > >> >> > >
> > > >> > > > > >> >>> > > >> >> >
> > > >> > > > > >> >>> > > >> >>
> > > >> > > > > >> >>> > > >>
> > > >> > > > > >> >>> > >
> > > >> > > > > >> >>> >
> > > >> > > > > >> >>>
> > > >> > > > > >>
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > -
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > Denis
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at
> > 9:47
> > > PM
> > > >> > > Saikat
> > > >> > > > > >> >>> Maitra <
> > > >> > > > > >> >>> > > >> >> > > > >>> > saikat.maitra@gmail.com
> > > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > wrote:
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > Hi Emmanouil,
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > The changes looks good
> to
> > > >> me. I
> > > >> > > > wanted
> > > >> > > > > >> to
> > > >> > > > > >> >>> check
> > > >> > > > > >> >>> > > if
> > > >> > > > > >> >>> > > >> you
> > > >> > > > > >> >>> > > >> >> > are
> > > >> > > > > >> >>> > > >> >> > > > also
> > > >> > > > > >> >>> > > >> >> > > > >>> > able
> > > >> > > > > >> >>> > > >> >> > > > >>> > > to
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > validate that the once
> > data
> > > >> is
> > > >> > > added
> > > >> > > > > to
> > > >> > > > > >> >>> Ignite
> > > >> > > > > >> >>> > > >> cluster
> > > >> > > > > >> >>> > > >> >> > you
> > > >> > > > > >> >>> > > >> >> > > > are
> > > >> > > > > >> >>> > > >> >> > > > >>> also
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > able
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > to
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > access it using another
> > > >> Ignite
> > > >> > > > client
> > > >> > > > > or
> > > >> > > > > >> >>> using
> > > >> > > > > >> >>> > > rest
> > > >> > > > > >> >>> > > >> >> > > > endpoints.
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > We use Teamcity for CI
> > > >> process,
> > > >> > > the
> > > >> > > > > >> >>> details are
> > > >> > > > > >> >>> > > as
> > > >> > > > > >> >>> > > >> >> > follows
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > Trigger validation of
> > those
> > > >> test
> > > >> > > > > suites
> > > >> > > > > >> >>> that
> > > >> > > > > >> >>> > have
> > > >> > > > > >> >>> > > >> been
> > > >> > > > > >> >>> > > >> >> > > > >>> affected by
> > > >> > > > > >> >>> > > >> >> > > > >>> > > your
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > changes on TeamCity <
> > > >> > > > > >> >>> > > http://ci.ignite.apache.org/>:
> > > >> > > > > >> >>> > > >> <
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > http://204.14.53.153/
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Locate a test
> suite
> > > you
> > > >> > have
> > > >> > > to
> > > >> > > > > >> check,
> > > >> > > > > >> >>> > press
> > > >> > > > > >> >>> > > >> >> button
> > > >> > > > > >> >>> > > >> >> > > > named
> > > >> > > > > >> >>> > > >> >> > > > >>> > "..."
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > that
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    is located on the
> left
> > > of
> > > >> > "Run"
> > > >> > > > > >> button.
> > > >> > > > > >> >>> "Run
> > > >> > > > > >> >>> > > >> custom
> > > >> > > > > >> >>> > > >> >> > > build"
> > > >> > > > > >> >>> > > >> >> > > > >>> > dialog
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > will
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    appear;
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Go to "Changes"
> tab
> > > and
> > > >> > > choose
> > > >> > > > > >> >>> > > >> >> > > > >>> >
> "pull/<pull-request-number>/head"
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > in
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    "Build branch"
> > dropdown
> > > >> list;
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Press "Run build"
> > > button
> > > >> > and
> > > >> > > > > >> monitor
> > > >> > > > > >> >>> tests
> > > >> > > > > >> >>> > > >> >> results.
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > @Roman
> rshtykh@yahoo.com
> > -
> > > >> can
> > > >> > > you
> > > >> > > > > also
> > > >> > > > > >> >>> please
> > > >> > > > > >> >>> > > >> take a
> > > >> > > > > >> >>> > > >> >> > > look?
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > Regards,
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > Saikat
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at
> > > 6:20
> > > >> PM
> > > >> > > > > >> Emmanouil
> > > >> > > > > >> >>> > > >> Gkatziouras
> > > >> > > > > >> >>> > > >> >> <
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > gkatzioura@gmail.com>
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > wrote:
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Hi @Saikat Maitra
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > I have my
> > implementation
> > > >> on an
> > > >> > > > > Ignite
> > > >> > > > > >> >>> fork on
> > > >> > > > > >> >>> > > my
> > > >> > > > > >> >>> > > >> >> > personal
> > > >> > > > > >> >>> > > >> >> > > > >>> GitHub
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > account
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > [1]
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > It is based on the
> > Kafka
> > > >> > > > > >> implementation
> > > >> > > > > >> >>> which
> > > >> > > > > >> >>> > > is
> > > >> > > > > >> >>> > > >> of
> > > >> > > > > >> >>> > > >> >> a
> > > >> > > > > >> >>> > > >> >> > > > polling
> > > >> > > > > >> >>> > > >> >> > > > >>> > > nature.
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > GCP does not provide
> a
> > > >> service
> > > >> > > > like
> > > >> > > > > >> >>> Kafka.
> > > >> > > > > >> >>> > > >> Instead
> > > >> > > > > >> >>> > > >> >> it
> > > >> > > > > >> >>> > > >> >> > > > >>> provides us
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > with
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Pub/Sub which is just
> > > like
> > > >> > > Kafka.
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Therefore using the
> > Kafka
> > > >> > > streamer
> > > >> > > > > as
> > > >> > > > > >> a
> > > >> > > > > >> >>> > > guideline
> > > >> > > > > >> >>> > > >> >> made
> > > >> > > > > >> >>> > > >> >> > > more
> > > >> > > > > >> >>> > > >> >> > > > >>> sense
> > > >> > > > > >> >>> > > >> >> > > > >>> > > to
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > me.
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Apart from the
> > > >> implementation
> > > >> > I
> > > >> > > > have
> > > >> > > > > >> also
> > > >> > > > > >> >>> > > added a
> > > >> > > > > >> >>> > > >> >> unit
> > > >> > > > > >> >>> > > >> >> > > > test.
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > It definitely needs
> > some
> > > >> > > polishing
> > > >> > > > > >> and a
> > > >> > > > > >> >>> > double
> > > >> > > > > >> >>> > > >> >> check
> > > >> > > > > >> >>> > > >> >> > on
> > > >> > > > > >> >>> > > >> >> > > my
> > > >> > > > > >> >>> > > >> >> > > > >>> side
> > > >> > > > > >> >>> > > >> >> > > > >>> > to
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > make
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > the review smooth.
> > > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > For the tests a local
> > > >> Pub/Sub
> > > >> > > [2]
> > > >> > > > > >> server
> > > >> > > >
> > > >
> > > >
> > >
> >
>

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra <sa...@gmail.com>]

Hi Emmanouil,

I looked into the build logs and I observed that since I added artifacts
dependencies to "Build apache Ignite"[1] it was able to pull the required
dependencies and was able to run the existing tests. It is however not
identifying new modules like I changed from flink to flink-ext or pub-sub.

[1]
https://ci.ignite.apache.org/admin/editDependencies.html?id=buildType:IgniteExtensions_Build

I will look into the issue and debug further.

Regards,
Saikat


On Sat, Nov 30, 2019 at 5:35 PM Emmanouil Gkatziouras <gk...@gmail.com>
wrote:

> Hi Saikat!
>
> From the logs[1] it seems that TC cannot find the project. However I did
> include the project on the parent pom (modules sections) [2]
> I tried to reproduce locally unfortunately I am not aware of the full
> context on TC. It seems as if a different parent pom is being picked up.
>
> Kind regards
>
> [1]
>
> https://ci.ignite.apache.org/viewLog.html?buildId=4804893&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=1704
> [2]
>
> https://github.com/gkatzioura/ignite-extensions/blob/d6a8beee7feff28f59d9a12be692016305564d25/pom.xml#L46
>
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>
>
> On Sat, 30 Nov 2019 at 22:20, Saikat Maitra <sa...@gmail.com>
> wrote:
>
> > Hello,
> >
> > I have made changes in Ignite Extensions build configurations.
> >
> > It is now running the tests as expected.
> >
> >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4804480&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> >
> > Regards,
> > Saikat
> >
> > On Thu, Nov 28, 2019 at 1:29 PM Saikat Maitra <sa...@gmail.com>
> > wrote:
> >
> > > Hi,
> > >
> > > We do run build with -DskipTests and then configure selective test
> suites
> > > to run.
> > >
> > > I will check why these test suites are not getting executed.
> > >
> > > Regards
> > > Saikat
> > >
> > > On Wed, 27 Nov 2019 at 3:05 PM, Emmanouil Gkatziouras <
> > > gkatzioura@gmail.com> wrote:
> > >
> > >> Hi all,
> > >>
> > >> I did add a test suite and run the build with the corresponding
> > arguments,
> > >> however the tests did not run [1].
> > >> I checked the Flink's build logs and the same message is displayed
> there
> > >> too `No tests to run` [2].
> > >>
> > >> [1]
> > >>
> > >>
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4798203&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=248
> > >> [2]
> > >>
> > >>
> >
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4789199&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=1256
> > >>
> > >> Kind regards
> > >> *Emmanouil Gkatziouras*
> > >> https://egkatzioura.com/ |
> > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> https://github.com/gkatzioura
> > >>
> > >>
> > >> On Wed, 27 Nov 2019 at 18:58, Saikat Maitra <sa...@gmail.com>
> > >> wrote:
> > >>
> > >> > Hi,
> > >> >
> > >> > We will need to add test suites for tests to be executed in build.
> We
> > >> pass
> > >> > them as TestSuites param. I had added Flink sink and source
> > testsuites.
> > >> >
> > >> > Can you please review and confirm.
> > >> >
> > >> > Regards
> > >> > Saikat
> > >> >
> > >> > On Wed, 27 Nov 2019 at 6:00 AM, Emmanouil Gkatziouras <
> > >> > gkatzioura@gmail.com>
> > >> > wrote:
> > >> >
> > >> > > Hi all!
> > >> > >
> > >> > > I was successful building and running my pull requests. It seems
> > that
> > >> the
> > >> > > tests do not run both for Flink and Pub/Sub [1]
> > >> > > If there is something I cannot do to make them discoverable please
> > >> let me
> > >> > > know.
> > >> > >
> > >> > > [1]
> > >> > >
> > >> > >
> > >> >
> > >>
> >
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4796722&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=283
> > >> > >
> > >> > > *Emmanouil Gkatziouras*
> > >> > > https://egkatzioura.com/ |
> > >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> > > https://github.com/gkatzioura
> > >> > >
> > >> > >
> > >> > > On Wed, 27 Nov 2019 at 04:57, Saikat Maitra <
> > saikat.maitra@gmail.com>
> > >> > > wrote:
> > >> > >
> > >> > > > Hi Emmanouil,
> > >> > > >
> > >> > > > I have added you as contributor in Ignite Extensions project.
> Can
> > >> you
> > >> > > > please check and confirm if you are able to see the project and
> > >> execute
> > >> > > > build on your pull request.
> > >> > > >
> > >> > > > Regards,
> > >> > > > Saikat
> > >> > > >
> > >> > > > On Tue, Nov 26, 2019 at 12:49 PM Emmanouil Gkatziouras <
> > >> > > > gkatzioura@gmail.com>
> > >> > > > wrote:
> > >> > > >
> > >> > > > > Hi all!
> > >> > > > >
> > >> > > > > If someone could give me read access to the Job of Ignite
> > >> Extensions
> > >> > on
> > >> > > > > Team City will greatly help me (username gkatzioura).
> > >> > > > > I suppose the builds get triggered automatically on a pull
> > >> request.
> > >> > > > >
> > >> > > > > Kind regards,
> > >> > > > > Emmanouil
> > >> > > > >
> > >> > > > > *Emmanouil Gkatziouras*
> > >> > > > > https://egkatzioura.com/ |
> > >> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> > > > > https://github.com/gkatzioura
> > >> > > > >
> > >> > > > >
> > >> > > > > On Sun, 24 Nov 2019 at 21:11, Emmanouil Gkatziouras <
> > >> > > > gkatzioura@gmail.com>
> > >> > > > > wrote:
> > >> > > > >
> > >> > > > > > Hi Saikat!
> > >> > > > > >
> > >> > > > > > I just rebased with the flink branch. Unfortunately I do not
> > >> have
> > >> > > read
> > >> > > > > > access to the team city link you provided and thus evaluate
> > the
> > >> > tests
> > >> > > > for
> > >> > > > > > my pull request.
> > >> > > > > > I guess it has to do with ignite extensions being new.
> > >> > > > > > My username is gkatzioura and email the one I am using on
> this
> > >> > email
> > >> > > > > >
> > >> > > > > > Thank you for your time.
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > *Emmanouil Gkatziouras*
> > >> > > > > > https://egkatzioura.com/ |
> > >> > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> > > > > > https://github.com/gkatzioura
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > On Sun, 24 Nov 2019 at 18:03, Saikat Maitra <
> > >> > saikat.maitra@gmail.com
> > >> > > >
> > >> > > > > > wrote:
> > >> > > > > >
> > >> > > > > >> Hi Emmanouil,
> > >> > > > > >>
> > >> > > > > >> The latest build on teamcity has passed on Flink pull
> > request.
> > >> > > > > >>
> > >> > > > > >>
> > >> > > > > >>
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4788928&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> > >> > > > > >>
> > >> > > > > >> you should be able to take the changes I made in my PR and
> > run
> > >> > build
> > >> > > > on
> > >> > > > > >> your pull request.
> > >> > > > > >>
> > >> > > > > >> Please let me know if you have any questions.
> > >> > > > > >>
> > >> > > > > >> Regards,
> > >> > > > > >> Saikat
> > >> > > > > >>
> > >> > > > > >> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra <
> > >> > > > saikat.maitra@gmail.com
> > >> > > > > >
> > >> > > > > >> wrote:
> > >> > > > > >>
> > >> > > > > >> > Hi Emmanouil,
> > >> > > > > >> >
> > >> > > > > >> > I have fixed the Flink module testsuites and have setup
> new
> > >> > > project
> > >> > > > in
> > >> > > > > >> > teamcity.
> > >> > > > > >> >
> > >> > > > > >> >
> > >> > > > > >> >
> > >> > > > > >>
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview
> > >> > > > > >> >
> > >> > > > > >> > I am looking into setting up the build.
> > >> > > > > >> >
> > >> > > > > >> > Regards,
> > >> > > > > >> > Saikat
> > >> > > > > >> >
> > >> > > > > >> > On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra <
> > >> > > > > saikat.maitra@gmail.com>
> > >> > > > > >> > wrote:
> > >> > > > > >> >
> > >> > > > > >> >> Hi Emmanouil,
> > >> > > > > >> >>
> > >> > > > > >> >> Thank you for your email. Yes, the plan is once the
> Flink
> > >> PR is
> > >> > > > > merged
> > >> > > > > >> it
> > >> > > > > >> >> will provide the Licence, parent POMs etc and you can
> > rebase
> > >> > from
> > >> > > > > >> master
> > >> > > > > >> >> branch and apply the changes on top of it.
> > >> > > > > >> >>
> > >> > > > > >> >> I am getting some issues with test failures in local
> with
> > >> > > > > >> >> GridTestProperties as the test.properties is not present
> > in
> > >> > this
> > >> > > > new
> > >> > > > > >> >> project but should be available from dependencies.
> > >> > > > > >> >>
> > >> > > > > >> >> Once I address this issue, I will go ahead and merge the
> > >> > changes
> > >> > > > and
> > >> > > > > >> then
> > >> > > > > >> >> we can take it from there.
> > >> > > > > >> >>
> > >> > > > > >> >> Regards,
> > >> > > > > >> >> Saikat
> > >> > > > > >> >>
> > >> > > > > >> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras <
> > >> > > > > >> >> gkatzioura@gmail.com> wrote:
> > >> > > > > >> >>
> > >> > > > > >> >>> Hi all,
> > >> > > > > >> >>>
> > >> > > > > >> >>> I made my first pull request [1]. Since this project is
> > >> brand
> > >> > > new
> > >> > > > > (no
> > >> > > > > >> >>> parent poms, licensing), I reckoned it was best to use
> > >> > Saikat's
> > >> > > > > >> branch on
> > >> > > > > >> >>> Flink.
> > >> > > > > >> >>> I suppose the Flink branch will be merged first. If not
> > >> please
> > >> > > > give
> > >> > > > > me
> > >> > > > > >> >>> guidelines on how I should proceed next.
> > >> > > > > >> >>>
> > >> > > > > >> >>> Kind regards
> > >> > > > > >> >>> Emmanouil
> > >> > > > > >> >>>
> > >> > > > > >> >>> [1] https://github.com/apache/ignite-extensions/pull/2
> > >> > > > > >> >>>
> > >> > > > > >> >>> *Emmanouil Gkatziouras*
> > >> > > > > >> >>> https://egkatzioura.com/ |
> > >> > > > > >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> > > > > >> >>> https://github.com/gkatzioura
> > >> > > > > >> >>>
> > >> > > > > >> >>>
> > >> > > > > >> >>> On Fri, 22 Nov 2019 at 20:55, Denis Magda <
> > >> dmagda@apache.org>
> > >> > > > > wrote:
> > >> > > > > >> >>>
> > >> > > > > >> >>> > Awesome, ping us whenever you're ready!
> > >> > > > > >> >>> >
> > >> > > > > >> >>> > -
> > >> > > > > >> >>> > Denis
> > >> > > > > >> >>> >
> > >> > > > > >> >>> >
> > >> > > > > >> >>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil
> Gkatziouras
> > <
> > >> > > > > >> >>> > gkatzioura@gmail.com>
> > >> > > > > >> >>> > wrote:
> > >> > > > > >> >>> >
> > >> > > > > >> >>> > > Hi all!
> > >> > > > > >> >>> > >
> > >> > > > > >> >>> > > I am sorry for being late on that. I was trying to
> > >> > refactor
> > >> > > > the
> > >> > > > > >> test
> > >> > > > > >> >>> in
> > >> > > > > >> >>> > > order not to be in need of any external tools or
> > >> spinning
> > >> > > up a
> > >> > > > > >> >>> server.
> > >> > > > > >> >>> > > I did forked the new repo and indeed my changes
> > there,
> > >> so
> > >> > a
> > >> > > > pull
> > >> > > > > >> >>> request
> > >> > > > > >> >>> > is
> > >> > > > > >> >>> > > a matter of time!
> > >> > > > > >> >>> > >
> > >> > > > > >> >>> > > Kind regards
> > >> > > > > >> >>> > >
> > >> > > > > >> >>> > > *Emmanouil Gkatziouras*
> > >> > > > > >> >>> > > https://egkatzioura.com/ |
> > >> > > > > >> >>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> > > > > >> >>> > > https://github.com/gkatzioura
> > >> > > > > >> >>> > >
> > >> > > > > >> >>> > >
> > >> > > > > >> >>> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda <
> > >> > > dmagda@apache.org>
> > >> > > > > >> wrote:
> > >> > > > > >> >>> > >
> > >> > > > > >> >>> > > > Hi Emmanouil,
> > >> > > > > >> >>> > > >
> > >> > > > > >> >>> > > > Do you have any questions or need any support
> from
> > >> the
> > >> > > > > >> community?
> > >> > > > > >> >>> > > >
> > >> > > > > >> >>> > > > -
> > >> > > > > >> >>> > > > Denis
> > >> > > > > >> >>> > > >
> > >> > > > > >> >>> > > >
> > >> > > > > >> >>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <
> > >> > > > > >> >>> saikat.maitra@gmail.com
> > >> > > > > >> >>> > >
> > >> > > > > >> >>> > > > wrote:
> > >> > > > > >> >>> > > >
> > >> > > > > >> >>> > > >> Hi Emmanouil,
> > >> > > > > >> >>> > > >>
> > >> > > > > >> >>> > > >> Can you please take a looks at dev utils, if
> this
> > is
> > >> > > > > something
> > >> > > > > >> >>> you are
> > >> > > > > >> >>> > > >> looking for
> > >> > > > > >> >>> > > >>
> > >> > > > >
> https://github.com/apache/ignite/tree/master/modules/dev-utils
> > >> > > > > >> ?
> > >> > > > > >> >>> > > >>
> > >> > > > > >> >>> > > >> IMO, if you can release Pub/Sub server in maven
> > and
> > >> > then
> > >> > > > use
> > >> > > > > >> it as
> > >> > > > > >> >>> > > >> dependency, that would be great.
> > >> > > > > >> >>> > > >>
> > >> > > > > >> >>> > > >> Regards,
> > >> > > > > >> >>> > > >> Saikat
> > >> > > > > >> >>> > > >>
> > >> > > > > >> >>> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <
> > >> > > > > >> >>> > saikat.maitra@gmail.com>
> > >> > > > > >> >>> > > >> wrote:
> > >> > > > > >> >>> > > >>
> > >> > > > > >> >>> > > >> > Hi Emmanouil,
> > >> > > > > >> >>> > > >> >
> > >> > > > > >> >>> > > >> > The master branch has been initialized. I have
> > >> > raised a
> > >> > > > PR
> > >> > > > > to
> > >> > > > > >> >>> > migrate
> > >> > > > > >> >>> > > >> > Flink module to ignite-extensions.
> > >> > > > > >> >>> > > >> >
> > >> > > > > >> >>> > > >> > PR
> > >> > https://github.com/apache/ignite-extensions/pull/1
> > >> > > > > >> >>> > > >> > Jira :
> > >> > > > https://issues.apache.org/jira/browse/IGNITE-12356
> > >> > > > > >> >>> > > >> >
> > >> > > > > >> >>> > > >> > Regards,
> > >> > > > > >> >>> > > >> > Saikat
> > >> > > > > >> >>> > > >> >
> > >> > > > > >> >>> > > >> >
> > >> > > > > >> >>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil
> > >> > Gkatziouras <
> > >> > > > > >> >>> > > >> > gkatzioura@gmail.com> wrote:
> > >> > > > > >> >>> > > >> >
> > >> > > > > >> >>> > > >> >> Hi all!
> > >> > > > > >> >>> > > >> >>
> > >> > > > > >> >>> > > >> >> This is an update. The integration is tested
> > with
> > >> > > mocks
> > >> > > > > >> based
> > >> > > > > >> >>> on
> > >> > > > > >> >>> > the
> > >> > > > > >> >>> > > >> >> original Ignite repo [1].
> > >> > > > > >> >>> > > >> >> I am currently waiting for the first commits
> on
> > >> the
> > >> > > new
> > >> > > > > >> >>> integration
> > >> > > > > >> >>> > > >> >> project
> > >> > > > > >> >>> > > >> >> to take place in order to create my pull
> > request
> > >> > > > > accordingly
> > >> > > > > >> >>> to the
> > >> > > > > >> >>> > > >> >> initial
> > >> > > > > >> >>> > > >> >> commits.
> > >> > > > > >> >>> > > >> >> In order to simulate the Pub/Sub server in a
> > >> test I
> > >> > > > used a
> > >> > > > > >> test
> > >> > > > > >> >>> > > server
> > >> > > > > >> >>> > > >> >> implementation [2]. Unfortunately they don't
> > have
> > >> > this
> > >> > > > > util
> > >> > > > > >> >>> > uploaded
> > >> > > > > >> >>> > > >> in a
> > >> > > > > >> >>> > > >> >> maven repo, thus it needs to be hosted on a
> > maven
> > >> > repo
> > >> > > > > >> (Apache
> > >> > > > > >> >>> > > License
> > >> > > > > >> >>> > > >> >> 2.0). Is there a repo for utilities that
> Ignite
> > >> uses
> > >> > > for
> > >> > > > > the
> > >> > > > > >> >>> > > >> test/build?
> > >> > > > > >> >>> > > >> >> I have also created a ticket [3]
> > >> > > > > >> >>> > > >> >>
> > >> > > > > >> >>> > > >> >> Kind regards
> > >> > > > > >> >>> > > >> >>
> > >> > > > > >> >>> > > >> >> [1]
> > >> > > > > >> >>> >
> > >> > > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > >> > > > > >> >>> > > >> >> [2]
> > >> > > > > >> >>>
> > >> https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> > >> > > > > >> >>> > > >> >> [3]
> > >> > > https://issues.apache.org/jira/browse/IGNITE-12262
> > >> > > > > >> >>> > > >> >>
> > >> > > > > >> >>> > > >> >> *Emmanouil Gkatziouras*
> > >> > > > > >> >>> > > >> >> https://egkatzioura.com/ |
> > >> > > > > >> >>> > > >> >>
> > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> > > > > >> >>> > > >> >> https://github.com/gkatzioura
> > >> > > > > >> >>> > > >> >>
> > >> > > > > >> >>> > > >> >>
> > >> > > > > >> >>> > > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <
> > >> > > > > >> >>> > saikat.maitra@gmail.com
> > >> > > > > >> >>> > > >
> > >> > > > > >> >>> > > >> >> wrote:
> > >> > > > > >> >>> > > >> >>
> > >> > > > > >> >>> > > >> >> > Hello Ilya,
> > >> > > > > >> >>> > > >> >> >
> > >> > > > > >> >>> > > >> >> > Thank you for your email. Yes, before we
> > remove
> > >> > any
> > >> > > > > >> >>> integration
> > >> > > > > >> >>> > we
> > >> > > > > >> >>> > > >> will
> > >> > > > > >> >>> > > >> >> > move them to another repository and
> release.
> > >> > > > > >> >>> > > >> >> >
> > >> > > > > >> >>> > > >> >> > We are thinking we will start with Pub/Sub
> > >> > Streamer
> > >> > > > > >> >>> > implementation
> > >> > > > > >> >>> > > >> and
> > >> > > > > >> >>> > > >> >> > continue to move other integration as
> > >> mentioned in
> > >> > > the
> > >> > > > > >> >>> following
> > >> > > > > >> >>> > > doc.
> > >> > > > > >> >>> > > >> >> >
> > >> > > > > >> >>> > > >> >> >
> > >> > > > > >> >>> > > >> >> >
> > >> > > > > >> >>> > > >> >>
> > >> > > > > >> >>> > > >>
> > >> > > > > >> >>> > >
> > >> > > > > >> >>> >
> > >> > > > > >> >>>
> > >> > > > > >>
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > >> > > > > >> >>> > > >> >> >
> > >> > > > > >> >>> > > >> >> > Regards,
> > >> > > > > >> >>> > > >> >> > Saikat
> > >> > > > > >> >>> > > >> >> >
> > >> > > > > >> >>> > > >> >> >
> > >> > > > > >> >>> > > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya
> > >> Kasnacheev <
> > >> > > > > >> >>> > > >> >> > ilya.kasnacheev@gmail.com>
> > >> > > > > >> >>> > > >> >> > wrote:
> > >> > > > > >> >>> > > >> >> >
> > >> > > > > >> >>> > > >> >> > > Hello!
> > >> > > > > >> >>> > > >> >> > >
> > >> > > > > >> >>> > > >> >> > > My take on this: we want to offload some
> of
> > >> > > > > >> integrations,
> > >> > > > > >> >>> but
> > >> > > > > >> >>> > > >> please
> > >> > > > > >> >>> > > >> >> > commit
> > >> > > > > >> >>> > > >> >> > > and release them to another project
> first,
> > >> and
> > >> > > only
> > >> > > > > >> then we
> > >> > > > > >> >>> > will
> > >> > > > > >> >>> > > >> >> decide
> > >> > > > > >> >>> > > >> >> > to
> > >> > > > > >> >>> > > >> >> > > remove them from our repository.
> > >> > > > > >> >>> > > >> >> > >
> > >> > > > > >> >>> > > >> >> > > Please also make sure they are available
> > with
> > >> > same
> > >> > > > > >> artifact
> > >> > > > > >> >>> > name
> > >> > > > > >> >>> > > >> and
> > >> > > > > >> >>> > > >> >> API.
> > >> > > > > >> >>> > > >> >> > > If this has to be changed, let's wait for
> > AI
> > >> 3.0
> > >> > > > > >> >>> > > >> >> > >
> > >> > > > > >> >>> > > >> >> > > Regards,
> > >> > > > > >> >>> > > >> >> > > --
> > >> > > > > >> >>> > > >> >> > > Ilya Kasnacheev
> > >> > > > > >> >>> > > >> >> > >
> > >> > > > > >> >>> > > >> >> > >
> > >> > > > > >> >>> > > >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat
> Maitra
> > <
> > >> > > > > >> >>> > > >> saikat.maitra@gmail.com>:
> > >> > > > > >> >>> > > >> >> > >
> > >> > > > > >> >>> > > >> >> > > > Hello Emmanouil,
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > > > As discussed earlier, I discussed with
> > >> Apache
> > >> > > > Bahir
> > >> > > > > >> >>> community
> > >> > > > > >> >>> > > and
> > >> > > > > >> >>> > > >> >> they
> > >> > > > > >> >>> > > >> >> > > are
> > >> > > > > >> >>> > > >> >> > > > interested to have Apache Ignite
> > >> extensions as
> > >> > > > part
> > >> > > > > of
> > >> > > > > >> >>> Apache
> > >> > > > > >> >>> > > >> Bahir
> > >> > > > > >> >>> > > >> >> > > > project.
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > > > Can you please share Pub/Sub streamer
> > >> > > > implementation
> > >> > > > > >> >>> details
> > >> > > > > >> >>> > > with
> > >> > > > > >> >>> > > >> >> > Apache
> > >> > > > > >> >>> > > >> >> > > > Bahir community and request for
> feedback.
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > > > https://bahir.apache.org/
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > > > I have also requested for contributor
> > >> access
> > >> > in
> > >> > > > Jira
> > >> > > > > >> for
> > >> > > > > >> >>> > Apache
> > >> > > > > >> >>> > > >> >> Bahir
> > >> > > > > >> >>> > > >> >> > > > project so that I can create issues and
> > >> assign
> > >> > > to
> > >> > > > > >> myself.
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > > > I can help with code reviews as well.
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > > > Here is the email thread for reference
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > >
> > >> > > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > > > Regards,
> > >> > > > > >> >>> > > >> >> > > > Saikat
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat
> > >> Maitra
> > >> > <
> > >> > > > > >> >>> > > >> >> saikat.maitra@gmail.com
> > >> > > > > >> >>> > > >> >> > >
> > >> > > > > >> >>> > > >> >> > > > wrote:
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > > > > Hello,
> > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > >> >>> > > >> >> > > > > As discussed I have created following
> > >> > > discussion
> > >> > > > > >> >>> threads on
> > >> > > > > >> >>> > > our
> > >> > > > > >> >>> > > >> >> > > migration
> > >> > > > > >> >>> > > >> >> > > > > proposals for Apache Ignite
> extensions:
> > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > >
> > >> > > > > >> >>> > > >> >> >
> > >> > > > > >> >>> > > >> >>
> > >> > > > > >> >>> > > >>
> > >> > > > > >> >>> > >
> > >> > > > > >> >>> >
> > >> > > > > >> >>>
> > >> > > > > >>
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > >> >>> > > >>
> > >> > > > > >>
> > >> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > >> >>> > > >> >> > > > > I will share update as I hear more
> > >> > > information.
> > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > >> >>> > > >> >> > > > > Regards,
> > >> > > > > >> >>> > > >> >> > > > > Saikat
> > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > >> >>> > > >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM
> Saikat
> > >> > Maitra
> > >> > > <
> > >> > > > > >> >>> > > >> >> > saikat.maitra@gmail.com
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > > > > wrote:
> > >> > > > > >> >>> > > >> >> > > > >
> > >> > > > > >> >>> > > >> >> > > > >> Hello Denis,
> > >> > > > > >> >>> > > >> >> > > > >>
> > >> > > > > >> >>> > > >> >> > > > >> Sure, sounds good. I will create a
> > >> separate
> > >> > > > > >> discussion
> > >> > > > > >> >>> > > thread
> > >> > > > > >> >>> > > >> to
> > >> > > > > >> >>> > > >> >> get
> > >> > > > > >> >>> > > >> >> > > > >> community feedback on whether we
> > >> > > > > >> >>> > > >> >> > > > >> should join the Bahir or kick-off
> > >> "Ignite
> > >> > > > > >> Extensions"
> > >> > > > > >> >>> > > project.
> > >> > > > > >> >>> > > >> >> > > > >>
> > >> > > > > >> >>> > > >> >> > > > >> Regards,
> > >> > > > > >> >>> > > >> >> > > > >> Saikat
> > >> > > > > >> >>> > > >> >> > > > >>
> > >> > > > > >> >>> > > >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM
> Denis
> > >> > Magda <
> > >> > > > > >> >>> > > >> dmagda@apache.org>
> > >> > > > > >> >>> > > >> >> > > wrote:
> > >> > > > > >> >>> > > >> >> > > > >>
> > >> > > > > >> >>> > > >> >> > > > >>> Folks,
> > >> > > > > >> >>> > > >> >> > > > >>>
> > >> > > > > >> >>> > > >> >> > > > >>> The concept of Apache Bahir is
> > >> precisely
> > >> > > what
> > >> > > > we
> > >> > > > > >> >>> need!
> > >> > > > > >> >>> > > >> Saikat,
> > >> > > > > >> >>> > > >> >> > thanks
> > >> > > > > >> >>> > > >> >> > > > for
> > >> > > > > >> >>> > > >> >> > > > >>> doing the research. Why I believe
> > the *
> > >> > > idea*
> > >> > > > > >> fits us
> > >> > > > > >> >>> > well:
> > >> > > > > >> >>> > > >> >> > > > >>>
> > >> > > > > >> >>> > > >> >> > > > >>>    - All integrations can be stored
> > in
> > >> > > > separate
> > >> > > > > >> >>> Github
> > >> > > > > >> >>> > > >> >> repositories
> > >> > > > > >> >>> > > >> >> > > and
> > >> > > > > >> >>> > > >> >> > > > >>>    have their dev lifecycles. We've
> > not
> > >> > > > obliged
> > >> > > > > to
> > >> > > > > >> >>> couple
> > >> > > > > >> >>> > > all
> > >> > > > > >> >>> > > >> >> the
> > >> > > > > >> >>> > > >> >> > > > >>> integrations
> > >> > > > > >> >>> > > >> >> > > > >>>    in a single repo. For instance,
> > >> Spark
> > >> > can
> > >> > > > be
> > >> > > > > >> >>> located
> > >> > > > > >> >>> > in
> > >> > > > > >> >>> > > a
> > >> > > > > >> >>> > > >> >> > > dedicated
> > >> > > > > >> >>> > > >> >> > > > >>> repo
> > >> > > > > >> >>> > > >> >> > > > >>>    while streaming integrations
> might
> > >> be
> > >> > in
> > >> > > a
> > >> > > > > >> single
> > >> > > > > >> >>> one.
> > >> > > > > >> >>> > > >> It's
> > >> > > > > >> >>> > > >> >> up
> > >> > > > > >> >>> > > >> >> > to
> > >> > > > > >> >>> > > >> >> > > > us.
> > >> > > > > >> >>> > > >> >> > > > >>>    - All the repositories and
> > >> integrations
> > >> > > > > belong
> > >> > > > > >> to
> > >> > > > > >> >>> ASF.
> > >> > > > > >> >>> > > >> We're
> > >> > > > > >> >>> > > >> >> not
> > >> > > > > >> >>> > > >> >> > > > >>> dumping
> > >> > > > > >> >>> > > >> >> > > > >>>    them on Github but rather keep
> > >> > supporting
> > >> > > > and
> > >> > > > > >> >>> > developing
> > >> > > > > >> >>> > > >> in
> > >> > > > > >> >>> > > >> >> > > > >>> accordance with
> > >> > > > > >> >>> > > >> >> > > > >>>    ASF vision and practices.
> > >> > > > > >> >>> > > >> >> > > > >>>    - There is a way to reward
> > >> contributors
> > >> > > via
> > >> > > > > >> >>> > > committership
> > >> > > > > >> >>> > > >> and
> > >> > > > > >> >>> > > >> >> > PMC
> > >> > > > > >> >>> > > >> >> > > > >>>    membership. You won't get this
> if
> > a
> > >> > > project
> > >> > > > > is
> > >> > > > > >> >>> just
> > >> > > > > >> >>> > one
> > >> > > > > >> >>> > > of
> > >> > > > > >> >>> > > >> >> the
> > >> > > > > >> >>> > > >> >> > > > >>> millions of
> > >> > > > > >> >>> > > >> >> > > > >>>    Github projects.
> > >> > > > > >> >>> > > >> >> > > > >>>
> > >> > > > > >> >>> > > >> >> > > > >>> Saikat, as Ignite PMC member, could
> > you
> > >> > > please
> > >> > > > > >> >>> kick-off a
> > >> > > > > >> >>> > > >> >> separate
> > >> > > > > >> >>> > > >> >> > > > >>> dev-list
> > >> > > > > >> >>> > > >> >> > > > >>> discussion to involve more
> community
> > >> > members
> > >> > > > and
> > >> > > > > >> >>> > referring
> > >> > > > > >> >>> > > to
> > >> > > > > >> >>> > > >> >> this
> > >> > > > > >> >>> > > >> >> > > > >>> thread?
> > >> > > > > >> >>> > > >> >> > > > >>> I think the primary question the
> > >> community
> > >> > > > needs
> > >> > > > > >> to
> > >> > > > > >> >>> > answer
> > >> > > > > >> >>> > > >> >> whether
> > >> > > > > >> >>> > > >> >> > we
> > >> > > > > >> >>> > > >> >> > > > >>> should join the Bahir or kick-off
> > >> "Ignite
> > >> > > > > >> Extensions"
> > >> > > > > >> >>> > > >> project?
> > >> > > > > >> >>> > > >> >> > > > >>>
> > >> > > > > >> >>> > > >> >> > > > >>> -
> > >> > > > > >> >>> > > >> >> > > > >>> Denis
> > >> > > > > >> >>> > > >> >> > > > >>>
> > >> > > > > >> >>> > > >> >> > > > >>>
> > >> > > > > >> >>> > > >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM
> > Alexey
> > >> > > > Zinoviev
> > >> > > > > <
> > >> > > > > >> >>> > > >> >> > > > zaleslaw.sin@gmail.com>
> > >> > > > > >> >>> > > >> >> > > > >>> wrote:
> > >> > > > > >> >>> > > >> >> > > > >>>
> > >> > > > > >> >>> > > >> >> > > > >>> > Maybe we could move all our
> > Streaming
> > >> > > > > >> Integrations
> > >> > > > > >> >>> > there,
> > >> > > > > >> >>> > > >> but
> > >> > > > > >> >>> > > >> >> > what
> > >> > > > > >> >>> > > >> >> > > is
> > >> > > > > >> >>> > > >> >> > > > >>> about
> > >> > > > > >> >>> > > >> >> > > > >>> > maintaining and committer
> > >> permissions to
> > >> > > the
> > >> > > > > new
> > >> > > > > >> >>> > > >> repositories?
> > >> > > > > >> >>> > > >> >> > > > >>> > I see the list of the committers
> > and
> > >> PMC
> > >> > > > > members
> > >> > > > > >> >>> there
> > >> > > > > >> >>> > > >> >> > > > >>> >
> > >> > > https://bahir.apache.org/community-members/
> > >> > > > > >> >>> > > >> >> > > > >>> >
> > >> > > > > >> >>> > > >> >> > > > >>> > Could somebody from Ignite
> > community
> > >> be
> > >> > > > added
> > >> > > > > to
> > >> > > > > >> >>> this
> > >> > > > > >> >>> > > list
> > >> > > > > >> >>> > > >> as
> > >> > > > > >> >>> > > >> >> a
> > >> > > > > >> >>> > > >> >> > > > >>> > PMC/committer to maintain Ignite
> > >> > > > integrations?
> > >> > > > > >> >>> > > >> >> > > > >>> > If yes, I'd happy to join this
> > >> project
> > >> > and
> > >> > > > > >> >>> collaborate
> > >> > > > > >> >>> > > with
> > >> > > > > >> >>> > > >> >> these
> > >> > > > > >> >>> > > >> >> > > > guys
> > >> > > > > >> >>> > > >> >> > > > >>> > together
> > >> > > > > >> >>> > > >> >> > > > >>> > If no, and we should start from
> the
> > >> zero
> > >> > > > level
> > >> > > > > >> with
> > >> > > > > >> >>> > > >> external
> > >> > > > > >> >>> > > >> >> PRs
> > >> > > > > >> >>> > > >> >> > -
> > >> > > > > >> >>> > > >> >> > > > >>> hmmm,
> > >> > > > > >> >>> > > >> >> > > > >>> > it's better to have our own
> > external
> > >> > > > > repository
> > >> > > > > >> >>> with
> > >> > > > > >> >>> > > >> >> ApacheBahirr
> > >> > > > > >> >>> > > >> >> > > > >>> ideology.
> > >> > > > > >> >>> > > >> >> > > > >>> >
> > >> > > > > >> >>> > > >> >> > > > >>> > Also, I agree, that all
> connectors
> > >> could
> > >> > > be
> > >> > > > > >> moved
> > >> > > > > >> >>> there
> > >> > > > > >> >>> > > (in
> > >> > > > > >> >>> > > >> >> > > > >>> ApacheBahirr
> > >> > > > > >> >>> > > >> >> > > > >>> > like repository), but not all
> > modules
> > >> > from
> > >> > > > the
> > >> > > > > >> page
> > >> > > > > >> >>> > > >> >> > > > >>> >
> > >> > > > > >> >>> > > >> >> > > > >>> >
> > >> > > > > >> >>> > > >> >> > > > >>>
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > >
> > >> > > > > >> >>> > > >> >> >
> > >> > > > > >> >>> > > >> >>
> > >> > > > > >> >>> > > >>
> > >> > > > > >> >>> > >
> > >> > > > > >> >>> >
> > >> > > > > >> >>>
> > >> > > > > >>
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > >> > > > > >> >>> > > >> >> > > > >>> > because
> > >> > > > > >> >>> > > >> >> > > > >>> > they use different parts of core
> > >> modules
> > >> > > and
> > >> > > > > >> could
> > >> > > > > >> >>> be
> > >> > > > > >> >>> > > >> >> developed
> > >> > > > > >> >>> > > >> >> > > with
> > >> > > > > >> >>> > > >> >> > > > >>> > different velocity.
> > >> > > > > >> >>> > > >> >> > > > >>> >
> > >> > > > > >> >>> > > >> >> > > > >>> > In reality, before creation of
> new
> > >> > > > > repositories
> > >> > > > > >> we
> > >> > > > > >> >>> need
> > >> > > > > >> >>> > > >> wide
> > >> > > > > >> >>> > > >> >> > > > discussion
> > >> > > > > >> >>> > > >> >> > > > >>> > based on the document mentioned
> > >> above.
> > >> > > > > >> >>> > > >> >> > > > >>> >
> > >> > > > > >> >>> > > >> >> > > > >>> > P.S. Of course, we could
> experiment
> > >> in
> > >> > the
> > >> > > > > next
> > >> > > > > >> >>> release
> > >> > > > > >> >>> > > 2.8
> > >> > > > > >> >>> > > >> >> with
> > >> > > > > >> >>> > > >> >> > > one
> > >> > > > > >> >>> > > >> >> > > > >>> or two
> > >> > > > > >> >>> > > >> >> > > > >>> > integrations like twitter or
> ZeroMQ
> > >> > > > > >> >>> > > >> >> > > > >>> > Also, I'd like Denis Magda idea
> for
> > >> the
> > >> > > > > separate
> > >> > > > > >> >>> > > >> repositories
> > >> > > > > >> >>> > > >> >> in
> > >> > > > > >> >>> > > >> >> > > > Apache
> > >> > > > > >> >>> > > >> >> > > > >>> > Github to maintain them by
> anybody.
> > >> > > > > >> >>> > > >> >> > > > >>> >
> > >> > > > > >> >>> > > >> >> > > > >>> >
> > >> > > > > >> >>> > > >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02,
> Saikat
> > >> > > Maitra <
> > >> > > > > >> >>> > > >> >> > > saikat.maitra@gmail.com
> > >> > > > > >> >>> > > >> >> > > > >:
> > >> > > > > >> >>> > > >> >> > > > >>> >
> > >> > > > > >> >>> > > >> >> > > > >>> > > Hi Denis, Emmanouil
> > >> > > > > >> >>> > > >> >> > > > >>> > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > Thank you for your email. I
> think
> > >> > > > creating a
> > >> > > > > >> >>> separate
> > >> > > > > >> >>> > > >> >> > integration
> > >> > > > > >> >>> > > >> >> > > > >>> project
> > >> > > > > >> >>> > > >> >> > > > >>> > > in github and also proposing it
> > as
> > >> an
> > >> > > > apache
> > >> > > > > >> >>> > incubator
> > >> > > > > >> >>> > > >> >> project
> > >> > > > > >> >>> > > >> >> > > will
> > >> > > > > >> >>> > > >> >> > > > >>> be a
> > >> > > > > >> >>> > > >> >> > > > >>> > > good move. The other separate
> > >> > > integration
> > >> > > > > >> modules
> > >> > > > > >> >>> > > >> >> > > > >>> > >
> > >> > > > > >> >>> > > >> >> > > > >>> > >
> > >> > > > > >> >>> > > >> >> > > > >>> >
> > >> > > > > >> >>> > > >> >> > > > >>>
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > >
> > >> > > > > >> >>> > > >> >> >
> > >> > > > > >> >>> > > >> >>
> > >> > > > > >> >>> > > >>
> > >> > > > > >> >>> > >
> > >> > > > > >> >>> >
> > >> > > > > >> >>>
> > >> > > > > >>
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > >> > > > > >> >>> > > >> >> > > > >>> > > can
> > >> > > > > >> >>> > > >> >> > > > >>> > > be moved to this new apache
> > >> incubator
> > >> > > > > project.
> > >> > > > > >> >>> > > >> >> > > > >>> > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > There are similar integration
> > >> > available
> > >> > > > for
> > >> > > > > >> >>> Flink and
> > >> > > > > >> >>> > > >> Spark
> > >> > > > > >> >>> > > >> >> in
> > >> > > > > >> >>> > > >> >> > > > Apache
> > >> > > > > >> >>> > > >> >> > > > >>> > Bahir
> > >> > > > > >> >>> > > >> >> > > > >>> > > https://bahir.apache.org/
> > >> > > > > >> >>> > > >> >> > > > >>> > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > Do you think we should reach
> out
> > to
> > >> > > Apache
> > >> > > > > >> Bahir
> > >> > > > > >> >>> > > >> community
> > >> > > > > >> >>> > > >> >> > with a
> > >> > > > > >> >>> > > >> >> > > > >>> > proposal
> > >> > > > > >> >>> > > >> >> > > > >>> > > or we should plan to create a
> > >> separate
> > >> > > > > Apache
> > >> > > > > >> >>> > Incubator
> > >> > > > > >> >>> > > >> >> > project?
> > >> > > > > >> >>> > > >> >> > > > >>> > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > Regards,
> > >> > > > > >> >>> > > >> >> > > > >>> > > Saikat
> > >> > > > > >> >>> > > >> >> > > > >>> > >
> > >> > > > > >> >>> > > >> >> > > > >>> > >
> > >> > > > > >> >>> > > >> >> > > > >>> > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM
> > >> > > Emmanouil
> > >> > > > > >> >>> > Gkatziouras <
> > >> > > > > >> >>> > > >> >> > > > >>> > > gkatzioura@gmail.com>
> > >> > > > > >> >>> > > >> >> > > > >>> > > wrote:
> > >> > > > > >> >>> > > >> >> > > > >>> > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > Hi all!
> > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > Based on the discussions most
> > >> > probably
> > >> > > > > this
> > >> > > > > >> is
> > >> > > > > >> >>> > going
> > >> > > > > >> >>> > > to
> > >> > > > > >> >>> > > >> >> be on
> > >> > > > > >> >>> > > >> >> > > > >>> another
> > >> > > > > >> >>> > > >> >> > > > >>> > > > GitHub repo. Therefore I will
> > >> > prepare
> > >> > > a
> > >> > > > > >> >>> standalone
> > >> > > > > >> >>> > > >> project
> > >> > > > > >> >>> > > >> >> > with
> > >> > > > > >> >>> > > >> >> > > > the
> > >> > > > > >> >>> > > >> >> > > > >>> > > Pub/Sub
> > >> > > > > >> >>> > > >> >> > > > >>> > > > module and once the
> repository
> > is
> > >> > > > created
> > >> > > > > a
> > >> > > > > >> >>> pull
> > >> > > > > >> >>> > > >> request
> > >> > > > > >> >>> > > >> >> > shall
> > >> > > > > >> >>> > > >> >> > > be
> > >> > > > > >> >>> > > >> >> > > > >>> > issued.
> > >> > > > > >> >>> > > >> >> > > > >>> > > > If there is anything else I
> can
> > >> do
> > >> > in
> > >> > > > > order
> > >> > > > > >> to
> > >> > > > > >> >>> > > >> facilitate
> > >> > > > > >> >>> > > >> >> > this
> > >> > > > > >> >>> > > >> >> > > > >>> please
> > >> > > > > >> >>> > > >> >> > > > >>> > let
> > >> > > > > >> >>> > > >> >> > > > >>> > > > me know.
> > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > If a ticket is created for
> this
> > >> > > issue, I
> > >> > > > > >> shall
> > >> > > > > >> >>> be
> > >> > > > > >> >>> > > able
> > >> > > > > >> >>> > > >> to
> > >> > > > > >> >>> > > >> >> > > create
> > >> > > > > >> >>> > > >> >> > > > a
> > >> > > > > >> >>> > > >> >> > > > >>> pull
> > >> > > > > >> >>> > > >> >> > > > >>> > > > request and thus a build will
> > >> take
> > >> > > place
> > >> > > > > on
> > >> > > > > >> >>> > ignite's
> > >> > > > > >> >>> > > >> CI.
> > >> > > > > >> >>> > > >> >> > > > >>> > > > If everyone is aligned with
> > >> adding a
> > >> > > > > Pub/Sub
> > >> > > > > >> >>> > > >> >> implementation
> > >> > > > > >> >>> > > >> >> > > may I
> > >> > > > > >> >>> > > >> >> > > > >>> > create
> > >> > > > > >> >>> > > >> >> > > > >>> > > a
> > >> > > > > >> >>> > > >> >> > > > >>> > > > ticket on JIRA?
> > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > My next question has to do on
> > how
> > >> > > Ignite
> > >> > > > > >> >>> configures
> > >> > > > > >> >>> > > the
> > >> > > > > >> >>> > > >> >> build
> > >> > > > > >> >>> > > >> >> > > > jobs
> > >> > > > > >> >>> > > >> >> > > > >>> in
> > >> > > > > >> >>> > > >> >> > > > >>> > > Team
> > >> > > > > >> >>> > > >> >> > > > >>> > > > City. The unit tests for my
> > >> > > > implementation
> > >> > > > > >> >>> need a
> > >> > > > > >> >>> > > local
> > >> > > > > >> >>> > > >> >> > Pub/Sub
> > >> > > > > >> >>> > > >> >> > > > >>> server
> > >> > > > > >> >>> > > >> >> > > > >>> > up
> > >> > > > > >> >>> > > >> >> > > > >>> > > > and running (they provide one
> > for
> > >> > > > testing
> > >> > > > > >> >>> > purposes).
> > >> > > > > >> >>> > > >> Will
> > >> > > > > >> >>> > > >> >> > this
> > >> > > > > >> >>> > > >> >> > > > take
> > >> > > > > >> >>> > > >> >> > > > >>> > > effect
> > >> > > > > >> >>> > > >> >> > > > >>> > > > in a form of a build script
> > >> inside
> > >> > my
> > >> > > > > >> >>> > implementation
> > >> > > > > >> >>> > > or
> > >> > > > > >> >>> > > >> >> is it
> > >> > > > > >> >>> > > >> >> > > > >>> something
> > >> > > > > >> >>> > > >> >> > > > >>> > > > that should be configured on
> > Team
> > >> > > City?
> > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > Kind regards,
> > >> > > > > >> >>> > > >> >> > > > >>> > > > Emmanouil
> > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > *Emmanouil Gkatziouras*
> > >> > > > > >> >>> > > >> >> > > > >>> > > > https://egkatzioura.com/ |
> > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > >> > > > > >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> > > > > >> >>> > > >> >> > > > >>> > > >
> https://github.com/gkatzioura
> > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27,
> > >> Denis
> > >> > > > Magda
> > >> > > > > <
> > >> > > > > >> >>> > > >> >> dmagda@apache.org
> > >> > > > > >> >>> > > >> >> > >
> > >> > > > > >> >>> > > >> >> > > > >>> wrote:
> > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > Emmanouil, Saikat,
> > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > After contemplating on
> this,
> > >> let
> > >> > me
> > >> > > > > >> propose
> > >> > > > > >> >>> > another
> > >> > > > > >> >>> > > >> way
> > >> > > > > >> >>> > > >> >> > with
> > >> > > > > >> >>> > > >> >> > > > this
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > particular integration that
> > is
> > >> > > aligned
> > >> > > > > >> with
> > >> > > > > >> >>> our
> > >> > > > > >> >>> > > >> >> > > modularization
> > >> > > > > >> >>> > > >> >> > > > >>> > > endeavor.
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > The modularization [1]
> > defines
> > >> the
> > >> > > > > Ignite
> > >> > > > > >> >>> core,
> > >> > > > > >> >>> > > >> Ignite
> > >> > > > > >> >>> > > >> >> > > modules,
> > >> > > > > >> >>> > > >> >> > > > >>> and
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > independent integrations.
> The
> > >> > first
> > >> > > > two
> > >> > > > > >> are
> > >> > > > > >> >>> to be
> > >> > > > > >> >>> > > >> >> located
> > >> > > > > >> >>> > > >> >> > in
> > >> > > > > >> >>> > > >> >> > > > >>> Ignite
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > repositories and to be
> > >> > > tested/updated
> > >> > > > by
> > >> > > > > >> the
> > >> > > > > >> >>> > > >> community
> > >> > > > > >> >>> > > >> >> for
> > >> > > > > >> >>> > > >> >> > > > every
> > >> > > > > >> >>> > > >> >> > > > >>> > > release.
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > The modules are a subset of
> > >> > existing
> > >> > > > > >> >>> integrations
> > >> > > > > >> >>> > > >> that
> > >> > > > > >> >>> > > >> >> are
> > >> > > > > >> >>> > > >> >> > > > >>> adopted
> > >> > > > > >> >>> > > >> >> > > > >>> > > widely
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > and require community
> > >> attention.
> > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > As for the independent
> > >> > integrations
> > >> > > > [1],
> > >> > > > > >> >>> those
> > >> > > > > >> >>> > are
> > >> > > > > >> >>> > > >> >> supposed
> > >> > > > > >> >>> > > >> >> > > to
> > >> > > > > >> >>> > > >> >> > > > be
> > >> > > > > >> >>> > > >> >> > > > >>> > > located
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > in a separate GitHub repo.
> I
> > >> would
> > >> > > > > advise
> > >> > > > > >> us
> > >> > > > > >> >>> to
> > >> > > > > >> >>> > > start
> > >> > > > > >> >>> > > >> >> > > > >>> implementing
> > >> > > > > >> >>> > > >> >> > > > >>> > this
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > concept with the Pub/Sub
> > >> > > integration.
> > >> > > > > Even
> > >> > > > > >> >>> though
> > >> > > > > >> >>> > > it
> > >> > > > > >> >>> > > >> >> will
> > >> > > > > >> >>> > > >> >> > be
> > >> > > > > >> >>> > > >> >> > > > >>> located
> > >> > > > > >> >>> > > >> >> > > > >>> > > in a
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > separate Github repo, it
> will
> > >> be
> > >> > > > listed
> > >> > > > > >> among
> > >> > > > > >> >>> > > >> officially
> > >> > > > > >> >>> > > >> >> > > > >>> available
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > integrations on the Ignite
> > >> website
> > >> > > > > >> featuring
> > >> > > > > >> >>> > > >> Emmanouil
> > >> > > > > >> >>> > > >> >> as a
> > >> > > > > >> >>> > > >> >> > > > >>> > > contributor.
> > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > What do you think?
> > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > [1]
> > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > >
> > >> > > > > >> >>> > > >> >> > > > >>> >
> > >> > > > > >> >>> > > >> >> > > > >>>
> > >> > > > > >> >>> > > >> >> > > >
> > >> > > > > >> >>> > > >> >> > >
> > >> > > > > >> >>> > > >> >> >
> > >> > > > > >> >>> > > >> >>
> > >> > > > > >> >>> > > >>
> > >> > > > > >> >>> > >
> > >> > > > > >> >>> >
> > >> > > > > >> >>>
> > >> > > > > >>
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > -
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > Denis
> > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at
> 9:47
> > PM
> > >> > > Saikat
> > >> > > > > >> >>> Maitra <
> > >> > > > > >> >>> > > >> >> > > > >>> > saikat.maitra@gmail.com
> > >> > > > > >> >>> > > >> >> > > > >>> > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > wrote:
> > >> > > > > >> >>> > > >> >> > > > >>> > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > Hi Emmanouil,
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > The changes looks good to
> > >> me. I
> > >> > > > wanted
> > >> > > > > >> to
> > >> > > > > >> >>> check
> > >> > > > > >> >>> > > if
> > >> > > > > >> >>> > > >> you
> > >> > > > > >> >>> > > >> >> > are
> > >> > > > > >> >>> > > >> >> > > > also
> > >> > > > > >> >>> > > >> >> > > > >>> > able
> > >> > > > > >> >>> > > >> >> > > > >>> > > to
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > validate that the once
> data
> > >> is
> > >> > > added
> > >> > > > > to
> > >> > > > > >> >>> Ignite
> > >> > > > > >> >>> > > >> cluster
> > >> > > > > >> >>> > > >> >> > you
> > >> > > > > >> >>> > > >> >> > > > are
> > >> > > > > >> >>> > > >> >> > > > >>> also
> > >> > > > > >> >>> > > >> >> > > > >>> > > > able
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > to
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > access it using another
> > >> Ignite
> > >> > > > client
> > >> > > > > or
> > >> > > > > >> >>> using
> > >> > > > > >> >>> > > rest
> > >> > > > > >> >>> > > >> >> > > > endpoints.
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > We use Teamcity for CI
> > >> process,
> > >> > > the
> > >> > > > > >> >>> details are
> > >> > > > > >> >>> > > as
> > >> > > > > >> >>> > > >> >> > follows
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > Trigger validation of
> those
> > >> test
> > >> > > > > suites
> > >> > > > > >> >>> that
> > >> > > > > >> >>> > have
> > >> > > > > >> >>> > > >> been
> > >> > > > > >> >>> > > >> >> > > > >>> affected by
> > >> > > > > >> >>> > > >> >> > > > >>> > > your
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > changes on TeamCity <
> > >> > > > > >> >>> > > http://ci.ignite.apache.org/>:
> > >> > > > > >> >>> > > >> <
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > http://204.14.53.153/
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Locate a test suite
> > you
> > >> > have
> > >> > > to
> > >> > > > > >> check,
> > >> > > > > >> >>> > press
> > >> > > > > >> >>> > > >> >> button
> > >> > > > > >> >>> > > >> >> > > > named
> > >> > > > > >> >>> > > >> >> > > > >>> > "..."
> > >> > > > > >> >>> > > >> >> > > > >>> > > > that
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    is located on the left
> > of
> > >> > "Run"
> > >> > > > > >> button.
> > >> > > > > >> >>> "Run
> > >> > > > > >> >>> > > >> custom
> > >> > > > > >> >>> > > >> >> > > build"
> > >> > > > > >> >>> > > >> >> > > > >>> > dialog
> > >> > > > > >> >>> > > >> >> > > > >>> > > > will
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    appear;
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Go to "Changes" tab
> > and
> > >> > > choose
> > >> > > > > >> >>> > > >> >> > > > >>> > "pull/<pull-request-number>/head"
> > >> > > > > >> >>> > > >> >> > > > >>> > > > in
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    "Build branch"
> dropdown
> > >> list;
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Press "Run build"
> > button
> > >> > and
> > >> > > > > >> monitor
> > >> > > > > >> >>> tests
> > >> > > > > >> >>> > > >> >> results.
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > @Roman rshtykh@yahoo.com
> -
> > >> can
> > >> > > you
> > >> > > > > also
> > >> > > > > >> >>> please
> > >> > > > > >> >>> > > >> take a
> > >> > > > > >> >>> > > >> >> > > look?
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > Regards,
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > Saikat
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at
> > 6:20
> > >> PM
> > >> > > > > >> Emmanouil
> > >> > > > > >> >>> > > >> Gkatziouras
> > >> > > > > >> >>> > > >> >> <
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > gkatzioura@gmail.com>
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > wrote:
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Hi @Saikat Maitra
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > I have my
> implementation
> > >> on an
> > >> > > > > Ignite
> > >> > > > > >> >>> fork on
> > >> > > > > >> >>> > > my
> > >> > > > > >> >>> > > >> >> > personal
> > >> > > > > >> >>> > > >> >> > > > >>> GitHub
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > account
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > [1]
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > It is based on the
> Kafka
> > >> > > > > >> implementation
> > >> > > > > >> >>> which
> > >> > > > > >> >>> > > is
> > >> > > > > >> >>> > > >> of
> > >> > > > > >> >>> > > >> >> a
> > >> > > > > >> >>> > > >> >> > > > polling
> > >> > > > > >> >>> > > >> >> > > > >>> > > nature.
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > GCP does not provide a
> > >> service
> > >> > > > like
> > >> > > > > >> >>> Kafka.
> > >> > > > > >> >>> > > >> Instead
> > >> > > > > >> >>> > > >> >> it
> > >> > > > > >> >>> > > >> >> > > > >>> provides us
> > >> > > > > >> >>> > > >> >> > > > >>> > > > with
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Pub/Sub which is just
> > like
> > >> > > Kafka.
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Therefore using the
> Kafka
> > >> > > streamer
> > >> > > > > as
> > >> > > > > >> a
> > >> > > > > >> >>> > > guideline
> > >> > > > > >> >>> > > >> >> made
> > >> > > > > >> >>> > > >> >> > > more
> > >> > > > > >> >>> > > >> >> > > > >>> sense
> > >> > > > > >> >>> > > >> >> > > > >>> > > to
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > me.
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > >
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Apart from the
> > >> implementation
> > >> > I
> > >> > > > have
> > >> > > > > >> also
> > >> > > > > >> >>> > > added a
> > >> > > > > >> >>> > > >> >> unit
> > >> > > > > >> >>> > > >> >> > > > test.
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > It definitely needs
> some
> > >> > > polishing
> > >> > > > > >> and a
> > >> > > > > >> >>> > double
> > >> > > > > >> >>> > > >> >> check
> > >> > > > > >> >>> > > >> >> > on
> > >> > > > > >> >>> > > >> >> > > my
> > >> > > > > >> >>> > > >> >> > > > >>> side
> > >> > > > > >> >>> > > >> >> > > > >>> > to
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > make
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > the review smooth.
> > >> > > > > >> >>> > > >> >> > > > >>> > > > > > > For the tests a local
> > >> Pub/Sub
> > >> > > [2]
> > >> > > > > >> server
> > >> > > >
> > >
> > >
> >
>

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Emmanouil Gkatziouras <gk...@gmail.com>]

Hi Saikat!

From the logs[1] it seems that TC cannot find the project. However I did
include the project on the parent pom (modules sections) [2]
I tried to reproduce locally unfortunately I am not aware of the full
context on TC. It seems as if a different parent pom is being picked up.

Kind regards

[1]
https://ci.ignite.apache.org/viewLog.html?buildId=4804893&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=1704
[2]
https://github.com/gkatzioura/ignite-extensions/blob/d6a8beee7feff28f59d9a12be692016305564d25/pom.xml#L46

*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura


On Sat, 30 Nov 2019 at 22:20, Saikat Maitra <sa...@gmail.com> wrote:

> Hello,
>
> I have made changes in Ignite Extensions build configurations.
>
> It is now running the tests as expected.
>
>
> https://ci.ignite.apache.org/viewLog.html?buildId=4804480&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
>
> Regards,
> Saikat
>
> On Thu, Nov 28, 2019 at 1:29 PM Saikat Maitra <sa...@gmail.com>
> wrote:
>
> > Hi,
> >
> > We do run build with -DskipTests and then configure selective test suites
> > to run.
> >
> > I will check why these test suites are not getting executed.
> >
> > Regards
> > Saikat
> >
> > On Wed, 27 Nov 2019 at 3:05 PM, Emmanouil Gkatziouras <
> > gkatzioura@gmail.com> wrote:
> >
> >> Hi all,
> >>
> >> I did add a test suite and run the build with the corresponding
> arguments,
> >> however the tests did not run [1].
> >> I checked the Flink's build logs and the same message is displayed there
> >> too `No tests to run` [2].
> >>
> >> [1]
> >>
> >>
> https://ci.ignite.apache.org/viewLog.html?buildId=4798203&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=248
> >> [2]
> >>
> >>
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4789199&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=1256
> >>
> >> Kind regards
> >> *Emmanouil Gkatziouras*
> >> https://egkatzioura.com/ |
> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> https://github.com/gkatzioura
> >>
> >>
> >> On Wed, 27 Nov 2019 at 18:58, Saikat Maitra <sa...@gmail.com>
> >> wrote:
> >>
> >> > Hi,
> >> >
> >> > We will need to add test suites for tests to be executed in build. We
> >> pass
> >> > them as TestSuites param. I had added Flink sink and source
> testsuites.
> >> >
> >> > Can you please review and confirm.
> >> >
> >> > Regards
> >> > Saikat
> >> >
> >> > On Wed, 27 Nov 2019 at 6:00 AM, Emmanouil Gkatziouras <
> >> > gkatzioura@gmail.com>
> >> > wrote:
> >> >
> >> > > Hi all!
> >> > >
> >> > > I was successful building and running my pull requests. It seems
> that
> >> the
> >> > > tests do not run both for Flink and Pub/Sub [1]
> >> > > If there is something I cannot do to make them discoverable please
> >> let me
> >> > > know.
> >> > >
> >> > > [1]
> >> > >
> >> > >
> >> >
> >>
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4796722&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=283
> >> > >
> >> > > *Emmanouil Gkatziouras*
> >> > > https://egkatzioura.com/ |
> >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > https://github.com/gkatzioura
> >> > >
> >> > >
> >> > > On Wed, 27 Nov 2019 at 04:57, Saikat Maitra <
> saikat.maitra@gmail.com>
> >> > > wrote:
> >> > >
> >> > > > Hi Emmanouil,
> >> > > >
> >> > > > I have added you as contributor in Ignite Extensions project. Can
> >> you
> >> > > > please check and confirm if you are able to see the project and
> >> execute
> >> > > > build on your pull request.
> >> > > >
> >> > > > Regards,
> >> > > > Saikat
> >> > > >
> >> > > > On Tue, Nov 26, 2019 at 12:49 PM Emmanouil Gkatziouras <
> >> > > > gkatzioura@gmail.com>
> >> > > > wrote:
> >> > > >
> >> > > > > Hi all!
> >> > > > >
> >> > > > > If someone could give me read access to the Job of Ignite
> >> Extensions
> >> > on
> >> > > > > Team City will greatly help me (username gkatzioura).
> >> > > > > I suppose the builds get triggered automatically on a pull
> >> request.
> >> > > > >
> >> > > > > Kind regards,
> >> > > > > Emmanouil
> >> > > > >
> >> > > > > *Emmanouil Gkatziouras*
> >> > > > > https://egkatzioura.com/ |
> >> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > > > https://github.com/gkatzioura
> >> > > > >
> >> > > > >
> >> > > > > On Sun, 24 Nov 2019 at 21:11, Emmanouil Gkatziouras <
> >> > > > gkatzioura@gmail.com>
> >> > > > > wrote:
> >> > > > >
> >> > > > > > Hi Saikat!
> >> > > > > >
> >> > > > > > I just rebased with the flink branch. Unfortunately I do not
> >> have
> >> > > read
> >> > > > > > access to the team city link you provided and thus evaluate
> the
> >> > tests
> >> > > > for
> >> > > > > > my pull request.
> >> > > > > > I guess it has to do with ignite extensions being new.
> >> > > > > > My username is gkatzioura and email the one I am using on this
> >> > email
> >> > > > > >
> >> > > > > > Thank you for your time.
> >> > > > > >
> >> > > > > >
> >> > > > > > *Emmanouil Gkatziouras*
> >> > > > > > https://egkatzioura.com/ |
> >> > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > > > > https://github.com/gkatzioura
> >> > > > > >
> >> > > > > >
> >> > > > > > On Sun, 24 Nov 2019 at 18:03, Saikat Maitra <
> >> > saikat.maitra@gmail.com
> >> > > >
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > >> Hi Emmanouil,
> >> > > > > >>
> >> > > > > >> The latest build on teamcity has passed on Flink pull
> request.
> >> > > > > >>
> >> > > > > >>
> >> > > > > >>
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://ci.ignite.apache.org/viewLog.html?buildId=4788928&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> >> > > > > >>
> >> > > > > >> you should be able to take the changes I made in my PR and
> run
> >> > build
> >> > > > on
> >> > > > > >> your pull request.
> >> > > > > >>
> >> > > > > >> Please let me know if you have any questions.
> >> > > > > >>
> >> > > > > >> Regards,
> >> > > > > >> Saikat
> >> > > > > >>
> >> > > > > >> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra <
> >> > > > saikat.maitra@gmail.com
> >> > > > > >
> >> > > > > >> wrote:
> >> > > > > >>
> >> > > > > >> > Hi Emmanouil,
> >> > > > > >> >
> >> > > > > >> > I have fixed the Flink module testsuites and have setup new
> >> > > project
> >> > > > in
> >> > > > > >> > teamcity.
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >>
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview
> >> > > > > >> >
> >> > > > > >> > I am looking into setting up the build.
> >> > > > > >> >
> >> > > > > >> > Regards,
> >> > > > > >> > Saikat
> >> > > > > >> >
> >> > > > > >> > On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra <
> >> > > > > saikat.maitra@gmail.com>
> >> > > > > >> > wrote:
> >> > > > > >> >
> >> > > > > >> >> Hi Emmanouil,
> >> > > > > >> >>
> >> > > > > >> >> Thank you for your email. Yes, the plan is once the Flink
> >> PR is
> >> > > > > merged
> >> > > > > >> it
> >> > > > > >> >> will provide the Licence, parent POMs etc and you can
> rebase
> >> > from
> >> > > > > >> master
> >> > > > > >> >> branch and apply the changes on top of it.
> >> > > > > >> >>
> >> > > > > >> >> I am getting some issues with test failures in local with
> >> > > > > >> >> GridTestProperties as the test.properties is not present
> in
> >> > this
> >> > > > new
> >> > > > > >> >> project but should be available from dependencies.
> >> > > > > >> >>
> >> > > > > >> >> Once I address this issue, I will go ahead and merge the
> >> > changes
> >> > > > and
> >> > > > > >> then
> >> > > > > >> >> we can take it from there.
> >> > > > > >> >>
> >> > > > > >> >> Regards,
> >> > > > > >> >> Saikat
> >> > > > > >> >>
> >> > > > > >> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras <
> >> > > > > >> >> gkatzioura@gmail.com> wrote:
> >> > > > > >> >>
> >> > > > > >> >>> Hi all,
> >> > > > > >> >>>
> >> > > > > >> >>> I made my first pull request [1]. Since this project is
> >> brand
> >> > > new
> >> > > > > (no
> >> > > > > >> >>> parent poms, licensing), I reckoned it was best to use
> >> > Saikat's
> >> > > > > >> branch on
> >> > > > > >> >>> Flink.
> >> > > > > >> >>> I suppose the Flink branch will be merged first. If not
> >> please
> >> > > > give
> >> > > > > me
> >> > > > > >> >>> guidelines on how I should proceed next.
> >> > > > > >> >>>
> >> > > > > >> >>> Kind regards
> >> > > > > >> >>> Emmanouil
> >> > > > > >> >>>
> >> > > > > >> >>> [1] https://github.com/apache/ignite-extensions/pull/2
> >> > > > > >> >>>
> >> > > > > >> >>> *Emmanouil Gkatziouras*
> >> > > > > >> >>> https://egkatzioura.com/ |
> >> > > > > >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > > > >> >>> https://github.com/gkatzioura
> >> > > > > >> >>>
> >> > > > > >> >>>
> >> > > > > >> >>> On Fri, 22 Nov 2019 at 20:55, Denis Magda <
> >> dmagda@apache.org>
> >> > > > > wrote:
> >> > > > > >> >>>
> >> > > > > >> >>> > Awesome, ping us whenever you're ready!
> >> > > > > >> >>> >
> >> > > > > >> >>> > -
> >> > > > > >> >>> > Denis
> >> > > > > >> >>> >
> >> > > > > >> >>> >
> >> > > > > >> >>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras
> <
> >> > > > > >> >>> > gkatzioura@gmail.com>
> >> > > > > >> >>> > wrote:
> >> > > > > >> >>> >
> >> > > > > >> >>> > > Hi all!
> >> > > > > >> >>> > >
> >> > > > > >> >>> > > I am sorry for being late on that. I was trying to
> >> > refactor
> >> > > > the
> >> > > > > >> test
> >> > > > > >> >>> in
> >> > > > > >> >>> > > order not to be in need of any external tools or
> >> spinning
> >> > > up a
> >> > > > > >> >>> server.
> >> > > > > >> >>> > > I did forked the new repo and indeed my changes
> there,
> >> so
> >> > a
> >> > > > pull
> >> > > > > >> >>> request
> >> > > > > >> >>> > is
> >> > > > > >> >>> > > a matter of time!
> >> > > > > >> >>> > >
> >> > > > > >> >>> > > Kind regards
> >> > > > > >> >>> > >
> >> > > > > >> >>> > > *Emmanouil Gkatziouras*
> >> > > > > >> >>> > > https://egkatzioura.com/ |
> >> > > > > >> >>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > > > >> >>> > > https://github.com/gkatzioura
> >> > > > > >> >>> > >
> >> > > > > >> >>> > >
> >> > > > > >> >>> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda <
> >> > > dmagda@apache.org>
> >> > > > > >> wrote:
> >> > > > > >> >>> > >
> >> > > > > >> >>> > > > Hi Emmanouil,
> >> > > > > >> >>> > > >
> >> > > > > >> >>> > > > Do you have any questions or need any support from
> >> the
> >> > > > > >> community?
> >> > > > > >> >>> > > >
> >> > > > > >> >>> > > > -
> >> > > > > >> >>> > > > Denis
> >> > > > > >> >>> > > >
> >> > > > > >> >>> > > >
> >> > > > > >> >>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <
> >> > > > > >> >>> saikat.maitra@gmail.com
> >> > > > > >> >>> > >
> >> > > > > >> >>> > > > wrote:
> >> > > > > >> >>> > > >
> >> > > > > >> >>> > > >> Hi Emmanouil,
> >> > > > > >> >>> > > >>
> >> > > > > >> >>> > > >> Can you please take a looks at dev utils, if this
> is
> >> > > > > something
> >> > > > > >> >>> you are
> >> > > > > >> >>> > > >> looking for
> >> > > > > >> >>> > > >>
> >> > > > > https://github.com/apache/ignite/tree/master/modules/dev-utils
> >> > > > > >> ?
> >> > > > > >> >>> > > >>
> >> > > > > >> >>> > > >> IMO, if you can release Pub/Sub server in maven
> and
> >> > then
> >> > > > use
> >> > > > > >> it as
> >> > > > > >> >>> > > >> dependency, that would be great.
> >> > > > > >> >>> > > >>
> >> > > > > >> >>> > > >> Regards,
> >> > > > > >> >>> > > >> Saikat
> >> > > > > >> >>> > > >>
> >> > > > > >> >>> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <
> >> > > > > >> >>> > saikat.maitra@gmail.com>
> >> > > > > >> >>> > > >> wrote:
> >> > > > > >> >>> > > >>
> >> > > > > >> >>> > > >> > Hi Emmanouil,
> >> > > > > >> >>> > > >> >
> >> > > > > >> >>> > > >> > The master branch has been initialized. I have
> >> > raised a
> >> > > > PR
> >> > > > > to
> >> > > > > >> >>> > migrate
> >> > > > > >> >>> > > >> > Flink module to ignite-extensions.
> >> > > > > >> >>> > > >> >
> >> > > > > >> >>> > > >> > PR
> >> > https://github.com/apache/ignite-extensions/pull/1
> >> > > > > >> >>> > > >> > Jira :
> >> > > > https://issues.apache.org/jira/browse/IGNITE-12356
> >> > > > > >> >>> > > >> >
> >> > > > > >> >>> > > >> > Regards,
> >> > > > > >> >>> > > >> > Saikat
> >> > > > > >> >>> > > >> >
> >> > > > > >> >>> > > >> >
> >> > > > > >> >>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil
> >> > Gkatziouras <
> >> > > > > >> >>> > > >> > gkatzioura@gmail.com> wrote:
> >> > > > > >> >>> > > >> >
> >> > > > > >> >>> > > >> >> Hi all!
> >> > > > > >> >>> > > >> >>
> >> > > > > >> >>> > > >> >> This is an update. The integration is tested
> with
> >> > > mocks
> >> > > > > >> based
> >> > > > > >> >>> on
> >> > > > > >> >>> > the
> >> > > > > >> >>> > > >> >> original Ignite repo [1].
> >> > > > > >> >>> > > >> >> I am currently waiting for the first commits on
> >> the
> >> > > new
> >> > > > > >> >>> integration
> >> > > > > >> >>> > > >> >> project
> >> > > > > >> >>> > > >> >> to take place in order to create my pull
> request
> >> > > > > accordingly
> >> > > > > >> >>> to the
> >> > > > > >> >>> > > >> >> initial
> >> > > > > >> >>> > > >> >> commits.
> >> > > > > >> >>> > > >> >> In order to simulate the Pub/Sub server in a
> >> test I
> >> > > > used a
> >> > > > > >> test
> >> > > > > >> >>> > > server
> >> > > > > >> >>> > > >> >> implementation [2]. Unfortunately they don't
> have
> >> > this
> >> > > > > util
> >> > > > > >> >>> > uploaded
> >> > > > > >> >>> > > >> in a
> >> > > > > >> >>> > > >> >> maven repo, thus it needs to be hosted on a
> maven
> >> > repo
> >> > > > > >> (Apache
> >> > > > > >> >>> > > License
> >> > > > > >> >>> > > >> >> 2.0). Is there a repo for utilities that Ignite
> >> uses
> >> > > for
> >> > > > > the
> >> > > > > >> >>> > > >> test/build?
> >> > > > > >> >>> > > >> >> I have also created a ticket [3]
> >> > > > > >> >>> > > >> >>
> >> > > > > >> >>> > > >> >> Kind regards
> >> > > > > >> >>> > > >> >>
> >> > > > > >> >>> > > >> >> [1]
> >> > > > > >> >>> >
> >> > > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> >> > > > > >> >>> > > >> >> [2]
> >> > > > > >> >>>
> >> https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> >> > > > > >> >>> > > >> >> [3]
> >> > > https://issues.apache.org/jira/browse/IGNITE-12262
> >> > > > > >> >>> > > >> >>
> >> > > > > >> >>> > > >> >> *Emmanouil Gkatziouras*
> >> > > > > >> >>> > > >> >> https://egkatzioura.com/ |
> >> > > > > >> >>> > > >> >>
> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > > > >> >>> > > >> >> https://github.com/gkatzioura
> >> > > > > >> >>> > > >> >>
> >> > > > > >> >>> > > >> >>
> >> > > > > >> >>> > > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <
> >> > > > > >> >>> > saikat.maitra@gmail.com
> >> > > > > >> >>> > > >
> >> > > > > >> >>> > > >> >> wrote:
> >> > > > > >> >>> > > >> >>
> >> > > > > >> >>> > > >> >> > Hello Ilya,
> >> > > > > >> >>> > > >> >> >
> >> > > > > >> >>> > > >> >> > Thank you for your email. Yes, before we
> remove
> >> > any
> >> > > > > >> >>> integration
> >> > > > > >> >>> > we
> >> > > > > >> >>> > > >> will
> >> > > > > >> >>> > > >> >> > move them to another repository and release.
> >> > > > > >> >>> > > >> >> >
> >> > > > > >> >>> > > >> >> > We are thinking we will start with Pub/Sub
> >> > Streamer
> >> > > > > >> >>> > implementation
> >> > > > > >> >>> > > >> and
> >> > > > > >> >>> > > >> >> > continue to move other integration as
> >> mentioned in
> >> > > the
> >> > > > > >> >>> following
> >> > > > > >> >>> > > doc.
> >> > > > > >> >>> > > >> >> >
> >> > > > > >> >>> > > >> >> >
> >> > > > > >> >>> > > >> >> >
> >> > > > > >> >>> > > >> >>
> >> > > > > >> >>> > > >>
> >> > > > > >> >>> > >
> >> > > > > >> >>> >
> >> > > > > >> >>>
> >> > > > > >>
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >> > > > > >> >>> > > >> >> >
> >> > > > > >> >>> > > >> >> > Regards,
> >> > > > > >> >>> > > >> >> > Saikat
> >> > > > > >> >>> > > >> >> >
> >> > > > > >> >>> > > >> >> >
> >> > > > > >> >>> > > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya
> >> Kasnacheev <
> >> > > > > >> >>> > > >> >> > ilya.kasnacheev@gmail.com>
> >> > > > > >> >>> > > >> >> > wrote:
> >> > > > > >> >>> > > >> >> >
> >> > > > > >> >>> > > >> >> > > Hello!
> >> > > > > >> >>> > > >> >> > >
> >> > > > > >> >>> > > >> >> > > My take on this: we want to offload some of
> >> > > > > >> integrations,
> >> > > > > >> >>> but
> >> > > > > >> >>> > > >> please
> >> > > > > >> >>> > > >> >> > commit
> >> > > > > >> >>> > > >> >> > > and release them to another project first,
> >> and
> >> > > only
> >> > > > > >> then we
> >> > > > > >> >>> > will
> >> > > > > >> >>> > > >> >> decide
> >> > > > > >> >>> > > >> >> > to
> >> > > > > >> >>> > > >> >> > > remove them from our repository.
> >> > > > > >> >>> > > >> >> > >
> >> > > > > >> >>> > > >> >> > > Please also make sure they are available
> with
> >> > same
> >> > > > > >> artifact
> >> > > > > >> >>> > name
> >> > > > > >> >>> > > >> and
> >> > > > > >> >>> > > >> >> API.
> >> > > > > >> >>> > > >> >> > > If this has to be changed, let's wait for
> AI
> >> 3.0
> >> > > > > >> >>> > > >> >> > >
> >> > > > > >> >>> > > >> >> > > Regards,
> >> > > > > >> >>> > > >> >> > > --
> >> > > > > >> >>> > > >> >> > > Ilya Kasnacheev
> >> > > > > >> >>> > > >> >> > >
> >> > > > > >> >>> > > >> >> > >
> >> > > > > >> >>> > > >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra
> <
> >> > > > > >> >>> > > >> saikat.maitra@gmail.com>:
> >> > > > > >> >>> > > >> >> > >
> >> > > > > >> >>> > > >> >> > > > Hello Emmanouil,
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > > > As discussed earlier, I discussed with
> >> Apache
> >> > > > Bahir
> >> > > > > >> >>> community
> >> > > > > >> >>> > > and
> >> > > > > >> >>> > > >> >> they
> >> > > > > >> >>> > > >> >> > > are
> >> > > > > >> >>> > > >> >> > > > interested to have Apache Ignite
> >> extensions as
> >> > > > part
> >> > > > > of
> >> > > > > >> >>> Apache
> >> > > > > >> >>> > > >> Bahir
> >> > > > > >> >>> > > >> >> > > > project.
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > > > Can you please share Pub/Sub streamer
> >> > > > implementation
> >> > > > > >> >>> details
> >> > > > > >> >>> > > with
> >> > > > > >> >>> > > >> >> > Apache
> >> > > > > >> >>> > > >> >> > > > Bahir community and request for feedback.
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > > > https://bahir.apache.org/
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > > > I have also requested for contributor
> >> access
> >> > in
> >> > > > Jira
> >> > > > > >> for
> >> > > > > >> >>> > Apache
> >> > > > > >> >>> > > >> >> Bahir
> >> > > > > >> >>> > > >> >> > > > project so that I can create issues and
> >> assign
> >> > > to
> >> > > > > >> myself.
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > > > I can help with code reviews as well.
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > > > Here is the email thread for reference
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > >
> >> > > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > > > Regards,
> >> > > > > >> >>> > > >> >> > > > Saikat
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat
> >> Maitra
> >> > <
> >> > > > > >> >>> > > >> >> saikat.maitra@gmail.com
> >> > > > > >> >>> > > >> >> > >
> >> > > > > >> >>> > > >> >> > > > wrote:
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > > > > Hello,
> >> > > > > >> >>> > > >> >> > > > >
> >> > > > > >> >>> > > >> >> > > > > As discussed I have created following
> >> > > discussion
> >> > > > > >> >>> threads on
> >> > > > > >> >>> > > our
> >> > > > > >> >>> > > >> >> > > migration
> >> > > > > >> >>> > > >> >> > > > > proposals for Apache Ignite extensions:
> >> > > > > >> >>> > > >> >> > > > >
> >> > > > > >> >>> > > >> >> > > > >
> >> > > > > >> >>> > > >> >> > > > >
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > >
> >> > > > > >> >>> > > >> >> >
> >> > > > > >> >>> > > >> >>
> >> > > > > >> >>> > > >>
> >> > > > > >> >>> > >
> >> > > > > >> >>> >
> >> > > > > >> >>>
> >> > > > > >>
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> >> > > > > >> >>> > > >> >> > > > >
> >> > > > > >> >>> > > >> >> > > > >
> >> > > > > >> >>> > > >>
> >> > > > > >>
> >> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> >> > > > > >> >>> > > >> >> > > > >
> >> > > > > >> >>> > > >> >> > > > > I will share update as I hear more
> >> > > information.
> >> > > > > >> >>> > > >> >> > > > >
> >> > > > > >> >>> > > >> >> > > > > Regards,
> >> > > > > >> >>> > > >> >> > > > > Saikat
> >> > > > > >> >>> > > >> >> > > > >
> >> > > > > >> >>> > > >> >> > > > >
> >> > > > > >> >>> > > >> >> > > > >
> >> > > > > >> >>> > > >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat
> >> > Maitra
> >> > > <
> >> > > > > >> >>> > > >> >> > saikat.maitra@gmail.com
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > > > > wrote:
> >> > > > > >> >>> > > >> >> > > > >
> >> > > > > >> >>> > > >> >> > > > >> Hello Denis,
> >> > > > > >> >>> > > >> >> > > > >>
> >> > > > > >> >>> > > >> >> > > > >> Sure, sounds good. I will create a
> >> separate
> >> > > > > >> discussion
> >> > > > > >> >>> > > thread
> >> > > > > >> >>> > > >> to
> >> > > > > >> >>> > > >> >> get
> >> > > > > >> >>> > > >> >> > > > >> community feedback on whether we
> >> > > > > >> >>> > > >> >> > > > >> should join the Bahir or kick-off
> >> "Ignite
> >> > > > > >> Extensions"
> >> > > > > >> >>> > > project.
> >> > > > > >> >>> > > >> >> > > > >>
> >> > > > > >> >>> > > >> >> > > > >> Regards,
> >> > > > > >> >>> > > >> >> > > > >> Saikat
> >> > > > > >> >>> > > >> >> > > > >>
> >> > > > > >> >>> > > >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis
> >> > Magda <
> >> > > > > >> >>> > > >> dmagda@apache.org>
> >> > > > > >> >>> > > >> >> > > wrote:
> >> > > > > >> >>> > > >> >> > > > >>
> >> > > > > >> >>> > > >> >> > > > >>> Folks,
> >> > > > > >> >>> > > >> >> > > > >>>
> >> > > > > >> >>> > > >> >> > > > >>> The concept of Apache Bahir is
> >> precisely
> >> > > what
> >> > > > we
> >> > > > > >> >>> need!
> >> > > > > >> >>> > > >> Saikat,
> >> > > > > >> >>> > > >> >> > thanks
> >> > > > > >> >>> > > >> >> > > > for
> >> > > > > >> >>> > > >> >> > > > >>> doing the research. Why I believe
> the *
> >> > > idea*
> >> > > > > >> fits us
> >> > > > > >> >>> > well:
> >> > > > > >> >>> > > >> >> > > > >>>
> >> > > > > >> >>> > > >> >> > > > >>>    - All integrations can be stored
> in
> >> > > > separate
> >> > > > > >> >>> Github
> >> > > > > >> >>> > > >> >> repositories
> >> > > > > >> >>> > > >> >> > > and
> >> > > > > >> >>> > > >> >> > > > >>>    have their dev lifecycles. We've
> not
> >> > > > obliged
> >> > > > > to
> >> > > > > >> >>> couple
> >> > > > > >> >>> > > all
> >> > > > > >> >>> > > >> >> the
> >> > > > > >> >>> > > >> >> > > > >>> integrations
> >> > > > > >> >>> > > >> >> > > > >>>    in a single repo. For instance,
> >> Spark
> >> > can
> >> > > > be
> >> > > > > >> >>> located
> >> > > > > >> >>> > in
> >> > > > > >> >>> > > a
> >> > > > > >> >>> > > >> >> > > dedicated
> >> > > > > >> >>> > > >> >> > > > >>> repo
> >> > > > > >> >>> > > >> >> > > > >>>    while streaming integrations might
> >> be
> >> > in
> >> > > a
> >> > > > > >> single
> >> > > > > >> >>> one.
> >> > > > > >> >>> > > >> It's
> >> > > > > >> >>> > > >> >> up
> >> > > > > >> >>> > > >> >> > to
> >> > > > > >> >>> > > >> >> > > > us.
> >> > > > > >> >>> > > >> >> > > > >>>    - All the repositories and
> >> integrations
> >> > > > > belong
> >> > > > > >> to
> >> > > > > >> >>> ASF.
> >> > > > > >> >>> > > >> We're
> >> > > > > >> >>> > > >> >> not
> >> > > > > >> >>> > > >> >> > > > >>> dumping
> >> > > > > >> >>> > > >> >> > > > >>>    them on Github but rather keep
> >> > supporting
> >> > > > and
> >> > > > > >> >>> > developing
> >> > > > > >> >>> > > >> in
> >> > > > > >> >>> > > >> >> > > > >>> accordance with
> >> > > > > >> >>> > > >> >> > > > >>>    ASF vision and practices.
> >> > > > > >> >>> > > >> >> > > > >>>    - There is a way to reward
> >> contributors
> >> > > via
> >> > > > > >> >>> > > committership
> >> > > > > >> >>> > > >> and
> >> > > > > >> >>> > > >> >> > PMC
> >> > > > > >> >>> > > >> >> > > > >>>    membership. You won't get this if
> a
> >> > > project
> >> > > > > is
> >> > > > > >> >>> just
> >> > > > > >> >>> > one
> >> > > > > >> >>> > > of
> >> > > > > >> >>> > > >> >> the
> >> > > > > >> >>> > > >> >> > > > >>> millions of
> >> > > > > >> >>> > > >> >> > > > >>>    Github projects.
> >> > > > > >> >>> > > >> >> > > > >>>
> >> > > > > >> >>> > > >> >> > > > >>> Saikat, as Ignite PMC member, could
> you
> >> > > please
> >> > > > > >> >>> kick-off a
> >> > > > > >> >>> > > >> >> separate
> >> > > > > >> >>> > > >> >> > > > >>> dev-list
> >> > > > > >> >>> > > >> >> > > > >>> discussion to involve more community
> >> > members
> >> > > > and
> >> > > > > >> >>> > referring
> >> > > > > >> >>> > > to
> >> > > > > >> >>> > > >> >> this
> >> > > > > >> >>> > > >> >> > > > >>> thread?
> >> > > > > >> >>> > > >> >> > > > >>> I think the primary question the
> >> community
> >> > > > needs
> >> > > > > >> to
> >> > > > > >> >>> > answer
> >> > > > > >> >>> > > >> >> whether
> >> > > > > >> >>> > > >> >> > we
> >> > > > > >> >>> > > >> >> > > > >>> should join the Bahir or kick-off
> >> "Ignite
> >> > > > > >> Extensions"
> >> > > > > >> >>> > > >> project?
> >> > > > > >> >>> > > >> >> > > > >>>
> >> > > > > >> >>> > > >> >> > > > >>> -
> >> > > > > >> >>> > > >> >> > > > >>> Denis
> >> > > > > >> >>> > > >> >> > > > >>>
> >> > > > > >> >>> > > >> >> > > > >>>
> >> > > > > >> >>> > > >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM
> Alexey
> >> > > > Zinoviev
> >> > > > > <
> >> > > > > >> >>> > > >> >> > > > zaleslaw.sin@gmail.com>
> >> > > > > >> >>> > > >> >> > > > >>> wrote:
> >> > > > > >> >>> > > >> >> > > > >>>
> >> > > > > >> >>> > > >> >> > > > >>> > Maybe we could move all our
> Streaming
> >> > > > > >> Integrations
> >> > > > > >> >>> > there,
> >> > > > > >> >>> > > >> but
> >> > > > > >> >>> > > >> >> > what
> >> > > > > >> >>> > > >> >> > > is
> >> > > > > >> >>> > > >> >> > > > >>> about
> >> > > > > >> >>> > > >> >> > > > >>> > maintaining and committer
> >> permissions to
> >> > > the
> >> > > > > new
> >> > > > > >> >>> > > >> repositories?
> >> > > > > >> >>> > > >> >> > > > >>> > I see the list of the committers
> and
> >> PMC
> >> > > > > members
> >> > > > > >> >>> there
> >> > > > > >> >>> > > >> >> > > > >>> >
> >> > > https://bahir.apache.org/community-members/
> >> > > > > >> >>> > > >> >> > > > >>> >
> >> > > > > >> >>> > > >> >> > > > >>> > Could somebody from Ignite
> community
> >> be
> >> > > > added
> >> > > > > to
> >> > > > > >> >>> this
> >> > > > > >> >>> > > list
> >> > > > > >> >>> > > >> as
> >> > > > > >> >>> > > >> >> a
> >> > > > > >> >>> > > >> >> > > > >>> > PMC/committer to maintain Ignite
> >> > > > integrations?
> >> > > > > >> >>> > > >> >> > > > >>> > If yes, I'd happy to join this
> >> project
> >> > and
> >> > > > > >> >>> collaborate
> >> > > > > >> >>> > > with
> >> > > > > >> >>> > > >> >> these
> >> > > > > >> >>> > > >> >> > > > guys
> >> > > > > >> >>> > > >> >> > > > >>> > together
> >> > > > > >> >>> > > >> >> > > > >>> > If no, and we should start from the
> >> zero
> >> > > > level
> >> > > > > >> with
> >> > > > > >> >>> > > >> external
> >> > > > > >> >>> > > >> >> PRs
> >> > > > > >> >>> > > >> >> > -
> >> > > > > >> >>> > > >> >> > > > >>> hmmm,
> >> > > > > >> >>> > > >> >> > > > >>> > it's better to have our own
> external
> >> > > > > repository
> >> > > > > >> >>> with
> >> > > > > >> >>> > > >> >> ApacheBahirr
> >> > > > > >> >>> > > >> >> > > > >>> ideology.
> >> > > > > >> >>> > > >> >> > > > >>> >
> >> > > > > >> >>> > > >> >> > > > >>> > Also, I agree, that all connectors
> >> could
> >> > > be
> >> > > > > >> moved
> >> > > > > >> >>> there
> >> > > > > >> >>> > > (in
> >> > > > > >> >>> > > >> >> > > > >>> ApacheBahirr
> >> > > > > >> >>> > > >> >> > > > >>> > like repository), but not all
> modules
> >> > from
> >> > > > the
> >> > > > > >> page
> >> > > > > >> >>> > > >> >> > > > >>> >
> >> > > > > >> >>> > > >> >> > > > >>> >
> >> > > > > >> >>> > > >> >> > > > >>>
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > >
> >> > > > > >> >>> > > >> >> >
> >> > > > > >> >>> > > >> >>
> >> > > > > >> >>> > > >>
> >> > > > > >> >>> > >
> >> > > > > >> >>> >
> >> > > > > >> >>>
> >> > > > > >>
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >> > > > > >> >>> > > >> >> > > > >>> > because
> >> > > > > >> >>> > > >> >> > > > >>> > they use different parts of core
> >> modules
> >> > > and
> >> > > > > >> could
> >> > > > > >> >>> be
> >> > > > > >> >>> > > >> >> developed
> >> > > > > >> >>> > > >> >> > > with
> >> > > > > >> >>> > > >> >> > > > >>> > different velocity.
> >> > > > > >> >>> > > >> >> > > > >>> >
> >> > > > > >> >>> > > >> >> > > > >>> > In reality, before creation of new
> >> > > > > repositories
> >> > > > > >> we
> >> > > > > >> >>> need
> >> > > > > >> >>> > > >> wide
> >> > > > > >> >>> > > >> >> > > > discussion
> >> > > > > >> >>> > > >> >> > > > >>> > based on the document mentioned
> >> above.
> >> > > > > >> >>> > > >> >> > > > >>> >
> >> > > > > >> >>> > > >> >> > > > >>> > P.S. Of course, we could experiment
> >> in
> >> > the
> >> > > > > next
> >> > > > > >> >>> release
> >> > > > > >> >>> > > 2.8
> >> > > > > >> >>> > > >> >> with
> >> > > > > >> >>> > > >> >> > > one
> >> > > > > >> >>> > > >> >> > > > >>> or two
> >> > > > > >> >>> > > >> >> > > > >>> > integrations like twitter or ZeroMQ
> >> > > > > >> >>> > > >> >> > > > >>> > Also, I'd like Denis Magda idea for
> >> the
> >> > > > > separate
> >> > > > > >> >>> > > >> repositories
> >> > > > > >> >>> > > >> >> in
> >> > > > > >> >>> > > >> >> > > > Apache
> >> > > > > >> >>> > > >> >> > > > >>> > Github to maintain them by anybody.
> >> > > > > >> >>> > > >> >> > > > >>> >
> >> > > > > >> >>> > > >> >> > > > >>> >
> >> > > > > >> >>> > > >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat
> >> > > Maitra <
> >> > > > > >> >>> > > >> >> > > saikat.maitra@gmail.com
> >> > > > > >> >>> > > >> >> > > > >:
> >> > > > > >> >>> > > >> >> > > > >>> >
> >> > > > > >> >>> > > >> >> > > > >>> > > Hi Denis, Emmanouil
> >> > > > > >> >>> > > >> >> > > > >>> > >
> >> > > > > >> >>> > > >> >> > > > >>> > > Thank you for your email. I think
> >> > > > creating a
> >> > > > > >> >>> separate
> >> > > > > >> >>> > > >> >> > integration
> >> > > > > >> >>> > > >> >> > > > >>> project
> >> > > > > >> >>> > > >> >> > > > >>> > > in github and also proposing it
> as
> >> an
> >> > > > apache
> >> > > > > >> >>> > incubator
> >> > > > > >> >>> > > >> >> project
> >> > > > > >> >>> > > >> >> > > will
> >> > > > > >> >>> > > >> >> > > > >>> be a
> >> > > > > >> >>> > > >> >> > > > >>> > > good move. The other separate
> >> > > integration
> >> > > > > >> modules
> >> > > > > >> >>> > > >> >> > > > >>> > >
> >> > > > > >> >>> > > >> >> > > > >>> > >
> >> > > > > >> >>> > > >> >> > > > >>> >
> >> > > > > >> >>> > > >> >> > > > >>>
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > >
> >> > > > > >> >>> > > >> >> >
> >> > > > > >> >>> > > >> >>
> >> > > > > >> >>> > > >>
> >> > > > > >> >>> > >
> >> > > > > >> >>> >
> >> > > > > >> >>>
> >> > > > > >>
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >> > > > > >> >>> > > >> >> > > > >>> > > can
> >> > > > > >> >>> > > >> >> > > > >>> > > be moved to this new apache
> >> incubator
> >> > > > > project.
> >> > > > > >> >>> > > >> >> > > > >>> > >
> >> > > > > >> >>> > > >> >> > > > >>> > > There are similar integration
> >> > available
> >> > > > for
> >> > > > > >> >>> Flink and
> >> > > > > >> >>> > > >> Spark
> >> > > > > >> >>> > > >> >> in
> >> > > > > >> >>> > > >> >> > > > Apache
> >> > > > > >> >>> > > >> >> > > > >>> > Bahir
> >> > > > > >> >>> > > >> >> > > > >>> > > https://bahir.apache.org/
> >> > > > > >> >>> > > >> >> > > > >>> > >
> >> > > > > >> >>> > > >> >> > > > >>> > > Do you think we should reach out
> to
> >> > > Apache
> >> > > > > >> Bahir
> >> > > > > >> >>> > > >> community
> >> > > > > >> >>> > > >> >> > with a
> >> > > > > >> >>> > > >> >> > > > >>> > proposal
> >> > > > > >> >>> > > >> >> > > > >>> > > or we should plan to create a
> >> separate
> >> > > > > Apache
> >> > > > > >> >>> > Incubator
> >> > > > > >> >>> > > >> >> > project?
> >> > > > > >> >>> > > >> >> > > > >>> > >
> >> > > > > >> >>> > > >> >> > > > >>> > > Regards,
> >> > > > > >> >>> > > >> >> > > > >>> > > Saikat
> >> > > > > >> >>> > > >> >> > > > >>> > >
> >> > > > > >> >>> > > >> >> > > > >>> > >
> >> > > > > >> >>> > > >> >> > > > >>> > >
> >> > > > > >> >>> > > >> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM
> >> > > Emmanouil
> >> > > > > >> >>> > Gkatziouras <
> >> > > > > >> >>> > > >> >> > > > >>> > > gkatzioura@gmail.com>
> >> > > > > >> >>> > > >> >> > > > >>> > > wrote:
> >> > > > > >> >>> > > >> >> > > > >>> > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > Hi all!
> >> > > > > >> >>> > > >> >> > > > >>> > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > Based on the discussions most
> >> > probably
> >> > > > > this
> >> > > > > >> is
> >> > > > > >> >>> > going
> >> > > > > >> >>> > > to
> >> > > > > >> >>> > > >> >> be on
> >> > > > > >> >>> > > >> >> > > > >>> another
> >> > > > > >> >>> > > >> >> > > > >>> > > > GitHub repo. Therefore I will
> >> > prepare
> >> > > a
> >> > > > > >> >>> standalone
> >> > > > > >> >>> > > >> project
> >> > > > > >> >>> > > >> >> > with
> >> > > > > >> >>> > > >> >> > > > the
> >> > > > > >> >>> > > >> >> > > > >>> > > Pub/Sub
> >> > > > > >> >>> > > >> >> > > > >>> > > > module and once the repository
> is
> >> > > > created
> >> > > > > a
> >> > > > > >> >>> pull
> >> > > > > >> >>> > > >> request
> >> > > > > >> >>> > > >> >> > shall
> >> > > > > >> >>> > > >> >> > > be
> >> > > > > >> >>> > > >> >> > > > >>> > issued.
> >> > > > > >> >>> > > >> >> > > > >>> > > > If there is anything else I can
> >> do
> >> > in
> >> > > > > order
> >> > > > > >> to
> >> > > > > >> >>> > > >> facilitate
> >> > > > > >> >>> > > >> >> > this
> >> > > > > >> >>> > > >> >> > > > >>> please
> >> > > > > >> >>> > > >> >> > > > >>> > let
> >> > > > > >> >>> > > >> >> > > > >>> > > > me know.
> >> > > > > >> >>> > > >> >> > > > >>> > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > If a ticket is created for this
> >> > > issue, I
> >> > > > > >> shall
> >> > > > > >> >>> be
> >> > > > > >> >>> > > able
> >> > > > > >> >>> > > >> to
> >> > > > > >> >>> > > >> >> > > create
> >> > > > > >> >>> > > >> >> > > > a
> >> > > > > >> >>> > > >> >> > > > >>> pull
> >> > > > > >> >>> > > >> >> > > > >>> > > > request and thus a build will
> >> take
> >> > > place
> >> > > > > on
> >> > > > > >> >>> > ignite's
> >> > > > > >> >>> > > >> CI.
> >> > > > > >> >>> > > >> >> > > > >>> > > > If everyone is aligned with
> >> adding a
> >> > > > > Pub/Sub
> >> > > > > >> >>> > > >> >> implementation
> >> > > > > >> >>> > > >> >> > > may I
> >> > > > > >> >>> > > >> >> > > > >>> > create
> >> > > > > >> >>> > > >> >> > > > >>> > > a
> >> > > > > >> >>> > > >> >> > > > >>> > > > ticket on JIRA?
> >> > > > > >> >>> > > >> >> > > > >>> > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > My next question has to do on
> how
> >> > > Ignite
> >> > > > > >> >>> configures
> >> > > > > >> >>> > > the
> >> > > > > >> >>> > > >> >> build
> >> > > > > >> >>> > > >> >> > > > jobs
> >> > > > > >> >>> > > >> >> > > > >>> in
> >> > > > > >> >>> > > >> >> > > > >>> > > Team
> >> > > > > >> >>> > > >> >> > > > >>> > > > City. The unit tests for my
> >> > > > implementation
> >> > > > > >> >>> need a
> >> > > > > >> >>> > > local
> >> > > > > >> >>> > > >> >> > Pub/Sub
> >> > > > > >> >>> > > >> >> > > > >>> server
> >> > > > > >> >>> > > >> >> > > > >>> > up
> >> > > > > >> >>> > > >> >> > > > >>> > > > and running (they provide one
> for
> >> > > > testing
> >> > > > > >> >>> > purposes).
> >> > > > > >> >>> > > >> Will
> >> > > > > >> >>> > > >> >> > this
> >> > > > > >> >>> > > >> >> > > > take
> >> > > > > >> >>> > > >> >> > > > >>> > > effect
> >> > > > > >> >>> > > >> >> > > > >>> > > > in a form of a build script
> >> inside
> >> > my
> >> > > > > >> >>> > implementation
> >> > > > > >> >>> > > or
> >> > > > > >> >>> > > >> >> is it
> >> > > > > >> >>> > > >> >> > > > >>> something
> >> > > > > >> >>> > > >> >> > > > >>> > > > that should be configured on
> Team
> >> > > City?
> >> > > > > >> >>> > > >> >> > > > >>> > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > Kind regards,
> >> > > > > >> >>> > > >> >> > > > >>> > > > Emmanouil
> >> > > > > >> >>> > > >> >> > > > >>> > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > *Emmanouil Gkatziouras*
> >> > > > > >> >>> > > >> >> > > > >>> > > > https://egkatzioura.com/ |
> >> > > > > >> >>> > > >> >> > > > >>> > > >
> >> > > > > >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > > > >> >>> > > >> >> > > > >>> > > > https://github.com/gkatzioura
> >> > > > > >> >>> > > >> >> > > > >>> > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27,
> >> Denis
> >> > > > Magda
> >> > > > > <
> >> > > > > >> >>> > > >> >> dmagda@apache.org
> >> > > > > >> >>> > > >> >> > >
> >> > > > > >> >>> > > >> >> > > > >>> wrote:
> >> > > > > >> >>> > > >> >> > > > >>> > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > Emmanouil, Saikat,
> >> > > > > >> >>> > > >> >> > > > >>> > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > After contemplating on this,
> >> let
> >> > me
> >> > > > > >> propose
> >> > > > > >> >>> > another
> >> > > > > >> >>> > > >> way
> >> > > > > >> >>> > > >> >> > with
> >> > > > > >> >>> > > >> >> > > > this
> >> > > > > >> >>> > > >> >> > > > >>> > > > > particular integration that
> is
> >> > > aligned
> >> > > > > >> with
> >> > > > > >> >>> our
> >> > > > > >> >>> > > >> >> > > modularization
> >> > > > > >> >>> > > >> >> > > > >>> > > endeavor.
> >> > > > > >> >>> > > >> >> > > > >>> > > > > The modularization [1]
> defines
> >> the
> >> > > > > Ignite
> >> > > > > >> >>> core,
> >> > > > > >> >>> > > >> Ignite
> >> > > > > >> >>> > > >> >> > > modules,
> >> > > > > >> >>> > > >> >> > > > >>> and
> >> > > > > >> >>> > > >> >> > > > >>> > > > > independent integrations. The
> >> > first
> >> > > > two
> >> > > > > >> are
> >> > > > > >> >>> to be
> >> > > > > >> >>> > > >> >> located
> >> > > > > >> >>> > > >> >> > in
> >> > > > > >> >>> > > >> >> > > > >>> Ignite
> >> > > > > >> >>> > > >> >> > > > >>> > > > > repositories and to be
> >> > > tested/updated
> >> > > > by
> >> > > > > >> the
> >> > > > > >> >>> > > >> community
> >> > > > > >> >>> > > >> >> for
> >> > > > > >> >>> > > >> >> > > > every
> >> > > > > >> >>> > > >> >> > > > >>> > > release.
> >> > > > > >> >>> > > >> >> > > > >>> > > > > The modules are a subset of
> >> > existing
> >> > > > > >> >>> integrations
> >> > > > > >> >>> > > >> that
> >> > > > > >> >>> > > >> >> are
> >> > > > > >> >>> > > >> >> > > > >>> adopted
> >> > > > > >> >>> > > >> >> > > > >>> > > widely
> >> > > > > >> >>> > > >> >> > > > >>> > > > > and require community
> >> attention.
> >> > > > > >> >>> > > >> >> > > > >>> > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > As for the independent
> >> > integrations
> >> > > > [1],
> >> > > > > >> >>> those
> >> > > > > >> >>> > are
> >> > > > > >> >>> > > >> >> supposed
> >> > > > > >> >>> > > >> >> > > to
> >> > > > > >> >>> > > >> >> > > > be
> >> > > > > >> >>> > > >> >> > > > >>> > > located
> >> > > > > >> >>> > > >> >> > > > >>> > > > > in a separate GitHub repo. I
> >> would
> >> > > > > advise
> >> > > > > >> us
> >> > > > > >> >>> to
> >> > > > > >> >>> > > start
> >> > > > > >> >>> > > >> >> > > > >>> implementing
> >> > > > > >> >>> > > >> >> > > > >>> > this
> >> > > > > >> >>> > > >> >> > > > >>> > > > > concept with the Pub/Sub
> >> > > integration.
> >> > > > > Even
> >> > > > > >> >>> though
> >> > > > > >> >>> > > it
> >> > > > > >> >>> > > >> >> will
> >> > > > > >> >>> > > >> >> > be
> >> > > > > >> >>> > > >> >> > > > >>> located
> >> > > > > >> >>> > > >> >> > > > >>> > > in a
> >> > > > > >> >>> > > >> >> > > > >>> > > > > separate Github repo, it will
> >> be
> >> > > > listed
> >> > > > > >> among
> >> > > > > >> >>> > > >> officially
> >> > > > > >> >>> > > >> >> > > > >>> available
> >> > > > > >> >>> > > >> >> > > > >>> > > > > integrations on the Ignite
> >> website
> >> > > > > >> featuring
> >> > > > > >> >>> > > >> Emmanouil
> >> > > > > >> >>> > > >> >> as a
> >> > > > > >> >>> > > >> >> > > > >>> > > contributor.
> >> > > > > >> >>> > > >> >> > > > >>> > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > What do you think?
> >> > > > > >> >>> > > >> >> > > > >>> > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > [1]
> >> > > > > >> >>> > > >> >> > > > >>> > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > >
> >> > > > > >> >>> > > >> >> > > > >>> > >
> >> > > > > >> >>> > > >> >> > > > >>> >
> >> > > > > >> >>> > > >> >> > > > >>>
> >> > > > > >> >>> > > >> >> > > >
> >> > > > > >> >>> > > >> >> > >
> >> > > > > >> >>> > > >> >> >
> >> > > > > >> >>> > > >> >>
> >> > > > > >> >>> > > >>
> >> > > > > >> >>> > >
> >> > > > > >> >>> >
> >> > > > > >> >>>
> >> > > > > >>
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >> > > > > >> >>> > > >> >> > > > >>> > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > -
> >> > > > > >> >>> > > >> >> > > > >>> > > > > Denis
> >> > > > > >> >>> > > >> >> > > > >>> > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47
> PM
> >> > > Saikat
> >> > > > > >> >>> Maitra <
> >> > > > > >> >>> > > >> >> > > > >>> > saikat.maitra@gmail.com
> >> > > > > >> >>> > > >> >> > > > >>> > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > wrote:
> >> > > > > >> >>> > > >> >> > > > >>> > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > Hi Emmanouil,
> >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > The changes looks good to
> >> me. I
> >> > > > wanted
> >> > > > > >> to
> >> > > > > >> >>> check
> >> > > > > >> >>> > > if
> >> > > > > >> >>> > > >> you
> >> > > > > >> >>> > > >> >> > are
> >> > > > > >> >>> > > >> >> > > > also
> >> > > > > >> >>> > > >> >> > > > >>> > able
> >> > > > > >> >>> > > >> >> > > > >>> > > to
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > validate that the once data
> >> is
> >> > > added
> >> > > > > to
> >> > > > > >> >>> Ignite
> >> > > > > >> >>> > > >> cluster
> >> > > > > >> >>> > > >> >> > you
> >> > > > > >> >>> > > >> >> > > > are
> >> > > > > >> >>> > > >> >> > > > >>> also
> >> > > > > >> >>> > > >> >> > > > >>> > > > able
> >> > > > > >> >>> > > >> >> > > > >>> > > > > to
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > access it using another
> >> Ignite
> >> > > > client
> >> > > > > or
> >> > > > > >> >>> using
> >> > > > > >> >>> > > rest
> >> > > > > >> >>> > > >> >> > > > endpoints.
> >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > We use Teamcity for CI
> >> process,
> >> > > the
> >> > > > > >> >>> details are
> >> > > > > >> >>> > > as
> >> > > > > >> >>> > > >> >> > follows
> >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > Trigger validation of those
> >> test
> >> > > > > suites
> >> > > > > >> >>> that
> >> > > > > >> >>> > have
> >> > > > > >> >>> > > >> been
> >> > > > > >> >>> > > >> >> > > > >>> affected by
> >> > > > > >> >>> > > >> >> > > > >>> > > your
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > changes on TeamCity <
> >> > > > > >> >>> > > http://ci.ignite.apache.org/>:
> >> > > > > >> >>> > > >> <
> >> > > > > >> >>> > > >> >> > > > >>> > > > > http://204.14.53.153/
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Locate a test suite
> you
> >> > have
> >> > > to
> >> > > > > >> check,
> >> > > > > >> >>> > press
> >> > > > > >> >>> > > >> >> button
> >> > > > > >> >>> > > >> >> > > > named
> >> > > > > >> >>> > > >> >> > > > >>> > "..."
> >> > > > > >> >>> > > >> >> > > > >>> > > > that
> >> > > > > >> >>> > > >> >> > > > >>> > > > > >    is located on the left
> of
> >> > "Run"
> >> > > > > >> button.
> >> > > > > >> >>> "Run
> >> > > > > >> >>> > > >> custom
> >> > > > > >> >>> > > >> >> > > build"
> >> > > > > >> >>> > > >> >> > > > >>> > dialog
> >> > > > > >> >>> > > >> >> > > > >>> > > > will
> >> > > > > >> >>> > > >> >> > > > >>> > > > > >    appear;
> >> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Go to "Changes" tab
> and
> >> > > choose
> >> > > > > >> >>> > > >> >> > > > >>> > "pull/<pull-request-number>/head"
> >> > > > > >> >>> > > >> >> > > > >>> > > > in
> >> > > > > >> >>> > > >> >> > > > >>> > > > > >    "Build branch" dropdown
> >> list;
> >> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Press "Run build"
> button
> >> > and
> >> > > > > >> monitor
> >> > > > > >> >>> tests
> >> > > > > >> >>> > > >> >> results.
> >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > @Roman rshtykh@yahoo.com -
> >> can
> >> > > you
> >> > > > > also
> >> > > > > >> >>> please
> >> > > > > >> >>> > > >> take a
> >> > > > > >> >>> > > >> >> > > look?
> >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > Regards,
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > Saikat
> >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at
> 6:20
> >> PM
> >> > > > > >> Emmanouil
> >> > > > > >> >>> > > >> Gkatziouras
> >> > > > > >> >>> > > >> >> <
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > gkatzioura@gmail.com>
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > wrote:
> >> > > > > >> >>> > > >> >> > > > >>> > > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Hi @Saikat Maitra
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > > I have my implementation
> >> on an
> >> > > > > Ignite
> >> > > > > >> >>> fork on
> >> > > > > >> >>> > > my
> >> > > > > >> >>> > > >> >> > personal
> >> > > > > >> >>> > > >> >> > > > >>> GitHub
> >> > > > > >> >>> > > >> >> > > > >>> > > > > account
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > > [1]
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > > It is based on the Kafka
> >> > > > > >> implementation
> >> > > > > >> >>> which
> >> > > > > >> >>> > > is
> >> > > > > >> >>> > > >> of
> >> > > > > >> >>> > > >> >> a
> >> > > > > >> >>> > > >> >> > > > polling
> >> > > > > >> >>> > > >> >> > > > >>> > > nature.
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > > GCP does not provide a
> >> service
> >> > > > like
> >> > > > > >> >>> Kafka.
> >> > > > > >> >>> > > >> Instead
> >> > > > > >> >>> > > >> >> it
> >> > > > > >> >>> > > >> >> > > > >>> provides us
> >> > > > > >> >>> > > >> >> > > > >>> > > > with
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Pub/Sub which is just
> like
> >> > > Kafka.
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Therefore using the Kafka
> >> > > streamer
> >> > > > > as
> >> > > > > >> a
> >> > > > > >> >>> > > guideline
> >> > > > > >> >>> > > >> >> made
> >> > > > > >> >>> > > >> >> > > more
> >> > > > > >> >>> > > >> >> > > > >>> sense
> >> > > > > >> >>> > > >> >> > > > >>> > > to
> >> > > > > >> >>> > > >> >> > > > >>> > > > > me.
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > >
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > > Apart from the
> >> implementation
> >> > I
> >> > > > have
> >> > > > > >> also
> >> > > > > >> >>> > > added a
> >> > > > > >> >>> > > >> >> unit
> >> > > > > >> >>> > > >> >> > > > test.
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > > It definitely needs some
> >> > > polishing
> >> > > > > >> and a
> >> > > > > >> >>> > double
> >> > > > > >> >>> > > >> >> check
> >> > > > > >> >>> > > >> >> > on
> >> > > > > >> >>> > > >> >> > > my
> >> > > > > >> >>> > > >> >> > > > >>> side
> >> > > > > >> >>> > > >> >> > > > >>> > to
> >> > > > > >> >>> > > >> >> > > > >>> > > > > make
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > > the review smooth.
> >> > > > > >> >>> > > >> >> > > > >>> > > > > > > For the tests a local
> >> Pub/Sub
> >> > > [2]
> >> > > > > >> server
> >> > > >
> >
> >
>

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra <sa...@gmail.com>]

Hello,

I have made changes in Ignite Extensions build configurations.

It is now running the tests as expected.

https://ci.ignite.apache.org/viewLog.html?buildId=4804480&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead

Regards,
Saikat

On Thu, Nov 28, 2019 at 1:29 PM Saikat Maitra <sa...@gmail.com>
wrote:

> Hi,
>
> We do run build with -DskipTests and then configure selective test suites
> to run.
>
> I will check why these test suites are not getting executed.
>
> Regards
> Saikat
>
> On Wed, 27 Nov 2019 at 3:05 PM, Emmanouil Gkatziouras <
> gkatzioura@gmail.com> wrote:
>
>> Hi all,
>>
>> I did add a test suite and run the build with the corresponding arguments,
>> however the tests did not run [1].
>> I checked the Flink's build logs and the same message is displayed there
>> too `No tests to run` [2].
>>
>> [1]
>>
>> https://ci.ignite.apache.org/viewLog.html?buildId=4798203&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=248
>> [2]
>>
>> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4789199&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=1256
>>
>> Kind regards
>> *Emmanouil Gkatziouras*
>> https://egkatzioura.com/ |
>> https://www.linkedin.com/in/gkatziourasemmanouil/
>> https://github.com/gkatzioura
>>
>>
>> On Wed, 27 Nov 2019 at 18:58, Saikat Maitra <sa...@gmail.com>
>> wrote:
>>
>> > Hi,
>> >
>> > We will need to add test suites for tests to be executed in build. We
>> pass
>> > them as TestSuites param. I had added Flink sink and source testsuites.
>> >
>> > Can you please review and confirm.
>> >
>> > Regards
>> > Saikat
>> >
>> > On Wed, 27 Nov 2019 at 6:00 AM, Emmanouil Gkatziouras <
>> > gkatzioura@gmail.com>
>> > wrote:
>> >
>> > > Hi all!
>> > >
>> > > I was successful building and running my pull requests. It seems that
>> the
>> > > tests do not run both for Flink and Pub/Sub [1]
>> > > If there is something I cannot do to make them discoverable please
>> let me
>> > > know.
>> > >
>> > > [1]
>> > >
>> > >
>> >
>> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4796722&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=283
>> > >
>> > > *Emmanouil Gkatziouras*
>> > > https://egkatzioura.com/ |
>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > https://github.com/gkatzioura
>> > >
>> > >
>> > > On Wed, 27 Nov 2019 at 04:57, Saikat Maitra <sa...@gmail.com>
>> > > wrote:
>> > >
>> > > > Hi Emmanouil,
>> > > >
>> > > > I have added you as contributor in Ignite Extensions project. Can
>> you
>> > > > please check and confirm if you are able to see the project and
>> execute
>> > > > build on your pull request.
>> > > >
>> > > > Regards,
>> > > > Saikat
>> > > >
>> > > > On Tue, Nov 26, 2019 at 12:49 PM Emmanouil Gkatziouras <
>> > > > gkatzioura@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > Hi all!
>> > > > >
>> > > > > If someone could give me read access to the Job of Ignite
>> Extensions
>> > on
>> > > > > Team City will greatly help me (username gkatzioura).
>> > > > > I suppose the builds get triggered automatically on a pull
>> request.
>> > > > >
>> > > > > Kind regards,
>> > > > > Emmanouil
>> > > > >
>> > > > > *Emmanouil Gkatziouras*
>> > > > > https://egkatzioura.com/ |
>> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > > https://github.com/gkatzioura
>> > > > >
>> > > > >
>> > > > > On Sun, 24 Nov 2019 at 21:11, Emmanouil Gkatziouras <
>> > > > gkatzioura@gmail.com>
>> > > > > wrote:
>> > > > >
>> > > > > > Hi Saikat!
>> > > > > >
>> > > > > > I just rebased with the flink branch. Unfortunately I do not
>> have
>> > > read
>> > > > > > access to the team city link you provided and thus evaluate the
>> > tests
>> > > > for
>> > > > > > my pull request.
>> > > > > > I guess it has to do with ignite extensions being new.
>> > > > > > My username is gkatzioura and email the one I am using on this
>> > email
>> > > > > >
>> > > > > > Thank you for your time.
>> > > > > >
>> > > > > >
>> > > > > > *Emmanouil Gkatziouras*
>> > > > > > https://egkatzioura.com/ |
>> > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > > > https://github.com/gkatzioura
>> > > > > >
>> > > > > >
>> > > > > > On Sun, 24 Nov 2019 at 18:03, Saikat Maitra <
>> > saikat.maitra@gmail.com
>> > > >
>> > > > > > wrote:
>> > > > > >
>> > > > > >> Hi Emmanouil,
>> > > > > >>
>> > > > > >> The latest build on teamcity has passed on Flink pull request.
>> > > > > >>
>> > > > > >>
>> > > > > >>
>> > > > >
>> > > >
>> > >
>> >
>> https://ci.ignite.apache.org/viewLog.html?buildId=4788928&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
>> > > > > >>
>> > > > > >> you should be able to take the changes I made in my PR and run
>> > build
>> > > > on
>> > > > > >> your pull request.
>> > > > > >>
>> > > > > >> Please let me know if you have any questions.
>> > > > > >>
>> > > > > >> Regards,
>> > > > > >> Saikat
>> > > > > >>
>> > > > > >> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra <
>> > > > saikat.maitra@gmail.com
>> > > > > >
>> > > > > >> wrote:
>> > > > > >>
>> > > > > >> > Hi Emmanouil,
>> > > > > >> >
>> > > > > >> > I have fixed the Flink module testsuites and have setup new
>> > > project
>> > > > in
>> > > > > >> > teamcity.
>> > > > > >> >
>> > > > > >> >
>> > > > > >> >
>> > > > > >>
>> > > > >
>> > > >
>> > >
>> >
>> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview
>> > > > > >> >
>> > > > > >> > I am looking into setting up the build.
>> > > > > >> >
>> > > > > >> > Regards,
>> > > > > >> > Saikat
>> > > > > >> >
>> > > > > >> > On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra <
>> > > > > saikat.maitra@gmail.com>
>> > > > > >> > wrote:
>> > > > > >> >
>> > > > > >> >> Hi Emmanouil,
>> > > > > >> >>
>> > > > > >> >> Thank you for your email. Yes, the plan is once the Flink
>> PR is
>> > > > > merged
>> > > > > >> it
>> > > > > >> >> will provide the Licence, parent POMs etc and you can rebase
>> > from
>> > > > > >> master
>> > > > > >> >> branch and apply the changes on top of it.
>> > > > > >> >>
>> > > > > >> >> I am getting some issues with test failures in local with
>> > > > > >> >> GridTestProperties as the test.properties is not present in
>> > this
>> > > > new
>> > > > > >> >> project but should be available from dependencies.
>> > > > > >> >>
>> > > > > >> >> Once I address this issue, I will go ahead and merge the
>> > changes
>> > > > and
>> > > > > >> then
>> > > > > >> >> we can take it from there.
>> > > > > >> >>
>> > > > > >> >> Regards,
>> > > > > >> >> Saikat
>> > > > > >> >>
>> > > > > >> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras <
>> > > > > >> >> gkatzioura@gmail.com> wrote:
>> > > > > >> >>
>> > > > > >> >>> Hi all,
>> > > > > >> >>>
>> > > > > >> >>> I made my first pull request [1]. Since this project is
>> brand
>> > > new
>> > > > > (no
>> > > > > >> >>> parent poms, licensing), I reckoned it was best to use
>> > Saikat's
>> > > > > >> branch on
>> > > > > >> >>> Flink.
>> > > > > >> >>> I suppose the Flink branch will be merged first. If not
>> please
>> > > > give
>> > > > > me
>> > > > > >> >>> guidelines on how I should proceed next.
>> > > > > >> >>>
>> > > > > >> >>> Kind regards
>> > > > > >> >>> Emmanouil
>> > > > > >> >>>
>> > > > > >> >>> [1] https://github.com/apache/ignite-extensions/pull/2
>> > > > > >> >>>
>> > > > > >> >>> *Emmanouil Gkatziouras*
>> > > > > >> >>> https://egkatzioura.com/ |
>> > > > > >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > > >> >>> https://github.com/gkatzioura
>> > > > > >> >>>
>> > > > > >> >>>
>> > > > > >> >>> On Fri, 22 Nov 2019 at 20:55, Denis Magda <
>> dmagda@apache.org>
>> > > > > wrote:
>> > > > > >> >>>
>> > > > > >> >>> > Awesome, ping us whenever you're ready!
>> > > > > >> >>> >
>> > > > > >> >>> > -
>> > > > > >> >>> > Denis
>> > > > > >> >>> >
>> > > > > >> >>> >
>> > > > > >> >>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <
>> > > > > >> >>> > gkatzioura@gmail.com>
>> > > > > >> >>> > wrote:
>> > > > > >> >>> >
>> > > > > >> >>> > > Hi all!
>> > > > > >> >>> > >
>> > > > > >> >>> > > I am sorry for being late on that. I was trying to
>> > refactor
>> > > > the
>> > > > > >> test
>> > > > > >> >>> in
>> > > > > >> >>> > > order not to be in need of any external tools or
>> spinning
>> > > up a
>> > > > > >> >>> server.
>> > > > > >> >>> > > I did forked the new repo and indeed my changes there,
>> so
>> > a
>> > > > pull
>> > > > > >> >>> request
>> > > > > >> >>> > is
>> > > > > >> >>> > > a matter of time!
>> > > > > >> >>> > >
>> > > > > >> >>> > > Kind regards
>> > > > > >> >>> > >
>> > > > > >> >>> > > *Emmanouil Gkatziouras*
>> > > > > >> >>> > > https://egkatzioura.com/ |
>> > > > > >> >>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > > >> >>> > > https://github.com/gkatzioura
>> > > > > >> >>> > >
>> > > > > >> >>> > >
>> > > > > >> >>> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda <
>> > > dmagda@apache.org>
>> > > > > >> wrote:
>> > > > > >> >>> > >
>> > > > > >> >>> > > > Hi Emmanouil,
>> > > > > >> >>> > > >
>> > > > > >> >>> > > > Do you have any questions or need any support from
>> the
>> > > > > >> community?
>> > > > > >> >>> > > >
>> > > > > >> >>> > > > -
>> > > > > >> >>> > > > Denis
>> > > > > >> >>> > > >
>> > > > > >> >>> > > >
>> > > > > >> >>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <
>> > > > > >> >>> saikat.maitra@gmail.com
>> > > > > >> >>> > >
>> > > > > >> >>> > > > wrote:
>> > > > > >> >>> > > >
>> > > > > >> >>> > > >> Hi Emmanouil,
>> > > > > >> >>> > > >>
>> > > > > >> >>> > > >> Can you please take a looks at dev utils, if this is
>> > > > > something
>> > > > > >> >>> you are
>> > > > > >> >>> > > >> looking for
>> > > > > >> >>> > > >>
>> > > > > https://github.com/apache/ignite/tree/master/modules/dev-utils
>> > > > > >> ?
>> > > > > >> >>> > > >>
>> > > > > >> >>> > > >> IMO, if you can release Pub/Sub server in maven and
>> > then
>> > > > use
>> > > > > >> it as
>> > > > > >> >>> > > >> dependency, that would be great.
>> > > > > >> >>> > > >>
>> > > > > >> >>> > > >> Regards,
>> > > > > >> >>> > > >> Saikat
>> > > > > >> >>> > > >>
>> > > > > >> >>> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <
>> > > > > >> >>> > saikat.maitra@gmail.com>
>> > > > > >> >>> > > >> wrote:
>> > > > > >> >>> > > >>
>> > > > > >> >>> > > >> > Hi Emmanouil,
>> > > > > >> >>> > > >> >
>> > > > > >> >>> > > >> > The master branch has been initialized. I have
>> > raised a
>> > > > PR
>> > > > > to
>> > > > > >> >>> > migrate
>> > > > > >> >>> > > >> > Flink module to ignite-extensions.
>> > > > > >> >>> > > >> >
>> > > > > >> >>> > > >> > PR
>> > https://github.com/apache/ignite-extensions/pull/1
>> > > > > >> >>> > > >> > Jira :
>> > > > https://issues.apache.org/jira/browse/IGNITE-12356
>> > > > > >> >>> > > >> >
>> > > > > >> >>> > > >> > Regards,
>> > > > > >> >>> > > >> > Saikat
>> > > > > >> >>> > > >> >
>> > > > > >> >>> > > >> >
>> > > > > >> >>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil
>> > Gkatziouras <
>> > > > > >> >>> > > >> > gkatzioura@gmail.com> wrote:
>> > > > > >> >>> > > >> >
>> > > > > >> >>> > > >> >> Hi all!
>> > > > > >> >>> > > >> >>
>> > > > > >> >>> > > >> >> This is an update. The integration is tested with
>> > > mocks
>> > > > > >> based
>> > > > > >> >>> on
>> > > > > >> >>> > the
>> > > > > >> >>> > > >> >> original Ignite repo [1].
>> > > > > >> >>> > > >> >> I am currently waiting for the first commits on
>> the
>> > > new
>> > > > > >> >>> integration
>> > > > > >> >>> > > >> >> project
>> > > > > >> >>> > > >> >> to take place in order to create my pull request
>> > > > > accordingly
>> > > > > >> >>> to the
>> > > > > >> >>> > > >> >> initial
>> > > > > >> >>> > > >> >> commits.
>> > > > > >> >>> > > >> >> In order to simulate the Pub/Sub server in a
>> test I
>> > > > used a
>> > > > > >> test
>> > > > > >> >>> > > server
>> > > > > >> >>> > > >> >> implementation [2]. Unfortunately they don't have
>> > this
>> > > > > util
>> > > > > >> >>> > uploaded
>> > > > > >> >>> > > >> in a
>> > > > > >> >>> > > >> >> maven repo, thus it needs to be hosted on a maven
>> > repo
>> > > > > >> (Apache
>> > > > > >> >>> > > License
>> > > > > >> >>> > > >> >> 2.0). Is there a repo for utilities that Ignite
>> uses
>> > > for
>> > > > > the
>> > > > > >> >>> > > >> test/build?
>> > > > > >> >>> > > >> >> I have also created a ticket [3]
>> > > > > >> >>> > > >> >>
>> > > > > >> >>> > > >> >> Kind regards
>> > > > > >> >>> > > >> >>
>> > > > > >> >>> > > >> >> [1]
>> > > > > >> >>> >
>> > > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
>> > > > > >> >>> > > >> >> [2]
>> > > > > >> >>>
>> https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
>> > > > > >> >>> > > >> >> [3]
>> > > https://issues.apache.org/jira/browse/IGNITE-12262
>> > > > > >> >>> > > >> >>
>> > > > > >> >>> > > >> >> *Emmanouil Gkatziouras*
>> > > > > >> >>> > > >> >> https://egkatzioura.com/ |
>> > > > > >> >>> > > >> >>
>> https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > > >> >>> > > >> >> https://github.com/gkatzioura
>> > > > > >> >>> > > >> >>
>> > > > > >> >>> > > >> >>
>> > > > > >> >>> > > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <
>> > > > > >> >>> > saikat.maitra@gmail.com
>> > > > > >> >>> > > >
>> > > > > >> >>> > > >> >> wrote:
>> > > > > >> >>> > > >> >>
>> > > > > >> >>> > > >> >> > Hello Ilya,
>> > > > > >> >>> > > >> >> >
>> > > > > >> >>> > > >> >> > Thank you for your email. Yes, before we remove
>> > any
>> > > > > >> >>> integration
>> > > > > >> >>> > we
>> > > > > >> >>> > > >> will
>> > > > > >> >>> > > >> >> > move them to another repository and release.
>> > > > > >> >>> > > >> >> >
>> > > > > >> >>> > > >> >> > We are thinking we will start with Pub/Sub
>> > Streamer
>> > > > > >> >>> > implementation
>> > > > > >> >>> > > >> and
>> > > > > >> >>> > > >> >> > continue to move other integration as
>> mentioned in
>> > > the
>> > > > > >> >>> following
>> > > > > >> >>> > > doc.
>> > > > > >> >>> > > >> >> >
>> > > > > >> >>> > > >> >> >
>> > > > > >> >>> > > >> >> >
>> > > > > >> >>> > > >> >>
>> > > > > >> >>> > > >>
>> > > > > >> >>> > >
>> > > > > >> >>> >
>> > > > > >> >>>
>> > > > > >>
>> > > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> > > > > >> >>> > > >> >> >
>> > > > > >> >>> > > >> >> > Regards,
>> > > > > >> >>> > > >> >> > Saikat
>> > > > > >> >>> > > >> >> >
>> > > > > >> >>> > > >> >> >
>> > > > > >> >>> > > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya
>> Kasnacheev <
>> > > > > >> >>> > > >> >> > ilya.kasnacheev@gmail.com>
>> > > > > >> >>> > > >> >> > wrote:
>> > > > > >> >>> > > >> >> >
>> > > > > >> >>> > > >> >> > > Hello!
>> > > > > >> >>> > > >> >> > >
>> > > > > >> >>> > > >> >> > > My take on this: we want to offload some of
>> > > > > >> integrations,
>> > > > > >> >>> but
>> > > > > >> >>> > > >> please
>> > > > > >> >>> > > >> >> > commit
>> > > > > >> >>> > > >> >> > > and release them to another project first,
>> and
>> > > only
>> > > > > >> then we
>> > > > > >> >>> > will
>> > > > > >> >>> > > >> >> decide
>> > > > > >> >>> > > >> >> > to
>> > > > > >> >>> > > >> >> > > remove them from our repository.
>> > > > > >> >>> > > >> >> > >
>> > > > > >> >>> > > >> >> > > Please also make sure they are available with
>> > same
>> > > > > >> artifact
>> > > > > >> >>> > name
>> > > > > >> >>> > > >> and
>> > > > > >> >>> > > >> >> API.
>> > > > > >> >>> > > >> >> > > If this has to be changed, let's wait for AI
>> 3.0
>> > > > > >> >>> > > >> >> > >
>> > > > > >> >>> > > >> >> > > Regards,
>> > > > > >> >>> > > >> >> > > --
>> > > > > >> >>> > > >> >> > > Ilya Kasnacheev
>> > > > > >> >>> > > >> >> > >
>> > > > > >> >>> > > >> >> > >
>> > > > > >> >>> > > >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <
>> > > > > >> >>> > > >> saikat.maitra@gmail.com>:
>> > > > > >> >>> > > >> >> > >
>> > > > > >> >>> > > >> >> > > > Hello Emmanouil,
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > > > As discussed earlier, I discussed with
>> Apache
>> > > > Bahir
>> > > > > >> >>> community
>> > > > > >> >>> > > and
>> > > > > >> >>> > > >> >> they
>> > > > > >> >>> > > >> >> > > are
>> > > > > >> >>> > > >> >> > > > interested to have Apache Ignite
>> extensions as
>> > > > part
>> > > > > of
>> > > > > >> >>> Apache
>> > > > > >> >>> > > >> Bahir
>> > > > > >> >>> > > >> >> > > > project.
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > > > Can you please share Pub/Sub streamer
>> > > > implementation
>> > > > > >> >>> details
>> > > > > >> >>> > > with
>> > > > > >> >>> > > >> >> > Apache
>> > > > > >> >>> > > >> >> > > > Bahir community and request for feedback.
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > > > https://bahir.apache.org/
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > > > I have also requested for contributor
>> access
>> > in
>> > > > Jira
>> > > > > >> for
>> > > > > >> >>> > Apache
>> > > > > >> >>> > > >> >> Bahir
>> > > > > >> >>> > > >> >> > > > project so that I can create issues and
>> assign
>> > > to
>> > > > > >> myself.
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > > > I can help with code reviews as well.
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > > > Here is the email thread for reference
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > >
>> > > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > > > Regards,
>> > > > > >> >>> > > >> >> > > > Saikat
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat
>> Maitra
>> > <
>> > > > > >> >>> > > >> >> saikat.maitra@gmail.com
>> > > > > >> >>> > > >> >> > >
>> > > > > >> >>> > > >> >> > > > wrote:
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > > > > Hello,
>> > > > > >> >>> > > >> >> > > > >
>> > > > > >> >>> > > >> >> > > > > As discussed I have created following
>> > > discussion
>> > > > > >> >>> threads on
>> > > > > >> >>> > > our
>> > > > > >> >>> > > >> >> > > migration
>> > > > > >> >>> > > >> >> > > > > proposals for Apache Ignite extensions:
>> > > > > >> >>> > > >> >> > > > >
>> > > > > >> >>> > > >> >> > > > >
>> > > > > >> >>> > > >> >> > > > >
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > >
>> > > > > >> >>> > > >> >> >
>> > > > > >> >>> > > >> >>
>> > > > > >> >>> > > >>
>> > > > > >> >>> > >
>> > > > > >> >>> >
>> > > > > >> >>>
>> > > > > >>
>> > > > >
>> > > >
>> > >
>> >
>> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
>> > > > > >> >>> > > >> >> > > > >
>> > > > > >> >>> > > >> >> > > > >
>> > > > > >> >>> > > >>
>> > > > > >>
>> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
>> > > > > >> >>> > > >> >> > > > >
>> > > > > >> >>> > > >> >> > > > > I will share update as I hear more
>> > > information.
>> > > > > >> >>> > > >> >> > > > >
>> > > > > >> >>> > > >> >> > > > > Regards,
>> > > > > >> >>> > > >> >> > > > > Saikat
>> > > > > >> >>> > > >> >> > > > >
>> > > > > >> >>> > > >> >> > > > >
>> > > > > >> >>> > > >> >> > > > >
>> > > > > >> >>> > > >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat
>> > Maitra
>> > > <
>> > > > > >> >>> > > >> >> > saikat.maitra@gmail.com
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > > > > wrote:
>> > > > > >> >>> > > >> >> > > > >
>> > > > > >> >>> > > >> >> > > > >> Hello Denis,
>> > > > > >> >>> > > >> >> > > > >>
>> > > > > >> >>> > > >> >> > > > >> Sure, sounds good. I will create a
>> separate
>> > > > > >> discussion
>> > > > > >> >>> > > thread
>> > > > > >> >>> > > >> to
>> > > > > >> >>> > > >> >> get
>> > > > > >> >>> > > >> >> > > > >> community feedback on whether we
>> > > > > >> >>> > > >> >> > > > >> should join the Bahir or kick-off
>> "Ignite
>> > > > > >> Extensions"
>> > > > > >> >>> > > project.
>> > > > > >> >>> > > >> >> > > > >>
>> > > > > >> >>> > > >> >> > > > >> Regards,
>> > > > > >> >>> > > >> >> > > > >> Saikat
>> > > > > >> >>> > > >> >> > > > >>
>> > > > > >> >>> > > >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis
>> > Magda <
>> > > > > >> >>> > > >> dmagda@apache.org>
>> > > > > >> >>> > > >> >> > > wrote:
>> > > > > >> >>> > > >> >> > > > >>
>> > > > > >> >>> > > >> >> > > > >>> Folks,
>> > > > > >> >>> > > >> >> > > > >>>
>> > > > > >> >>> > > >> >> > > > >>> The concept of Apache Bahir is
>> precisely
>> > > what
>> > > > we
>> > > > > >> >>> need!
>> > > > > >> >>> > > >> Saikat,
>> > > > > >> >>> > > >> >> > thanks
>> > > > > >> >>> > > >> >> > > > for
>> > > > > >> >>> > > >> >> > > > >>> doing the research. Why I believe the *
>> > > idea*
>> > > > > >> fits us
>> > > > > >> >>> > well:
>> > > > > >> >>> > > >> >> > > > >>>
>> > > > > >> >>> > > >> >> > > > >>>    - All integrations can be stored in
>> > > > separate
>> > > > > >> >>> Github
>> > > > > >> >>> > > >> >> repositories
>> > > > > >> >>> > > >> >> > > and
>> > > > > >> >>> > > >> >> > > > >>>    have their dev lifecycles. We've not
>> > > > obliged
>> > > > > to
>> > > > > >> >>> couple
>> > > > > >> >>> > > all
>> > > > > >> >>> > > >> >> the
>> > > > > >> >>> > > >> >> > > > >>> integrations
>> > > > > >> >>> > > >> >> > > > >>>    in a single repo. For instance,
>> Spark
>> > can
>> > > > be
>> > > > > >> >>> located
>> > > > > >> >>> > in
>> > > > > >> >>> > > a
>> > > > > >> >>> > > >> >> > > dedicated
>> > > > > >> >>> > > >> >> > > > >>> repo
>> > > > > >> >>> > > >> >> > > > >>>    while streaming integrations might
>> be
>> > in
>> > > a
>> > > > > >> single
>> > > > > >> >>> one.
>> > > > > >> >>> > > >> It's
>> > > > > >> >>> > > >> >> up
>> > > > > >> >>> > > >> >> > to
>> > > > > >> >>> > > >> >> > > > us.
>> > > > > >> >>> > > >> >> > > > >>>    - All the repositories and
>> integrations
>> > > > > belong
>> > > > > >> to
>> > > > > >> >>> ASF.
>> > > > > >> >>> > > >> We're
>> > > > > >> >>> > > >> >> not
>> > > > > >> >>> > > >> >> > > > >>> dumping
>> > > > > >> >>> > > >> >> > > > >>>    them on Github but rather keep
>> > supporting
>> > > > and
>> > > > > >> >>> > developing
>> > > > > >> >>> > > >> in
>> > > > > >> >>> > > >> >> > > > >>> accordance with
>> > > > > >> >>> > > >> >> > > > >>>    ASF vision and practices.
>> > > > > >> >>> > > >> >> > > > >>>    - There is a way to reward
>> contributors
>> > > via
>> > > > > >> >>> > > committership
>> > > > > >> >>> > > >> and
>> > > > > >> >>> > > >> >> > PMC
>> > > > > >> >>> > > >> >> > > > >>>    membership. You won't get this if a
>> > > project
>> > > > > is
>> > > > > >> >>> just
>> > > > > >> >>> > one
>> > > > > >> >>> > > of
>> > > > > >> >>> > > >> >> the
>> > > > > >> >>> > > >> >> > > > >>> millions of
>> > > > > >> >>> > > >> >> > > > >>>    Github projects.
>> > > > > >> >>> > > >> >> > > > >>>
>> > > > > >> >>> > > >> >> > > > >>> Saikat, as Ignite PMC member, could you
>> > > please
>> > > > > >> >>> kick-off a
>> > > > > >> >>> > > >> >> separate
>> > > > > >> >>> > > >> >> > > > >>> dev-list
>> > > > > >> >>> > > >> >> > > > >>> discussion to involve more community
>> > members
>> > > > and
>> > > > > >> >>> > referring
>> > > > > >> >>> > > to
>> > > > > >> >>> > > >> >> this
>> > > > > >> >>> > > >> >> > > > >>> thread?
>> > > > > >> >>> > > >> >> > > > >>> I think the primary question the
>> community
>> > > > needs
>> > > > > >> to
>> > > > > >> >>> > answer
>> > > > > >> >>> > > >> >> whether
>> > > > > >> >>> > > >> >> > we
>> > > > > >> >>> > > >> >> > > > >>> should join the Bahir or kick-off
>> "Ignite
>> > > > > >> Extensions"
>> > > > > >> >>> > > >> project?
>> > > > > >> >>> > > >> >> > > > >>>
>> > > > > >> >>> > > >> >> > > > >>> -
>> > > > > >> >>> > > >> >> > > > >>> Denis
>> > > > > >> >>> > > >> >> > > > >>>
>> > > > > >> >>> > > >> >> > > > >>>
>> > > > > >> >>> > > >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey
>> > > > Zinoviev
>> > > > > <
>> > > > > >> >>> > > >> >> > > > zaleslaw.sin@gmail.com>
>> > > > > >> >>> > > >> >> > > > >>> wrote:
>> > > > > >> >>> > > >> >> > > > >>>
>> > > > > >> >>> > > >> >> > > > >>> > Maybe we could move all our Streaming
>> > > > > >> Integrations
>> > > > > >> >>> > there,
>> > > > > >> >>> > > >> but
>> > > > > >> >>> > > >> >> > what
>> > > > > >> >>> > > >> >> > > is
>> > > > > >> >>> > > >> >> > > > >>> about
>> > > > > >> >>> > > >> >> > > > >>> > maintaining and committer
>> permissions to
>> > > the
>> > > > > new
>> > > > > >> >>> > > >> repositories?
>> > > > > >> >>> > > >> >> > > > >>> > I see the list of the committers and
>> PMC
>> > > > > members
>> > > > > >> >>> there
>> > > > > >> >>> > > >> >> > > > >>> >
>> > > https://bahir.apache.org/community-members/
>> > > > > >> >>> > > >> >> > > > >>> >
>> > > > > >> >>> > > >> >> > > > >>> > Could somebody from Ignite community
>> be
>> > > > added
>> > > > > to
>> > > > > >> >>> this
>> > > > > >> >>> > > list
>> > > > > >> >>> > > >> as
>> > > > > >> >>> > > >> >> a
>> > > > > >> >>> > > >> >> > > > >>> > PMC/committer to maintain Ignite
>> > > > integrations?
>> > > > > >> >>> > > >> >> > > > >>> > If yes, I'd happy to join this
>> project
>> > and
>> > > > > >> >>> collaborate
>> > > > > >> >>> > > with
>> > > > > >> >>> > > >> >> these
>> > > > > >> >>> > > >> >> > > > guys
>> > > > > >> >>> > > >> >> > > > >>> > together
>> > > > > >> >>> > > >> >> > > > >>> > If no, and we should start from the
>> zero
>> > > > level
>> > > > > >> with
>> > > > > >> >>> > > >> external
>> > > > > >> >>> > > >> >> PRs
>> > > > > >> >>> > > >> >> > -
>> > > > > >> >>> > > >> >> > > > >>> hmmm,
>> > > > > >> >>> > > >> >> > > > >>> > it's better to have our own external
>> > > > > repository
>> > > > > >> >>> with
>> > > > > >> >>> > > >> >> ApacheBahirr
>> > > > > >> >>> > > >> >> > > > >>> ideology.
>> > > > > >> >>> > > >> >> > > > >>> >
>> > > > > >> >>> > > >> >> > > > >>> > Also, I agree, that all connectors
>> could
>> > > be
>> > > > > >> moved
>> > > > > >> >>> there
>> > > > > >> >>> > > (in
>> > > > > >> >>> > > >> >> > > > >>> ApacheBahirr
>> > > > > >> >>> > > >> >> > > > >>> > like repository), but not all modules
>> > from
>> > > > the
>> > > > > >> page
>> > > > > >> >>> > > >> >> > > > >>> >
>> > > > > >> >>> > > >> >> > > > >>> >
>> > > > > >> >>> > > >> >> > > > >>>
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > >
>> > > > > >> >>> > > >> >> >
>> > > > > >> >>> > > >> >>
>> > > > > >> >>> > > >>
>> > > > > >> >>> > >
>> > > > > >> >>> >
>> > > > > >> >>>
>> > > > > >>
>> > > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> > > > > >> >>> > > >> >> > > > >>> > because
>> > > > > >> >>> > > >> >> > > > >>> > they use different parts of core
>> modules
>> > > and
>> > > > > >> could
>> > > > > >> >>> be
>> > > > > >> >>> > > >> >> developed
>> > > > > >> >>> > > >> >> > > with
>> > > > > >> >>> > > >> >> > > > >>> > different velocity.
>> > > > > >> >>> > > >> >> > > > >>> >
>> > > > > >> >>> > > >> >> > > > >>> > In reality, before creation of new
>> > > > > repositories
>> > > > > >> we
>> > > > > >> >>> need
>> > > > > >> >>> > > >> wide
>> > > > > >> >>> > > >> >> > > > discussion
>> > > > > >> >>> > > >> >> > > > >>> > based on the document mentioned
>> above.
>> > > > > >> >>> > > >> >> > > > >>> >
>> > > > > >> >>> > > >> >> > > > >>> > P.S. Of course, we could experiment
>> in
>> > the
>> > > > > next
>> > > > > >> >>> release
>> > > > > >> >>> > > 2.8
>> > > > > >> >>> > > >> >> with
>> > > > > >> >>> > > >> >> > > one
>> > > > > >> >>> > > >> >> > > > >>> or two
>> > > > > >> >>> > > >> >> > > > >>> > integrations like twitter or ZeroMQ
>> > > > > >> >>> > > >> >> > > > >>> > Also, I'd like Denis Magda idea for
>> the
>> > > > > separate
>> > > > > >> >>> > > >> repositories
>> > > > > >> >>> > > >> >> in
>> > > > > >> >>> > > >> >> > > > Apache
>> > > > > >> >>> > > >> >> > > > >>> > Github to maintain them by anybody.
>> > > > > >> >>> > > >> >> > > > >>> >
>> > > > > >> >>> > > >> >> > > > >>> >
>> > > > > >> >>> > > >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat
>> > > Maitra <
>> > > > > >> >>> > > >> >> > > saikat.maitra@gmail.com
>> > > > > >> >>> > > >> >> > > > >:
>> > > > > >> >>> > > >> >> > > > >>> >
>> > > > > >> >>> > > >> >> > > > >>> > > Hi Denis, Emmanouil
>> > > > > >> >>> > > >> >> > > > >>> > >
>> > > > > >> >>> > > >> >> > > > >>> > > Thank you for your email. I think
>> > > > creating a
>> > > > > >> >>> separate
>> > > > > >> >>> > > >> >> > integration
>> > > > > >> >>> > > >> >> > > > >>> project
>> > > > > >> >>> > > >> >> > > > >>> > > in github and also proposing it as
>> an
>> > > > apache
>> > > > > >> >>> > incubator
>> > > > > >> >>> > > >> >> project
>> > > > > >> >>> > > >> >> > > will
>> > > > > >> >>> > > >> >> > > > >>> be a
>> > > > > >> >>> > > >> >> > > > >>> > > good move. The other separate
>> > > integration
>> > > > > >> modules
>> > > > > >> >>> > > >> >> > > > >>> > >
>> > > > > >> >>> > > >> >> > > > >>> > >
>> > > > > >> >>> > > >> >> > > > >>> >
>> > > > > >> >>> > > >> >> > > > >>>
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > >
>> > > > > >> >>> > > >> >> >
>> > > > > >> >>> > > >> >>
>> > > > > >> >>> > > >>
>> > > > > >> >>> > >
>> > > > > >> >>> >
>> > > > > >> >>>
>> > > > > >>
>> > > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> > > > > >> >>> > > >> >> > > > >>> > > can
>> > > > > >> >>> > > >> >> > > > >>> > > be moved to this new apache
>> incubator
>> > > > > project.
>> > > > > >> >>> > > >> >> > > > >>> > >
>> > > > > >> >>> > > >> >> > > > >>> > > There are similar integration
>> > available
>> > > > for
>> > > > > >> >>> Flink and
>> > > > > >> >>> > > >> Spark
>> > > > > >> >>> > > >> >> in
>> > > > > >> >>> > > >> >> > > > Apache
>> > > > > >> >>> > > >> >> > > > >>> > Bahir
>> > > > > >> >>> > > >> >> > > > >>> > > https://bahir.apache.org/
>> > > > > >> >>> > > >> >> > > > >>> > >
>> > > > > >> >>> > > >> >> > > > >>> > > Do you think we should reach out to
>> > > Apache
>> > > > > >> Bahir
>> > > > > >> >>> > > >> community
>> > > > > >> >>> > > >> >> > with a
>> > > > > >> >>> > > >> >> > > > >>> > proposal
>> > > > > >> >>> > > >> >> > > > >>> > > or we should plan to create a
>> separate
>> > > > > Apache
>> > > > > >> >>> > Incubator
>> > > > > >> >>> > > >> >> > project?
>> > > > > >> >>> > > >> >> > > > >>> > >
>> > > > > >> >>> > > >> >> > > > >>> > > Regards,
>> > > > > >> >>> > > >> >> > > > >>> > > Saikat
>> > > > > >> >>> > > >> >> > > > >>> > >
>> > > > > >> >>> > > >> >> > > > >>> > >
>> > > > > >> >>> > > >> >> > > > >>> > >
>> > > > > >> >>> > > >> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM
>> > > Emmanouil
>> > > > > >> >>> > Gkatziouras <
>> > > > > >> >>> > > >> >> > > > >>> > > gkatzioura@gmail.com>
>> > > > > >> >>> > > >> >> > > > >>> > > wrote:
>> > > > > >> >>> > > >> >> > > > >>> > >
>> > > > > >> >>> > > >> >> > > > >>> > > > Hi all!
>> > > > > >> >>> > > >> >> > > > >>> > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > Based on the discussions most
>> > probably
>> > > > > this
>> > > > > >> is
>> > > > > >> >>> > going
>> > > > > >> >>> > > to
>> > > > > >> >>> > > >> >> be on
>> > > > > >> >>> > > >> >> > > > >>> another
>> > > > > >> >>> > > >> >> > > > >>> > > > GitHub repo. Therefore I will
>> > prepare
>> > > a
>> > > > > >> >>> standalone
>> > > > > >> >>> > > >> project
>> > > > > >> >>> > > >> >> > with
>> > > > > >> >>> > > >> >> > > > the
>> > > > > >> >>> > > >> >> > > > >>> > > Pub/Sub
>> > > > > >> >>> > > >> >> > > > >>> > > > module and once the repository is
>> > > > created
>> > > > > a
>> > > > > >> >>> pull
>> > > > > >> >>> > > >> request
>> > > > > >> >>> > > >> >> > shall
>> > > > > >> >>> > > >> >> > > be
>> > > > > >> >>> > > >> >> > > > >>> > issued.
>> > > > > >> >>> > > >> >> > > > >>> > > > If there is anything else I can
>> do
>> > in
>> > > > > order
>> > > > > >> to
>> > > > > >> >>> > > >> facilitate
>> > > > > >> >>> > > >> >> > this
>> > > > > >> >>> > > >> >> > > > >>> please
>> > > > > >> >>> > > >> >> > > > >>> > let
>> > > > > >> >>> > > >> >> > > > >>> > > > me know.
>> > > > > >> >>> > > >> >> > > > >>> > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > If a ticket is created for this
>> > > issue, I
>> > > > > >> shall
>> > > > > >> >>> be
>> > > > > >> >>> > > able
>> > > > > >> >>> > > >> to
>> > > > > >> >>> > > >> >> > > create
>> > > > > >> >>> > > >> >> > > > a
>> > > > > >> >>> > > >> >> > > > >>> pull
>> > > > > >> >>> > > >> >> > > > >>> > > > request and thus a build will
>> take
>> > > place
>> > > > > on
>> > > > > >> >>> > ignite's
>> > > > > >> >>> > > >> CI.
>> > > > > >> >>> > > >> >> > > > >>> > > > If everyone is aligned with
>> adding a
>> > > > > Pub/Sub
>> > > > > >> >>> > > >> >> implementation
>> > > > > >> >>> > > >> >> > > may I
>> > > > > >> >>> > > >> >> > > > >>> > create
>> > > > > >> >>> > > >> >> > > > >>> > > a
>> > > > > >> >>> > > >> >> > > > >>> > > > ticket on JIRA?
>> > > > > >> >>> > > >> >> > > > >>> > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > My next question has to do on how
>> > > Ignite
>> > > > > >> >>> configures
>> > > > > >> >>> > > the
>> > > > > >> >>> > > >> >> build
>> > > > > >> >>> > > >> >> > > > jobs
>> > > > > >> >>> > > >> >> > > > >>> in
>> > > > > >> >>> > > >> >> > > > >>> > > Team
>> > > > > >> >>> > > >> >> > > > >>> > > > City. The unit tests for my
>> > > > implementation
>> > > > > >> >>> need a
>> > > > > >> >>> > > local
>> > > > > >> >>> > > >> >> > Pub/Sub
>> > > > > >> >>> > > >> >> > > > >>> server
>> > > > > >> >>> > > >> >> > > > >>> > up
>> > > > > >> >>> > > >> >> > > > >>> > > > and running (they provide one for
>> > > > testing
>> > > > > >> >>> > purposes).
>> > > > > >> >>> > > >> Will
>> > > > > >> >>> > > >> >> > this
>> > > > > >> >>> > > >> >> > > > take
>> > > > > >> >>> > > >> >> > > > >>> > > effect
>> > > > > >> >>> > > >> >> > > > >>> > > > in a form of a build script
>> inside
>> > my
>> > > > > >> >>> > implementation
>> > > > > >> >>> > > or
>> > > > > >> >>> > > >> >> is it
>> > > > > >> >>> > > >> >> > > > >>> something
>> > > > > >> >>> > > >> >> > > > >>> > > > that should be configured on Team
>> > > City?
>> > > > > >> >>> > > >> >> > > > >>> > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > Kind regards,
>> > > > > >> >>> > > >> >> > > > >>> > > > Emmanouil
>> > > > > >> >>> > > >> >> > > > >>> > > >
>> > > > > >> >>> > > >> >> > > > >>> > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > *Emmanouil Gkatziouras*
>> > > > > >> >>> > > >> >> > > > >>> > > > https://egkatzioura.com/ |
>> > > > > >> >>> > > >> >> > > > >>> > > >
>> > > > > >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > > >> >>> > > >> >> > > > >>> > > > https://github.com/gkatzioura
>> > > > > >> >>> > > >> >> > > > >>> > > >
>> > > > > >> >>> > > >> >> > > > >>> > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27,
>> Denis
>> > > > Magda
>> > > > > <
>> > > > > >> >>> > > >> >> dmagda@apache.org
>> > > > > >> >>> > > >> >> > >
>> > > > > >> >>> > > >> >> > > > >>> wrote:
>> > > > > >> >>> > > >> >> > > > >>> > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > Emmanouil, Saikat,
>> > > > > >> >>> > > >> >> > > > >>> > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > After contemplating on this,
>> let
>> > me
>> > > > > >> propose
>> > > > > >> >>> > another
>> > > > > >> >>> > > >> way
>> > > > > >> >>> > > >> >> > with
>> > > > > >> >>> > > >> >> > > > this
>> > > > > >> >>> > > >> >> > > > >>> > > > > particular integration that is
>> > > aligned
>> > > > > >> with
>> > > > > >> >>> our
>> > > > > >> >>> > > >> >> > > modularization
>> > > > > >> >>> > > >> >> > > > >>> > > endeavor.
>> > > > > >> >>> > > >> >> > > > >>> > > > > The modularization [1] defines
>> the
>> > > > > Ignite
>> > > > > >> >>> core,
>> > > > > >> >>> > > >> Ignite
>> > > > > >> >>> > > >> >> > > modules,
>> > > > > >> >>> > > >> >> > > > >>> and
>> > > > > >> >>> > > >> >> > > > >>> > > > > independent integrations. The
>> > first
>> > > > two
>> > > > > >> are
>> > > > > >> >>> to be
>> > > > > >> >>> > > >> >> located
>> > > > > >> >>> > > >> >> > in
>> > > > > >> >>> > > >> >> > > > >>> Ignite
>> > > > > >> >>> > > >> >> > > > >>> > > > > repositories and to be
>> > > tested/updated
>> > > > by
>> > > > > >> the
>> > > > > >> >>> > > >> community
>> > > > > >> >>> > > >> >> for
>> > > > > >> >>> > > >> >> > > > every
>> > > > > >> >>> > > >> >> > > > >>> > > release.
>> > > > > >> >>> > > >> >> > > > >>> > > > > The modules are a subset of
>> > existing
>> > > > > >> >>> integrations
>> > > > > >> >>> > > >> that
>> > > > > >> >>> > > >> >> are
>> > > > > >> >>> > > >> >> > > > >>> adopted
>> > > > > >> >>> > > >> >> > > > >>> > > widely
>> > > > > >> >>> > > >> >> > > > >>> > > > > and require community
>> attention.
>> > > > > >> >>> > > >> >> > > > >>> > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > As for the independent
>> > integrations
>> > > > [1],
>> > > > > >> >>> those
>> > > > > >> >>> > are
>> > > > > >> >>> > > >> >> supposed
>> > > > > >> >>> > > >> >> > > to
>> > > > > >> >>> > > >> >> > > > be
>> > > > > >> >>> > > >> >> > > > >>> > > located
>> > > > > >> >>> > > >> >> > > > >>> > > > > in a separate GitHub repo. I
>> would
>> > > > > advise
>> > > > > >> us
>> > > > > >> >>> to
>> > > > > >> >>> > > start
>> > > > > >> >>> > > >> >> > > > >>> implementing
>> > > > > >> >>> > > >> >> > > > >>> > this
>> > > > > >> >>> > > >> >> > > > >>> > > > > concept with the Pub/Sub
>> > > integration.
>> > > > > Even
>> > > > > >> >>> though
>> > > > > >> >>> > > it
>> > > > > >> >>> > > >> >> will
>> > > > > >> >>> > > >> >> > be
>> > > > > >> >>> > > >> >> > > > >>> located
>> > > > > >> >>> > > >> >> > > > >>> > > in a
>> > > > > >> >>> > > >> >> > > > >>> > > > > separate Github repo, it will
>> be
>> > > > listed
>> > > > > >> among
>> > > > > >> >>> > > >> officially
>> > > > > >> >>> > > >> >> > > > >>> available
>> > > > > >> >>> > > >> >> > > > >>> > > > > integrations on the Ignite
>> website
>> > > > > >> featuring
>> > > > > >> >>> > > >> Emmanouil
>> > > > > >> >>> > > >> >> as a
>> > > > > >> >>> > > >> >> > > > >>> > > contributor.
>> > > > > >> >>> > > >> >> > > > >>> > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > What do you think?
>> > > > > >> >>> > > >> >> > > > >>> > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > [1]
>> > > > > >> >>> > > >> >> > > > >>> > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > >
>> > > > > >> >>> > > >> >> > > > >>> > >
>> > > > > >> >>> > > >> >> > > > >>> >
>> > > > > >> >>> > > >> >> > > > >>>
>> > > > > >> >>> > > >> >> > > >
>> > > > > >> >>> > > >> >> > >
>> > > > > >> >>> > > >> >> >
>> > > > > >> >>> > > >> >>
>> > > > > >> >>> > > >>
>> > > > > >> >>> > >
>> > > > > >> >>> >
>> > > > > >> >>>
>> > > > > >>
>> > > > >
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> > > > > >> >>> > > >> >> > > > >>> > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > -
>> > > > > >> >>> > > >> >> > > > >>> > > > > Denis
>> > > > > >> >>> > > >> >> > > > >>> > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM
>> > > Saikat
>> > > > > >> >>> Maitra <
>> > > > > >> >>> > > >> >> > > > >>> > saikat.maitra@gmail.com
>> > > > > >> >>> > > >> >> > > > >>> > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > wrote:
>> > > > > >> >>> > > >> >> > > > >>> > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > > Hi Emmanouil,
>> > > > > >> >>> > > >> >> > > > >>> > > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > > The changes looks good to
>> me. I
>> > > > wanted
>> > > > > >> to
>> > > > > >> >>> check
>> > > > > >> >>> > > if
>> > > > > >> >>> > > >> you
>> > > > > >> >>> > > >> >> > are
>> > > > > >> >>> > > >> >> > > > also
>> > > > > >> >>> > > >> >> > > > >>> > able
>> > > > > >> >>> > > >> >> > > > >>> > > to
>> > > > > >> >>> > > >> >> > > > >>> > > > > > validate that the once data
>> is
>> > > added
>> > > > > to
>> > > > > >> >>> Ignite
>> > > > > >> >>> > > >> cluster
>> > > > > >> >>> > > >> >> > you
>> > > > > >> >>> > > >> >> > > > are
>> > > > > >> >>> > > >> >> > > > >>> also
>> > > > > >> >>> > > >> >> > > > >>> > > > able
>> > > > > >> >>> > > >> >> > > > >>> > > > > to
>> > > > > >> >>> > > >> >> > > > >>> > > > > > access it using another
>> Ignite
>> > > > client
>> > > > > or
>> > > > > >> >>> using
>> > > > > >> >>> > > rest
>> > > > > >> >>> > > >> >> > > > endpoints.
>> > > > > >> >>> > > >> >> > > > >>> > > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > > We use Teamcity for CI
>> process,
>> > > the
>> > > > > >> >>> details are
>> > > > > >> >>> > > as
>> > > > > >> >>> > > >> >> > follows
>> > > > > >> >>> > > >> >> > > > >>> > > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > > Trigger validation of those
>> test
>> > > > > suites
>> > > > > >> >>> that
>> > > > > >> >>> > have
>> > > > > >> >>> > > >> been
>> > > > > >> >>> > > >> >> > > > >>> affected by
>> > > > > >> >>> > > >> >> > > > >>> > > your
>> > > > > >> >>> > > >> >> > > > >>> > > > > > changes on TeamCity <
>> > > > > >> >>> > > http://ci.ignite.apache.org/>:
>> > > > > >> >>> > > >> <
>> > > > > >> >>> > > >> >> > > > >>> > > > > http://204.14.53.153/
>> > > > > >> >>> > > >> >> > > > >>> > > > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Locate a test suite you
>> > have
>> > > to
>> > > > > >> check,
>> > > > > >> >>> > press
>> > > > > >> >>> > > >> >> button
>> > > > > >> >>> > > >> >> > > > named
>> > > > > >> >>> > > >> >> > > > >>> > "..."
>> > > > > >> >>> > > >> >> > > > >>> > > > that
>> > > > > >> >>> > > >> >> > > > >>> > > > > >    is located on the left of
>> > "Run"
>> > > > > >> button.
>> > > > > >> >>> "Run
>> > > > > >> >>> > > >> custom
>> > > > > >> >>> > > >> >> > > build"
>> > > > > >> >>> > > >> >> > > > >>> > dialog
>> > > > > >> >>> > > >> >> > > > >>> > > > will
>> > > > > >> >>> > > >> >> > > > >>> > > > > >    appear;
>> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Go to "Changes" tab and
>> > > choose
>> > > > > >> >>> > > >> >> > > > >>> > "pull/<pull-request-number>/head"
>> > > > > >> >>> > > >> >> > > > >>> > > > in
>> > > > > >> >>> > > >> >> > > > >>> > > > > >    "Build branch" dropdown
>> list;
>> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Press "Run build" button
>> > and
>> > > > > >> monitor
>> > > > > >> >>> tests
>> > > > > >> >>> > > >> >> results.
>> > > > > >> >>> > > >> >> > > > >>> > > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > > @Roman rshtykh@yahoo.com -
>> can
>> > > you
>> > > > > also
>> > > > > >> >>> please
>> > > > > >> >>> > > >> take a
>> > > > > >> >>> > > >> >> > > look?
>> > > > > >> >>> > > >> >> > > > >>> > > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > > Regards,
>> > > > > >> >>> > > >> >> > > > >>> > > > > > Saikat
>> > > > > >> >>> > > >> >> > > > >>> > > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20
>> PM
>> > > > > >> Emmanouil
>> > > > > >> >>> > > >> Gkatziouras
>> > > > > >> >>> > > >> >> <
>> > > > > >> >>> > > >> >> > > > >>> > > > > > gkatzioura@gmail.com>
>> > > > > >> >>> > > >> >> > > > >>> > > > > > wrote:
>> > > > > >> >>> > > >> >> > > > >>> > > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > > > Hi @Saikat Maitra
>> > > > > >> >>> > > >> >> > > > >>> > > > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > > > I have my implementation
>> on an
>> > > > > Ignite
>> > > > > >> >>> fork on
>> > > > > >> >>> > > my
>> > > > > >> >>> > > >> >> > personal
>> > > > > >> >>> > > >> >> > > > >>> GitHub
>> > > > > >> >>> > > >> >> > > > >>> > > > > account
>> > > > > >> >>> > > >> >> > > > >>> > > > > > > [1]
>> > > > > >> >>> > > >> >> > > > >>> > > > > > > It is based on the Kafka
>> > > > > >> implementation
>> > > > > >> >>> which
>> > > > > >> >>> > > is
>> > > > > >> >>> > > >> of
>> > > > > >> >>> > > >> >> a
>> > > > > >> >>> > > >> >> > > > polling
>> > > > > >> >>> > > >> >> > > > >>> > > nature.
>> > > > > >> >>> > > >> >> > > > >>> > > > > > > GCP does not provide a
>> service
>> > > > like
>> > > > > >> >>> Kafka.
>> > > > > >> >>> > > >> Instead
>> > > > > >> >>> > > >> >> it
>> > > > > >> >>> > > >> >> > > > >>> provides us
>> > > > > >> >>> > > >> >> > > > >>> > > > with
>> > > > > >> >>> > > >> >> > > > >>> > > > > > > Pub/Sub which is just like
>> > > Kafka.
>> > > > > >> >>> > > >> >> > > > >>> > > > > > > Therefore using the Kafka
>> > > streamer
>> > > > > as
>> > > > > >> a
>> > > > > >> >>> > > guideline
>> > > > > >> >>> > > >> >> made
>> > > > > >> >>> > > >> >> > > more
>> > > > > >> >>> > > >> >> > > > >>> sense
>> > > > > >> >>> > > >> >> > > > >>> > > to
>> > > > > >> >>> > > >> >> > > > >>> > > > > me.
>> > > > > >> >>> > > >> >> > > > >>> > > > > > >
>> > > > > >> >>> > > >> >> > > > >>> > > > > > > Apart from the
>> implementation
>> > I
>> > > > have
>> > > > > >> also
>> > > > > >> >>> > > added a
>> > > > > >> >>> > > >> >> unit
>> > > > > >> >>> > > >> >> > > > test.
>> > > > > >> >>> > > >> >> > > > >>> > > > > > > It definitely needs some
>> > > polishing
>> > > > > >> and a
>> > > > > >> >>> > double
>> > > > > >> >>> > > >> >> check
>> > > > > >> >>> > > >> >> > on
>> > > > > >> >>> > > >> >> > > my
>> > > > > >> >>> > > >> >> > > > >>> side
>> > > > > >> >>> > > >> >> > > > >>> > to
>> > > > > >> >>> > > >> >> > > > >>> > > > > make
>> > > > > >> >>> > > >> >> > > > >>> > > > > > > the review smooth.
>> > > > > >> >>> > > >> >> > > > >>> > > > > > > For the tests a local
>> Pub/Sub
>> > > [2]
>> > > > > >> server
>> > > >
>
>

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra <sa...@gmail.com>]

Hi,

We do run build with -DskipTests and then configure selective test suites
to run.

I will check why these test suites are not getting executed.

Regards
Saikat

On Wed, 27 Nov 2019 at 3:05 PM, Emmanouil Gkatziouras <gk...@gmail.com>
wrote:

> Hi all,
>
> I did add a test suite and run the build with the corresponding arguments,
> however the tests did not run [1].
> I checked the Flink's build logs and the same message is displayed there
> too `No tests to run` [2].
>
> [1]
>
> https://ci.ignite.apache.org/viewLog.html?buildId=4798203&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=248
> [2]
>
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4789199&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=1256
>
> Kind regards
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>
>
> On Wed, 27 Nov 2019 at 18:58, Saikat Maitra <sa...@gmail.com>
> wrote:
>
> > Hi,
> >
> > We will need to add test suites for tests to be executed in build. We
> pass
> > them as TestSuites param. I had added Flink sink and source testsuites.
> >
> > Can you please review and confirm.
> >
> > Regards
> > Saikat
> >
> > On Wed, 27 Nov 2019 at 6:00 AM, Emmanouil Gkatziouras <
> > gkatzioura@gmail.com>
> > wrote:
> >
> > > Hi all!
> > >
> > > I was successful building and running my pull requests. It seems that
> the
> > > tests do not run both for Flink and Pub/Sub [1]
> > > If there is something I cannot do to make them discoverable please let
> me
> > > know.
> > >
> > > [1]
> > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4796722&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=283
> > >
> > > *Emmanouil Gkatziouras*
> > > https://egkatzioura.com/ |
> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > https://github.com/gkatzioura
> > >
> > >
> > > On Wed, 27 Nov 2019 at 04:57, Saikat Maitra <sa...@gmail.com>
> > > wrote:
> > >
> > > > Hi Emmanouil,
> > > >
> > > > I have added you as contributor in Ignite Extensions project. Can you
> > > > please check and confirm if you are able to see the project and
> execute
> > > > build on your pull request.
> > > >
> > > > Regards,
> > > > Saikat
> > > >
> > > > On Tue, Nov 26, 2019 at 12:49 PM Emmanouil Gkatziouras <
> > > > gkatzioura@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi all!
> > > > >
> > > > > If someone could give me read access to the Job of Ignite
> Extensions
> > on
> > > > > Team City will greatly help me (username gkatzioura).
> > > > > I suppose the builds get triggered automatically on a pull request.
> > > > >
> > > > > Kind regards,
> > > > > Emmanouil
> > > > >
> > > > > *Emmanouil Gkatziouras*
> > > > > https://egkatzioura.com/ |
> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > https://github.com/gkatzioura
> > > > >
> > > > >
> > > > > On Sun, 24 Nov 2019 at 21:11, Emmanouil Gkatziouras <
> > > > gkatzioura@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi Saikat!
> > > > > >
> > > > > > I just rebased with the flink branch. Unfortunately I do not have
> > > read
> > > > > > access to the team city link you provided and thus evaluate the
> > tests
> > > > for
> > > > > > my pull request.
> > > > > > I guess it has to do with ignite extensions being new.
> > > > > > My username is gkatzioura and email the one I am using on this
> > email
> > > > > >
> > > > > > Thank you for your time.
> > > > > >
> > > > > >
> > > > > > *Emmanouil Gkatziouras*
> > > > > > https://egkatzioura.com/ |
> > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > > https://github.com/gkatzioura
> > > > > >
> > > > > >
> > > > > > On Sun, 24 Nov 2019 at 18:03, Saikat Maitra <
> > saikat.maitra@gmail.com
> > > >
> > > > > > wrote:
> > > > > >
> > > > > >> Hi Emmanouil,
> > > > > >>
> > > > > >> The latest build on teamcity has passed on Flink pull request.
> > > > > >>
> > > > > >>
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4788928&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> > > > > >>
> > > > > >> you should be able to take the changes I made in my PR and run
> > build
> > > > on
> > > > > >> your pull request.
> > > > > >>
> > > > > >> Please let me know if you have any questions.
> > > > > >>
> > > > > >> Regards,
> > > > > >> Saikat
> > > > > >>
> > > > > >> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra <
> > > > saikat.maitra@gmail.com
> > > > > >
> > > > > >> wrote:
> > > > > >>
> > > > > >> > Hi Emmanouil,
> > > > > >> >
> > > > > >> > I have fixed the Flink module testsuites and have setup new
> > > project
> > > > in
> > > > > >> > teamcity.
> > > > > >> >
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview
> > > > > >> >
> > > > > >> > I am looking into setting up the build.
> > > > > >> >
> > > > > >> > Regards,
> > > > > >> > Saikat
> > > > > >> >
> > > > > >> > On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra <
> > > > > saikat.maitra@gmail.com>
> > > > > >> > wrote:
> > > > > >> >
> > > > > >> >> Hi Emmanouil,
> > > > > >> >>
> > > > > >> >> Thank you for your email. Yes, the plan is once the Flink PR
> is
> > > > > merged
> > > > > >> it
> > > > > >> >> will provide the Licence, parent POMs etc and you can rebase
> > from
> > > > > >> master
> > > > > >> >> branch and apply the changes on top of it.
> > > > > >> >>
> > > > > >> >> I am getting some issues with test failures in local with
> > > > > >> >> GridTestProperties as the test.properties is not present in
> > this
> > > > new
> > > > > >> >> project but should be available from dependencies.
> > > > > >> >>
> > > > > >> >> Once I address this issue, I will go ahead and merge the
> > changes
> > > > and
> > > > > >> then
> > > > > >> >> we can take it from there.
> > > > > >> >>
> > > > > >> >> Regards,
> > > > > >> >> Saikat
> > > > > >> >>
> > > > > >> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras <
> > > > > >> >> gkatzioura@gmail.com> wrote:
> > > > > >> >>
> > > > > >> >>> Hi all,
> > > > > >> >>>
> > > > > >> >>> I made my first pull request [1]. Since this project is
> brand
> > > new
> > > > > (no
> > > > > >> >>> parent poms, licensing), I reckoned it was best to use
> > Saikat's
> > > > > >> branch on
> > > > > >> >>> Flink.
> > > > > >> >>> I suppose the Flink branch will be merged first. If not
> please
> > > > give
> > > > > me
> > > > > >> >>> guidelines on how I should proceed next.
> > > > > >> >>>
> > > > > >> >>> Kind regards
> > > > > >> >>> Emmanouil
> > > > > >> >>>
> > > > > >> >>> [1] https://github.com/apache/ignite-extensions/pull/2
> > > > > >> >>>
> > > > > >> >>> *Emmanouil Gkatziouras*
> > > > > >> >>> https://egkatzioura.com/ |
> > > > > >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > >> >>> https://github.com/gkatzioura
> > > > > >> >>>
> > > > > >> >>>
> > > > > >> >>> On Fri, 22 Nov 2019 at 20:55, Denis Magda <
> dmagda@apache.org>
> > > > > wrote:
> > > > > >> >>>
> > > > > >> >>> > Awesome, ping us whenever you're ready!
> > > > > >> >>> >
> > > > > >> >>> > -
> > > > > >> >>> > Denis
> > > > > >> >>> >
> > > > > >> >>> >
> > > > > >> >>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <
> > > > > >> >>> > gkatzioura@gmail.com>
> > > > > >> >>> > wrote:
> > > > > >> >>> >
> > > > > >> >>> > > Hi all!
> > > > > >> >>> > >
> > > > > >> >>> > > I am sorry for being late on that. I was trying to
> > refactor
> > > > the
> > > > > >> test
> > > > > >> >>> in
> > > > > >> >>> > > order not to be in need of any external tools or
> spinning
> > > up a
> > > > > >> >>> server.
> > > > > >> >>> > > I did forked the new repo and indeed my changes there,
> so
> > a
> > > > pull
> > > > > >> >>> request
> > > > > >> >>> > is
> > > > > >> >>> > > a matter of time!
> > > > > >> >>> > >
> > > > > >> >>> > > Kind regards
> > > > > >> >>> > >
> > > > > >> >>> > > *Emmanouil Gkatziouras*
> > > > > >> >>> > > https://egkatzioura.com/ |
> > > > > >> >>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > >> >>> > > https://github.com/gkatzioura
> > > > > >> >>> > >
> > > > > >> >>> > >
> > > > > >> >>> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda <
> > > dmagda@apache.org>
> > > > > >> wrote:
> > > > > >> >>> > >
> > > > > >> >>> > > > Hi Emmanouil,
> > > > > >> >>> > > >
> > > > > >> >>> > > > Do you have any questions or need any support from the
> > > > > >> community?
> > > > > >> >>> > > >
> > > > > >> >>> > > > -
> > > > > >> >>> > > > Denis
> > > > > >> >>> > > >
> > > > > >> >>> > > >
> > > > > >> >>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <
> > > > > >> >>> saikat.maitra@gmail.com
> > > > > >> >>> > >
> > > > > >> >>> > > > wrote:
> > > > > >> >>> > > >
> > > > > >> >>> > > >> Hi Emmanouil,
> > > > > >> >>> > > >>
> > > > > >> >>> > > >> Can you please take a looks at dev utils, if this is
> > > > > something
> > > > > >> >>> you are
> > > > > >> >>> > > >> looking for
> > > > > >> >>> > > >>
> > > > > https://github.com/apache/ignite/tree/master/modules/dev-utils
> > > > > >> ?
> > > > > >> >>> > > >>
> > > > > >> >>> > > >> IMO, if you can release Pub/Sub server in maven and
> > then
> > > > use
> > > > > >> it as
> > > > > >> >>> > > >> dependency, that would be great.
> > > > > >> >>> > > >>
> > > > > >> >>> > > >> Regards,
> > > > > >> >>> > > >> Saikat
> > > > > >> >>> > > >>
> > > > > >> >>> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <
> > > > > >> >>> > saikat.maitra@gmail.com>
> > > > > >> >>> > > >> wrote:
> > > > > >> >>> > > >>
> > > > > >> >>> > > >> > Hi Emmanouil,
> > > > > >> >>> > > >> >
> > > > > >> >>> > > >> > The master branch has been initialized. I have
> > raised a
> > > > PR
> > > > > to
> > > > > >> >>> > migrate
> > > > > >> >>> > > >> > Flink module to ignite-extensions.
> > > > > >> >>> > > >> >
> > > > > >> >>> > > >> > PR
> > https://github.com/apache/ignite-extensions/pull/1
> > > > > >> >>> > > >> > Jira :
> > > > https://issues.apache.org/jira/browse/IGNITE-12356
> > > > > >> >>> > > >> >
> > > > > >> >>> > > >> > Regards,
> > > > > >> >>> > > >> > Saikat
> > > > > >> >>> > > >> >
> > > > > >> >>> > > >> >
> > > > > >> >>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil
> > Gkatziouras <
> > > > > >> >>> > > >> > gkatzioura@gmail.com> wrote:
> > > > > >> >>> > > >> >
> > > > > >> >>> > > >> >> Hi all!
> > > > > >> >>> > > >> >>
> > > > > >> >>> > > >> >> This is an update. The integration is tested with
> > > mocks
> > > > > >> based
> > > > > >> >>> on
> > > > > >> >>> > the
> > > > > >> >>> > > >> >> original Ignite repo [1].
> > > > > >> >>> > > >> >> I am currently waiting for the first commits on
> the
> > > new
> > > > > >> >>> integration
> > > > > >> >>> > > >> >> project
> > > > > >> >>> > > >> >> to take place in order to create my pull request
> > > > > accordingly
> > > > > >> >>> to the
> > > > > >> >>> > > >> >> initial
> > > > > >> >>> > > >> >> commits.
> > > > > >> >>> > > >> >> In order to simulate the Pub/Sub server in a test
> I
> > > > used a
> > > > > >> test
> > > > > >> >>> > > server
> > > > > >> >>> > > >> >> implementation [2]. Unfortunately they don't have
> > this
> > > > > util
> > > > > >> >>> > uploaded
> > > > > >> >>> > > >> in a
> > > > > >> >>> > > >> >> maven repo, thus it needs to be hosted on a maven
> > repo
> > > > > >> (Apache
> > > > > >> >>> > > License
> > > > > >> >>> > > >> >> 2.0). Is there a repo for utilities that Ignite
> uses
> > > for
> > > > > the
> > > > > >> >>> > > >> test/build?
> > > > > >> >>> > > >> >> I have also created a ticket [3]
> > > > > >> >>> > > >> >>
> > > > > >> >>> > > >> >> Kind regards
> > > > > >> >>> > > >> >>
> > > > > >> >>> > > >> >> [1]
> > > > > >> >>> >
> > > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > > > > >> >>> > > >> >> [2]
> > > > > >> >>>
> https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> > > > > >> >>> > > >> >> [3]
> > > https://issues.apache.org/jira/browse/IGNITE-12262
> > > > > >> >>> > > >> >>
> > > > > >> >>> > > >> >> *Emmanouil Gkatziouras*
> > > > > >> >>> > > >> >> https://egkatzioura.com/ |
> > > > > >> >>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > >> >>> > > >> >> https://github.com/gkatzioura
> > > > > >> >>> > > >> >>
> > > > > >> >>> > > >> >>
> > > > > >> >>> > > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <
> > > > > >> >>> > saikat.maitra@gmail.com
> > > > > >> >>> > > >
> > > > > >> >>> > > >> >> wrote:
> > > > > >> >>> > > >> >>
> > > > > >> >>> > > >> >> > Hello Ilya,
> > > > > >> >>> > > >> >> >
> > > > > >> >>> > > >> >> > Thank you for your email. Yes, before we remove
> > any
> > > > > >> >>> integration
> > > > > >> >>> > we
> > > > > >> >>> > > >> will
> > > > > >> >>> > > >> >> > move them to another repository and release.
> > > > > >> >>> > > >> >> >
> > > > > >> >>> > > >> >> > We are thinking we will start with Pub/Sub
> > Streamer
> > > > > >> >>> > implementation
> > > > > >> >>> > > >> and
> > > > > >> >>> > > >> >> > continue to move other integration as mentioned
> in
> > > the
> > > > > >> >>> following
> > > > > >> >>> > > doc.
> > > > > >> >>> > > >> >> >
> > > > > >> >>> > > >> >> >
> > > > > >> >>> > > >> >> >
> > > > > >> >>> > > >> >>
> > > > > >> >>> > > >>
> > > > > >> >>> > >
> > > > > >> >>> >
> > > > > >> >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > > >> >>> > > >> >> >
> > > > > >> >>> > > >> >> > Regards,
> > > > > >> >>> > > >> >> > Saikat
> > > > > >> >>> > > >> >> >
> > > > > >> >>> > > >> >> >
> > > > > >> >>> > > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya
> Kasnacheev <
> > > > > >> >>> > > >> >> > ilya.kasnacheev@gmail.com>
> > > > > >> >>> > > >> >> > wrote:
> > > > > >> >>> > > >> >> >
> > > > > >> >>> > > >> >> > > Hello!
> > > > > >> >>> > > >> >> > >
> > > > > >> >>> > > >> >> > > My take on this: we want to offload some of
> > > > > >> integrations,
> > > > > >> >>> but
> > > > > >> >>> > > >> please
> > > > > >> >>> > > >> >> > commit
> > > > > >> >>> > > >> >> > > and release them to another project first, and
> > > only
> > > > > >> then we
> > > > > >> >>> > will
> > > > > >> >>> > > >> >> decide
> > > > > >> >>> > > >> >> > to
> > > > > >> >>> > > >> >> > > remove them from our repository.
> > > > > >> >>> > > >> >> > >
> > > > > >> >>> > > >> >> > > Please also make sure they are available with
> > same
> > > > > >> artifact
> > > > > >> >>> > name
> > > > > >> >>> > > >> and
> > > > > >> >>> > > >> >> API.
> > > > > >> >>> > > >> >> > > If this has to be changed, let's wait for AI
> 3.0
> > > > > >> >>> > > >> >> > >
> > > > > >> >>> > > >> >> > > Regards,
> > > > > >> >>> > > >> >> > > --
> > > > > >> >>> > > >> >> > > Ilya Kasnacheev
> > > > > >> >>> > > >> >> > >
> > > > > >> >>> > > >> >> > >
> > > > > >> >>> > > >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <
> > > > > >> >>> > > >> saikat.maitra@gmail.com>:
> > > > > >> >>> > > >> >> > >
> > > > > >> >>> > > >> >> > > > Hello Emmanouil,
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > > > As discussed earlier, I discussed with
> Apache
> > > > Bahir
> > > > > >> >>> community
> > > > > >> >>> > > and
> > > > > >> >>> > > >> >> they
> > > > > >> >>> > > >> >> > > are
> > > > > >> >>> > > >> >> > > > interested to have Apache Ignite extensions
> as
> > > > part
> > > > > of
> > > > > >> >>> Apache
> > > > > >> >>> > > >> Bahir
> > > > > >> >>> > > >> >> > > > project.
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > > > Can you please share Pub/Sub streamer
> > > > implementation
> > > > > >> >>> details
> > > > > >> >>> > > with
> > > > > >> >>> > > >> >> > Apache
> > > > > >> >>> > > >> >> > > > Bahir community and request for feedback.
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > > > https://bahir.apache.org/
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > > > I have also requested for contributor access
> > in
> > > > Jira
> > > > > >> for
> > > > > >> >>> > Apache
> > > > > >> >>> > > >> >> Bahir
> > > > > >> >>> > > >> >> > > > project so that I can create issues and
> assign
> > > to
> > > > > >> myself.
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > > > I can help with code reviews as well.
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > > > Here is the email thread for reference
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > >
> > > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > > > Regards,
> > > > > >> >>> > > >> >> > > > Saikat
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat
> Maitra
> > <
> > > > > >> >>> > > >> >> saikat.maitra@gmail.com
> > > > > >> >>> > > >> >> > >
> > > > > >> >>> > > >> >> > > > wrote:
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > > > > Hello,
> > > > > >> >>> > > >> >> > > > >
> > > > > >> >>> > > >> >> > > > > As discussed I have created following
> > > discussion
> > > > > >> >>> threads on
> > > > > >> >>> > > our
> > > > > >> >>> > > >> >> > > migration
> > > > > >> >>> > > >> >> > > > > proposals for Apache Ignite extensions:
> > > > > >> >>> > > >> >> > > > >
> > > > > >> >>> > > >> >> > > > >
> > > > > >> >>> > > >> >> > > > >
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > >
> > > > > >> >>> > > >> >> >
> > > > > >> >>> > > >> >>
> > > > > >> >>> > > >>
> > > > > >> >>> > >
> > > > > >> >>> >
> > > > > >> >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> > > > > >> >>> > > >> >> > > > >
> > > > > >> >>> > > >> >> > > > >
> > > > > >> >>> > > >>
> > > > > >> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > > > > >> >>> > > >> >> > > > >
> > > > > >> >>> > > >> >> > > > > I will share update as I hear more
> > > information.
> > > > > >> >>> > > >> >> > > > >
> > > > > >> >>> > > >> >> > > > > Regards,
> > > > > >> >>> > > >> >> > > > > Saikat
> > > > > >> >>> > > >> >> > > > >
> > > > > >> >>> > > >> >> > > > >
> > > > > >> >>> > > >> >> > > > >
> > > > > >> >>> > > >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat
> > Maitra
> > > <
> > > > > >> >>> > > >> >> > saikat.maitra@gmail.com
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > > > > wrote:
> > > > > >> >>> > > >> >> > > > >
> > > > > >> >>> > > >> >> > > > >> Hello Denis,
> > > > > >> >>> > > >> >> > > > >>
> > > > > >> >>> > > >> >> > > > >> Sure, sounds good. I will create a
> separate
> > > > > >> discussion
> > > > > >> >>> > > thread
> > > > > >> >>> > > >> to
> > > > > >> >>> > > >> >> get
> > > > > >> >>> > > >> >> > > > >> community feedback on whether we
> > > > > >> >>> > > >> >> > > > >> should join the Bahir or kick-off "Ignite
> > > > > >> Extensions"
> > > > > >> >>> > > project.
> > > > > >> >>> > > >> >> > > > >>
> > > > > >> >>> > > >> >> > > > >> Regards,
> > > > > >> >>> > > >> >> > > > >> Saikat
> > > > > >> >>> > > >> >> > > > >>
> > > > > >> >>> > > >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis
> > Magda <
> > > > > >> >>> > > >> dmagda@apache.org>
> > > > > >> >>> > > >> >> > > wrote:
> > > > > >> >>> > > >> >> > > > >>
> > > > > >> >>> > > >> >> > > > >>> Folks,
> > > > > >> >>> > > >> >> > > > >>>
> > > > > >> >>> > > >> >> > > > >>> The concept of Apache Bahir is precisely
> > > what
> > > > we
> > > > > >> >>> need!
> > > > > >> >>> > > >> Saikat,
> > > > > >> >>> > > >> >> > thanks
> > > > > >> >>> > > >> >> > > > for
> > > > > >> >>> > > >> >> > > > >>> doing the research. Why I believe the *
> > > idea*
> > > > > >> fits us
> > > > > >> >>> > well:
> > > > > >> >>> > > >> >> > > > >>>
> > > > > >> >>> > > >> >> > > > >>>    - All integrations can be stored in
> > > > separate
> > > > > >> >>> Github
> > > > > >> >>> > > >> >> repositories
> > > > > >> >>> > > >> >> > > and
> > > > > >> >>> > > >> >> > > > >>>    have their dev lifecycles. We've not
> > > > obliged
> > > > > to
> > > > > >> >>> couple
> > > > > >> >>> > > all
> > > > > >> >>> > > >> >> the
> > > > > >> >>> > > >> >> > > > >>> integrations
> > > > > >> >>> > > >> >> > > > >>>    in a single repo. For instance, Spark
> > can
> > > > be
> > > > > >> >>> located
> > > > > >> >>> > in
> > > > > >> >>> > > a
> > > > > >> >>> > > >> >> > > dedicated
> > > > > >> >>> > > >> >> > > > >>> repo
> > > > > >> >>> > > >> >> > > > >>>    while streaming integrations might be
> > in
> > > a
> > > > > >> single
> > > > > >> >>> one.
> > > > > >> >>> > > >> It's
> > > > > >> >>> > > >> >> up
> > > > > >> >>> > > >> >> > to
> > > > > >> >>> > > >> >> > > > us.
> > > > > >> >>> > > >> >> > > > >>>    - All the repositories and
> integrations
> > > > > belong
> > > > > >> to
> > > > > >> >>> ASF.
> > > > > >> >>> > > >> We're
> > > > > >> >>> > > >> >> not
> > > > > >> >>> > > >> >> > > > >>> dumping
> > > > > >> >>> > > >> >> > > > >>>    them on Github but rather keep
> > supporting
> > > > and
> > > > > >> >>> > developing
> > > > > >> >>> > > >> in
> > > > > >> >>> > > >> >> > > > >>> accordance with
> > > > > >> >>> > > >> >> > > > >>>    ASF vision and practices.
> > > > > >> >>> > > >> >> > > > >>>    - There is a way to reward
> contributors
> > > via
> > > > > >> >>> > > committership
> > > > > >> >>> > > >> and
> > > > > >> >>> > > >> >> > PMC
> > > > > >> >>> > > >> >> > > > >>>    membership. You won't get this if a
> > > project
> > > > > is
> > > > > >> >>> just
> > > > > >> >>> > one
> > > > > >> >>> > > of
> > > > > >> >>> > > >> >> the
> > > > > >> >>> > > >> >> > > > >>> millions of
> > > > > >> >>> > > >> >> > > > >>>    Github projects.
> > > > > >> >>> > > >> >> > > > >>>
> > > > > >> >>> > > >> >> > > > >>> Saikat, as Ignite PMC member, could you
> > > please
> > > > > >> >>> kick-off a
> > > > > >> >>> > > >> >> separate
> > > > > >> >>> > > >> >> > > > >>> dev-list
> > > > > >> >>> > > >> >> > > > >>> discussion to involve more community
> > members
> > > > and
> > > > > >> >>> > referring
> > > > > >> >>> > > to
> > > > > >> >>> > > >> >> this
> > > > > >> >>> > > >> >> > > > >>> thread?
> > > > > >> >>> > > >> >> > > > >>> I think the primary question the
> community
> > > > needs
> > > > > >> to
> > > > > >> >>> > answer
> > > > > >> >>> > > >> >> whether
> > > > > >> >>> > > >> >> > we
> > > > > >> >>> > > >> >> > > > >>> should join the Bahir or kick-off
> "Ignite
> > > > > >> Extensions"
> > > > > >> >>> > > >> project?
> > > > > >> >>> > > >> >> > > > >>>
> > > > > >> >>> > > >> >> > > > >>> -
> > > > > >> >>> > > >> >> > > > >>> Denis
> > > > > >> >>> > > >> >> > > > >>>
> > > > > >> >>> > > >> >> > > > >>>
> > > > > >> >>> > > >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey
> > > > Zinoviev
> > > > > <
> > > > > >> >>> > > >> >> > > > zaleslaw.sin@gmail.com>
> > > > > >> >>> > > >> >> > > > >>> wrote:
> > > > > >> >>> > > >> >> > > > >>>
> > > > > >> >>> > > >> >> > > > >>> > Maybe we could move all our Streaming
> > > > > >> Integrations
> > > > > >> >>> > there,
> > > > > >> >>> > > >> but
> > > > > >> >>> > > >> >> > what
> > > > > >> >>> > > >> >> > > is
> > > > > >> >>> > > >> >> > > > >>> about
> > > > > >> >>> > > >> >> > > > >>> > maintaining and committer permissions
> to
> > > the
> > > > > new
> > > > > >> >>> > > >> repositories?
> > > > > >> >>> > > >> >> > > > >>> > I see the list of the committers and
> PMC
> > > > > members
> > > > > >> >>> there
> > > > > >> >>> > > >> >> > > > >>> >
> > > https://bahir.apache.org/community-members/
> > > > > >> >>> > > >> >> > > > >>> >
> > > > > >> >>> > > >> >> > > > >>> > Could somebody from Ignite community
> be
> > > > added
> > > > > to
> > > > > >> >>> this
> > > > > >> >>> > > list
> > > > > >> >>> > > >> as
> > > > > >> >>> > > >> >> a
> > > > > >> >>> > > >> >> > > > >>> > PMC/committer to maintain Ignite
> > > > integrations?
> > > > > >> >>> > > >> >> > > > >>> > If yes, I'd happy to join this project
> > and
> > > > > >> >>> collaborate
> > > > > >> >>> > > with
> > > > > >> >>> > > >> >> these
> > > > > >> >>> > > >> >> > > > guys
> > > > > >> >>> > > >> >> > > > >>> > together
> > > > > >> >>> > > >> >> > > > >>> > If no, and we should start from the
> zero
> > > > level
> > > > > >> with
> > > > > >> >>> > > >> external
> > > > > >> >>> > > >> >> PRs
> > > > > >> >>> > > >> >> > -
> > > > > >> >>> > > >> >> > > > >>> hmmm,
> > > > > >> >>> > > >> >> > > > >>> > it's better to have our own external
> > > > > repository
> > > > > >> >>> with
> > > > > >> >>> > > >> >> ApacheBahirr
> > > > > >> >>> > > >> >> > > > >>> ideology.
> > > > > >> >>> > > >> >> > > > >>> >
> > > > > >> >>> > > >> >> > > > >>> > Also, I agree, that all connectors
> could
> > > be
> > > > > >> moved
> > > > > >> >>> there
> > > > > >> >>> > > (in
> > > > > >> >>> > > >> >> > > > >>> ApacheBahirr
> > > > > >> >>> > > >> >> > > > >>> > like repository), but not all modules
> > from
> > > > the
> > > > > >> page
> > > > > >> >>> > > >> >> > > > >>> >
> > > > > >> >>> > > >> >> > > > >>> >
> > > > > >> >>> > > >> >> > > > >>>
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > >
> > > > > >> >>> > > >> >> >
> > > > > >> >>> > > >> >>
> > > > > >> >>> > > >>
> > > > > >> >>> > >
> > > > > >> >>> >
> > > > > >> >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > > >> >>> > > >> >> > > > >>> > because
> > > > > >> >>> > > >> >> > > > >>> > they use different parts of core
> modules
> > > and
> > > > > >> could
> > > > > >> >>> be
> > > > > >> >>> > > >> >> developed
> > > > > >> >>> > > >> >> > > with
> > > > > >> >>> > > >> >> > > > >>> > different velocity.
> > > > > >> >>> > > >> >> > > > >>> >
> > > > > >> >>> > > >> >> > > > >>> > In reality, before creation of new
> > > > > repositories
> > > > > >> we
> > > > > >> >>> need
> > > > > >> >>> > > >> wide
> > > > > >> >>> > > >> >> > > > discussion
> > > > > >> >>> > > >> >> > > > >>> > based on the document mentioned above.
> > > > > >> >>> > > >> >> > > > >>> >
> > > > > >> >>> > > >> >> > > > >>> > P.S. Of course, we could experiment in
> > the
> > > > > next
> > > > > >> >>> release
> > > > > >> >>> > > 2.8
> > > > > >> >>> > > >> >> with
> > > > > >> >>> > > >> >> > > one
> > > > > >> >>> > > >> >> > > > >>> or two
> > > > > >> >>> > > >> >> > > > >>> > integrations like twitter or ZeroMQ
> > > > > >> >>> > > >> >> > > > >>> > Also, I'd like Denis Magda idea for
> the
> > > > > separate
> > > > > >> >>> > > >> repositories
> > > > > >> >>> > > >> >> in
> > > > > >> >>> > > >> >> > > > Apache
> > > > > >> >>> > > >> >> > > > >>> > Github to maintain them by anybody.
> > > > > >> >>> > > >> >> > > > >>> >
> > > > > >> >>> > > >> >> > > > >>> >
> > > > > >> >>> > > >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat
> > > Maitra <
> > > > > >> >>> > > >> >> > > saikat.maitra@gmail.com
> > > > > >> >>> > > >> >> > > > >:
> > > > > >> >>> > > >> >> > > > >>> >
> > > > > >> >>> > > >> >> > > > >>> > > Hi Denis, Emmanouil
> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > >> >>> > > >> >> > > > >>> > > Thank you for your email. I think
> > > > creating a
> > > > > >> >>> separate
> > > > > >> >>> > > >> >> > integration
> > > > > >> >>> > > >> >> > > > >>> project
> > > > > >> >>> > > >> >> > > > >>> > > in github and also proposing it as
> an
> > > > apache
> > > > > >> >>> > incubator
> > > > > >> >>> > > >> >> project
> > > > > >> >>> > > >> >> > > will
> > > > > >> >>> > > >> >> > > > >>> be a
> > > > > >> >>> > > >> >> > > > >>> > > good move. The other separate
> > > integration
> > > > > >> modules
> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > >> >>> > > >> >> > > > >>> >
> > > > > >> >>> > > >> >> > > > >>>
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > >
> > > > > >> >>> > > >> >> >
> > > > > >> >>> > > >> >>
> > > > > >> >>> > > >>
> > > > > >> >>> > >
> > > > > >> >>> >
> > > > > >> >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > > >> >>> > > >> >> > > > >>> > > can
> > > > > >> >>> > > >> >> > > > >>> > > be moved to this new apache
> incubator
> > > > > project.
> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > >> >>> > > >> >> > > > >>> > > There are similar integration
> > available
> > > > for
> > > > > >> >>> Flink and
> > > > > >> >>> > > >> Spark
> > > > > >> >>> > > >> >> in
> > > > > >> >>> > > >> >> > > > Apache
> > > > > >> >>> > > >> >> > > > >>> > Bahir
> > > > > >> >>> > > >> >> > > > >>> > > https://bahir.apache.org/
> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > >> >>> > > >> >> > > > >>> > > Do you think we should reach out to
> > > Apache
> > > > > >> Bahir
> > > > > >> >>> > > >> community
> > > > > >> >>> > > >> >> > with a
> > > > > >> >>> > > >> >> > > > >>> > proposal
> > > > > >> >>> > > >> >> > > > >>> > > or we should plan to create a
> separate
> > > > > Apache
> > > > > >> >>> > Incubator
> > > > > >> >>> > > >> >> > project?
> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > >> >>> > > >> >> > > > >>> > > Regards,
> > > > > >> >>> > > >> >> > > > >>> > > Saikat
> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > >> >>> > > >> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM
> > > Emmanouil
> > > > > >> >>> > Gkatziouras <
> > > > > >> >>> > > >> >> > > > >>> > > gkatzioura@gmail.com>
> > > > > >> >>> > > >> >> > > > >>> > > wrote:
> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > >> >>> > > >> >> > > > >>> > > > Hi all!
> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> >>> > > >> >> > > > >>> > > > Based on the discussions most
> > probably
> > > > > this
> > > > > >> is
> > > > > >> >>> > going
> > > > > >> >>> > > to
> > > > > >> >>> > > >> >> be on
> > > > > >> >>> > > >> >> > > > >>> another
> > > > > >> >>> > > >> >> > > > >>> > > > GitHub repo. Therefore I will
> > prepare
> > > a
> > > > > >> >>> standalone
> > > > > >> >>> > > >> project
> > > > > >> >>> > > >> >> > with
> > > > > >> >>> > > >> >> > > > the
> > > > > >> >>> > > >> >> > > > >>> > > Pub/Sub
> > > > > >> >>> > > >> >> > > > >>> > > > module and once the repository is
> > > > created
> > > > > a
> > > > > >> >>> pull
> > > > > >> >>> > > >> request
> > > > > >> >>> > > >> >> > shall
> > > > > >> >>> > > >> >> > > be
> > > > > >> >>> > > >> >> > > > >>> > issued.
> > > > > >> >>> > > >> >> > > > >>> > > > If there is anything else I can do
> > in
> > > > > order
> > > > > >> to
> > > > > >> >>> > > >> facilitate
> > > > > >> >>> > > >> >> > this
> > > > > >> >>> > > >> >> > > > >>> please
> > > > > >> >>> > > >> >> > > > >>> > let
> > > > > >> >>> > > >> >> > > > >>> > > > me know.
> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> >>> > > >> >> > > > >>> > > > If a ticket is created for this
> > > issue, I
> > > > > >> shall
> > > > > >> >>> be
> > > > > >> >>> > > able
> > > > > >> >>> > > >> to
> > > > > >> >>> > > >> >> > > create
> > > > > >> >>> > > >> >> > > > a
> > > > > >> >>> > > >> >> > > > >>> pull
> > > > > >> >>> > > >> >> > > > >>> > > > request and thus a build will take
> > > place
> > > > > on
> > > > > >> >>> > ignite's
> > > > > >> >>> > > >> CI.
> > > > > >> >>> > > >> >> > > > >>> > > > If everyone is aligned with
> adding a
> > > > > Pub/Sub
> > > > > >> >>> > > >> >> implementation
> > > > > >> >>> > > >> >> > > may I
> > > > > >> >>> > > >> >> > > > >>> > create
> > > > > >> >>> > > >> >> > > > >>> > > a
> > > > > >> >>> > > >> >> > > > >>> > > > ticket on JIRA?
> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> >>> > > >> >> > > > >>> > > > My next question has to do on how
> > > Ignite
> > > > > >> >>> configures
> > > > > >> >>> > > the
> > > > > >> >>> > > >> >> build
> > > > > >> >>> > > >> >> > > > jobs
> > > > > >> >>> > > >> >> > > > >>> in
> > > > > >> >>> > > >> >> > > > >>> > > Team
> > > > > >> >>> > > >> >> > > > >>> > > > City. The unit tests for my
> > > > implementation
> > > > > >> >>> need a
> > > > > >> >>> > > local
> > > > > >> >>> > > >> >> > Pub/Sub
> > > > > >> >>> > > >> >> > > > >>> server
> > > > > >> >>> > > >> >> > > > >>> > up
> > > > > >> >>> > > >> >> > > > >>> > > > and running (they provide one for
> > > > testing
> > > > > >> >>> > purposes).
> > > > > >> >>> > > >> Will
> > > > > >> >>> > > >> >> > this
> > > > > >> >>> > > >> >> > > > take
> > > > > >> >>> > > >> >> > > > >>> > > effect
> > > > > >> >>> > > >> >> > > > >>> > > > in a form of a build script inside
> > my
> > > > > >> >>> > implementation
> > > > > >> >>> > > or
> > > > > >> >>> > > >> >> is it
> > > > > >> >>> > > >> >> > > > >>> something
> > > > > >> >>> > > >> >> > > > >>> > > > that should be configured on Team
> > > City?
> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> >>> > > >> >> > > > >>> > > > Kind regards,
> > > > > >> >>> > > >> >> > > > >>> > > > Emmanouil
> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> >>> > > >> >> > > > >>> > > > *Emmanouil Gkatziouras*
> > > > > >> >>> > > >> >> > > > >>> > > > https://egkatzioura.com/ |
> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > >> >>> > > >> >> > > > >>> > > > https://github.com/gkatzioura
> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> >>> > > >> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27,
> Denis
> > > > Magda
> > > > > <
> > > > > >> >>> > > >> >> dmagda@apache.org
> > > > > >> >>> > > >> >> > >
> > > > > >> >>> > > >> >> > > > >>> wrote:
> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > Emmanouil, Saikat,
> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > After contemplating on this, let
> > me
> > > > > >> propose
> > > > > >> >>> > another
> > > > > >> >>> > > >> way
> > > > > >> >>> > > >> >> > with
> > > > > >> >>> > > >> >> > > > this
> > > > > >> >>> > > >> >> > > > >>> > > > > particular integration that is
> > > aligned
> > > > > >> with
> > > > > >> >>> our
> > > > > >> >>> > > >> >> > > modularization
> > > > > >> >>> > > >> >> > > > >>> > > endeavor.
> > > > > >> >>> > > >> >> > > > >>> > > > > The modularization [1] defines
> the
> > > > > Ignite
> > > > > >> >>> core,
> > > > > >> >>> > > >> Ignite
> > > > > >> >>> > > >> >> > > modules,
> > > > > >> >>> > > >> >> > > > >>> and
> > > > > >> >>> > > >> >> > > > >>> > > > > independent integrations. The
> > first
> > > > two
> > > > > >> are
> > > > > >> >>> to be
> > > > > >> >>> > > >> >> located
> > > > > >> >>> > > >> >> > in
> > > > > >> >>> > > >> >> > > > >>> Ignite
> > > > > >> >>> > > >> >> > > > >>> > > > > repositories and to be
> > > tested/updated
> > > > by
> > > > > >> the
> > > > > >> >>> > > >> community
> > > > > >> >>> > > >> >> for
> > > > > >> >>> > > >> >> > > > every
> > > > > >> >>> > > >> >> > > > >>> > > release.
> > > > > >> >>> > > >> >> > > > >>> > > > > The modules are a subset of
> > existing
> > > > > >> >>> integrations
> > > > > >> >>> > > >> that
> > > > > >> >>> > > >> >> are
> > > > > >> >>> > > >> >> > > > >>> adopted
> > > > > >> >>> > > >> >> > > > >>> > > widely
> > > > > >> >>> > > >> >> > > > >>> > > > > and require community attention.
> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > As for the independent
> > integrations
> > > > [1],
> > > > > >> >>> those
> > > > > >> >>> > are
> > > > > >> >>> > > >> >> supposed
> > > > > >> >>> > > >> >> > > to
> > > > > >> >>> > > >> >> > > > be
> > > > > >> >>> > > >> >> > > > >>> > > located
> > > > > >> >>> > > >> >> > > > >>> > > > > in a separate GitHub repo. I
> would
> > > > > advise
> > > > > >> us
> > > > > >> >>> to
> > > > > >> >>> > > start
> > > > > >> >>> > > >> >> > > > >>> implementing
> > > > > >> >>> > > >> >> > > > >>> > this
> > > > > >> >>> > > >> >> > > > >>> > > > > concept with the Pub/Sub
> > > integration.
> > > > > Even
> > > > > >> >>> though
> > > > > >> >>> > > it
> > > > > >> >>> > > >> >> will
> > > > > >> >>> > > >> >> > be
> > > > > >> >>> > > >> >> > > > >>> located
> > > > > >> >>> > > >> >> > > > >>> > > in a
> > > > > >> >>> > > >> >> > > > >>> > > > > separate Github repo, it will be
> > > > listed
> > > > > >> among
> > > > > >> >>> > > >> officially
> > > > > >> >>> > > >> >> > > > >>> available
> > > > > >> >>> > > >> >> > > > >>> > > > > integrations on the Ignite
> website
> > > > > >> featuring
> > > > > >> >>> > > >> Emmanouil
> > > > > >> >>> > > >> >> as a
> > > > > >> >>> > > >> >> > > > >>> > > contributor.
> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > What do you think?
> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > [1]
> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> >>> > > >> >> > > > >>> > >
> > > > > >> >>> > > >> >> > > > >>> >
> > > > > >> >>> > > >> >> > > > >>>
> > > > > >> >>> > > >> >> > > >
> > > > > >> >>> > > >> >> > >
> > > > > >> >>> > > >> >> >
> > > > > >> >>> > > >> >>
> > > > > >> >>> > > >>
> > > > > >> >>> > >
> > > > > >> >>> >
> > > > > >> >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > -
> > > > > >> >>> > > >> >> > > > >>> > > > > Denis
> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM
> > > Saikat
> > > > > >> >>> Maitra <
> > > > > >> >>> > > >> >> > > > >>> > saikat.maitra@gmail.com
> > > > > >> >>> > > >> >> > > > >>> > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > wrote:
> > > > > >> >>> > > >> >> > > > >>> > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > > Hi Emmanouil,
> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > > The changes looks good to me.
> I
> > > > wanted
> > > > > >> to
> > > > > >> >>> check
> > > > > >> >>> > > if
> > > > > >> >>> > > >> you
> > > > > >> >>> > > >> >> > are
> > > > > >> >>> > > >> >> > > > also
> > > > > >> >>> > > >> >> > > > >>> > able
> > > > > >> >>> > > >> >> > > > >>> > > to
> > > > > >> >>> > > >> >> > > > >>> > > > > > validate that the once data is
> > > added
> > > > > to
> > > > > >> >>> Ignite
> > > > > >> >>> > > >> cluster
> > > > > >> >>> > > >> >> > you
> > > > > >> >>> > > >> >> > > > are
> > > > > >> >>> > > >> >> > > > >>> also
> > > > > >> >>> > > >> >> > > > >>> > > > able
> > > > > >> >>> > > >> >> > > > >>> > > > > to
> > > > > >> >>> > > >> >> > > > >>> > > > > > access it using another Ignite
> > > > client
> > > > > or
> > > > > >> >>> using
> > > > > >> >>> > > rest
> > > > > >> >>> > > >> >> > > > endpoints.
> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > > We use Teamcity for CI
> process,
> > > the
> > > > > >> >>> details are
> > > > > >> >>> > > as
> > > > > >> >>> > > >> >> > follows
> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > > Trigger validation of those
> test
> > > > > suites
> > > > > >> >>> that
> > > > > >> >>> > have
> > > > > >> >>> > > >> been
> > > > > >> >>> > > >> >> > > > >>> affected by
> > > > > >> >>> > > >> >> > > > >>> > > your
> > > > > >> >>> > > >> >> > > > >>> > > > > > changes on TeamCity <
> > > > > >> >>> > > http://ci.ignite.apache.org/>:
> > > > > >> >>> > > >> <
> > > > > >> >>> > > >> >> > > > >>> > > > > http://204.14.53.153/
> > > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Locate a test suite you
> > have
> > > to
> > > > > >> check,
> > > > > >> >>> > press
> > > > > >> >>> > > >> >> button
> > > > > >> >>> > > >> >> > > > named
> > > > > >> >>> > > >> >> > > > >>> > "..."
> > > > > >> >>> > > >> >> > > > >>> > > > that
> > > > > >> >>> > > >> >> > > > >>> > > > > >    is located on the left of
> > "Run"
> > > > > >> button.
> > > > > >> >>> "Run
> > > > > >> >>> > > >> custom
> > > > > >> >>> > > >> >> > > build"
> > > > > >> >>> > > >> >> > > > >>> > dialog
> > > > > >> >>> > > >> >> > > > >>> > > > will
> > > > > >> >>> > > >> >> > > > >>> > > > > >    appear;
> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Go to "Changes" tab and
> > > choose
> > > > > >> >>> > > >> >> > > > >>> > "pull/<pull-request-number>/head"
> > > > > >> >>> > > >> >> > > > >>> > > > in
> > > > > >> >>> > > >> >> > > > >>> > > > > >    "Build branch" dropdown
> list;
> > > > > >> >>> > > >> >> > > > >>> > > > > >    - Press "Run build" button
> > and
> > > > > >> monitor
> > > > > >> >>> tests
> > > > > >> >>> > > >> >> results.
> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > > @Roman rshtykh@yahoo.com -
> can
> > > you
> > > > > also
> > > > > >> >>> please
> > > > > >> >>> > > >> take a
> > > > > >> >>> > > >> >> > > look?
> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > > Regards,
> > > > > >> >>> > > >> >> > > > >>> > > > > > Saikat
> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20
> PM
> > > > > >> Emmanouil
> > > > > >> >>> > > >> Gkatziouras
> > > > > >> >>> > > >> >> <
> > > > > >> >>> > > >> >> > > > >>> > > > > > gkatzioura@gmail.com>
> > > > > >> >>> > > >> >> > > > >>> > > > > > wrote:
> > > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > > > Hi @Saikat Maitra
> > > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > > > I have my implementation on
> an
> > > > > Ignite
> > > > > >> >>> fork on
> > > > > >> >>> > > my
> > > > > >> >>> > > >> >> > personal
> > > > > >> >>> > > >> >> > > > >>> GitHub
> > > > > >> >>> > > >> >> > > > >>> > > > > account
> > > > > >> >>> > > >> >> > > > >>> > > > > > > [1]
> > > > > >> >>> > > >> >> > > > >>> > > > > > > It is based on the Kafka
> > > > > >> implementation
> > > > > >> >>> which
> > > > > >> >>> > > is
> > > > > >> >>> > > >> of
> > > > > >> >>> > > >> >> a
> > > > > >> >>> > > >> >> > > > polling
> > > > > >> >>> > > >> >> > > > >>> > > nature.
> > > > > >> >>> > > >> >> > > > >>> > > > > > > GCP does not provide a
> service
> > > > like
> > > > > >> >>> Kafka.
> > > > > >> >>> > > >> Instead
> > > > > >> >>> > > >> >> it
> > > > > >> >>> > > >> >> > > > >>> provides us
> > > > > >> >>> > > >> >> > > > >>> > > > with
> > > > > >> >>> > > >> >> > > > >>> > > > > > > Pub/Sub which is just like
> > > Kafka.
> > > > > >> >>> > > >> >> > > > >>> > > > > > > Therefore using the Kafka
> > > streamer
> > > > > as
> > > > > >> a
> > > > > >> >>> > > guideline
> > > > > >> >>> > > >> >> made
> > > > > >> >>> > > >> >> > > more
> > > > > >> >>> > > >> >> > > > >>> sense
> > > > > >> >>> > > >> >> > > > >>> > > to
> > > > > >> >>> > > >> >> > > > >>> > > > > me.
> > > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > > > >> >>> > > >> >> > > > >>> > > > > > > Apart from the
> implementation
> > I
> > > > have
> > > > > >> also
> > > > > >> >>> > > added a
> > > > > >> >>> > > >> >> unit
> > > > > >> >>> > > >> >> > > > test.
> > > > > >> >>> > > >> >> > > > >>> > > > > > > It definitely needs some
> > > polishing
> > > > > >> and a
> > > > > >> >>> > double
> > > > > >> >>> > > >> >> check
> > > > > >> >>> > > >> >> > on
> > > > > >> >>> > > >> >> > > my
> > > > > >> >>> > > >> >> > > > >>> side
> > > > > >> >>> > > >> >> > > > >>> > to
> > > > > >> >>> > > >> >> > > > >>> > > > > make
> > > > > >> >>> > > >> >> > > > >>> > > > > > > the review smooth.
> > > > > >> >>> > > >> >> > > > >>> > > > > > > For the tests a local
> Pub/Sub
> > > [2]
> > > > > >> server
> > > >

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Emmanouil Gkatziouras <gk...@gmail.com>]

Hi all,

I did add a test suite and run the build with the corresponding arguments,
however the tests did not run [1].
I checked the Flink's build logs and the same message is displayed there
too `No tests to run` [2].

[1]
https://ci.ignite.apache.org/viewLog.html?buildId=4798203&buildTypeId=IgniteExtensions_Build&tab=buildLog&branch_IgniteExtensions=pull%2F2%2Fhead&_focus=248
[2]
https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4789199&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=1256

Kind regards
*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura


On Wed, 27 Nov 2019 at 18:58, Saikat Maitra <sa...@gmail.com> wrote:

> Hi,
>
> We will need to add test suites for tests to be executed in build. We pass
> them as TestSuites param. I had added Flink sink and source testsuites.
>
> Can you please review and confirm.
>
> Regards
> Saikat
>
> On Wed, 27 Nov 2019 at 6:00 AM, Emmanouil Gkatziouras <
> gkatzioura@gmail.com>
> wrote:
>
> > Hi all!
> >
> > I was successful building and running my pull requests. It seems that the
> > tests do not run both for Flink and Pub/Sub [1]
> > If there is something I cannot do to make them discoverable please let me
> > know.
> >
> > [1]
> >
> >
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4796722&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=283
> >
> > *Emmanouil Gkatziouras*
> > https://egkatzioura.com/ |
> > https://www.linkedin.com/in/gkatziourasemmanouil/
> > https://github.com/gkatzioura
> >
> >
> > On Wed, 27 Nov 2019 at 04:57, Saikat Maitra <sa...@gmail.com>
> > wrote:
> >
> > > Hi Emmanouil,
> > >
> > > I have added you as contributor in Ignite Extensions project. Can you
> > > please check and confirm if you are able to see the project and execute
> > > build on your pull request.
> > >
> > > Regards,
> > > Saikat
> > >
> > > On Tue, Nov 26, 2019 at 12:49 PM Emmanouil Gkatziouras <
> > > gkatzioura@gmail.com>
> > > wrote:
> > >
> > > > Hi all!
> > > >
> > > > If someone could give me read access to the Job of Ignite Extensions
> on
> > > > Team City will greatly help me (username gkatzioura).
> > > > I suppose the builds get triggered automatically on a pull request.
> > > >
> > > > Kind regards,
> > > > Emmanouil
> > > >
> > > > *Emmanouil Gkatziouras*
> > > > https://egkatzioura.com/ |
> > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > https://github.com/gkatzioura
> > > >
> > > >
> > > > On Sun, 24 Nov 2019 at 21:11, Emmanouil Gkatziouras <
> > > gkatzioura@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi Saikat!
> > > > >
> > > > > I just rebased with the flink branch. Unfortunately I do not have
> > read
> > > > > access to the team city link you provided and thus evaluate the
> tests
> > > for
> > > > > my pull request.
> > > > > I guess it has to do with ignite extensions being new.
> > > > > My username is gkatzioura and email the one I am using on this
> email
> > > > >
> > > > > Thank you for your time.
> > > > >
> > > > >
> > > > > *Emmanouil Gkatziouras*
> > > > > https://egkatzioura.com/ |
> > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > > https://github.com/gkatzioura
> > > > >
> > > > >
> > > > > On Sun, 24 Nov 2019 at 18:03, Saikat Maitra <
> saikat.maitra@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > >> Hi Emmanouil,
> > > > >>
> > > > >> The latest build on teamcity has passed on Flink pull request.
> > > > >>
> > > > >>
> > > > >>
> > > >
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4788928&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> > > > >>
> > > > >> you should be able to take the changes I made in my PR and run
> build
> > > on
> > > > >> your pull request.
> > > > >>
> > > > >> Please let me know if you have any questions.
> > > > >>
> > > > >> Regards,
> > > > >> Saikat
> > > > >>
> > > > >> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra <
> > > saikat.maitra@gmail.com
> > > > >
> > > > >> wrote:
> > > > >>
> > > > >> > Hi Emmanouil,
> > > > >> >
> > > > >> > I have fixed the Flink module testsuites and have setup new
> > project
> > > in
> > > > >> > teamcity.
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >>
> > > >
> > >
> >
> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview
> > > > >> >
> > > > >> > I am looking into setting up the build.
> > > > >> >
> > > > >> > Regards,
> > > > >> > Saikat
> > > > >> >
> > > > >> > On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra <
> > > > saikat.maitra@gmail.com>
> > > > >> > wrote:
> > > > >> >
> > > > >> >> Hi Emmanouil,
> > > > >> >>
> > > > >> >> Thank you for your email. Yes, the plan is once the Flink PR is
> > > > merged
> > > > >> it
> > > > >> >> will provide the Licence, parent POMs etc and you can rebase
> from
> > > > >> master
> > > > >> >> branch and apply the changes on top of it.
> > > > >> >>
> > > > >> >> I am getting some issues with test failures in local with
> > > > >> >> GridTestProperties as the test.properties is not present in
> this
> > > new
> > > > >> >> project but should be available from dependencies.
> > > > >> >>
> > > > >> >> Once I address this issue, I will go ahead and merge the
> changes
> > > and
> > > > >> then
> > > > >> >> we can take it from there.
> > > > >> >>
> > > > >> >> Regards,
> > > > >> >> Saikat
> > > > >> >>
> > > > >> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras <
> > > > >> >> gkatzioura@gmail.com> wrote:
> > > > >> >>
> > > > >> >>> Hi all,
> > > > >> >>>
> > > > >> >>> I made my first pull request [1]. Since this project is brand
> > new
> > > > (no
> > > > >> >>> parent poms, licensing), I reckoned it was best to use
> Saikat's
> > > > >> branch on
> > > > >> >>> Flink.
> > > > >> >>> I suppose the Flink branch will be merged first. If not please
> > > give
> > > > me
> > > > >> >>> guidelines on how I should proceed next.
> > > > >> >>>
> > > > >> >>> Kind regards
> > > > >> >>> Emmanouil
> > > > >> >>>
> > > > >> >>> [1] https://github.com/apache/ignite-extensions/pull/2
> > > > >> >>>
> > > > >> >>> *Emmanouil Gkatziouras*
> > > > >> >>> https://egkatzioura.com/ |
> > > > >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > >> >>> https://github.com/gkatzioura
> > > > >> >>>
> > > > >> >>>
> > > > >> >>> On Fri, 22 Nov 2019 at 20:55, Denis Magda <dm...@apache.org>
> > > > wrote:
> > > > >> >>>
> > > > >> >>> > Awesome, ping us whenever you're ready!
> > > > >> >>> >
> > > > >> >>> > -
> > > > >> >>> > Denis
> > > > >> >>> >
> > > > >> >>> >
> > > > >> >>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <
> > > > >> >>> > gkatzioura@gmail.com>
> > > > >> >>> > wrote:
> > > > >> >>> >
> > > > >> >>> > > Hi all!
> > > > >> >>> > >
> > > > >> >>> > > I am sorry for being late on that. I was trying to
> refactor
> > > the
> > > > >> test
> > > > >> >>> in
> > > > >> >>> > > order not to be in need of any external tools or spinning
> > up a
> > > > >> >>> server.
> > > > >> >>> > > I did forked the new repo and indeed my changes there, so
> a
> > > pull
> > > > >> >>> request
> > > > >> >>> > is
> > > > >> >>> > > a matter of time!
> > > > >> >>> > >
> > > > >> >>> > > Kind regards
> > > > >> >>> > >
> > > > >> >>> > > *Emmanouil Gkatziouras*
> > > > >> >>> > > https://egkatzioura.com/ |
> > > > >> >>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > >> >>> > > https://github.com/gkatzioura
> > > > >> >>> > >
> > > > >> >>> > >
> > > > >> >>> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda <
> > dmagda@apache.org>
> > > > >> wrote:
> > > > >> >>> > >
> > > > >> >>> > > > Hi Emmanouil,
> > > > >> >>> > > >
> > > > >> >>> > > > Do you have any questions or need any support from the
> > > > >> community?
> > > > >> >>> > > >
> > > > >> >>> > > > -
> > > > >> >>> > > > Denis
> > > > >> >>> > > >
> > > > >> >>> > > >
> > > > >> >>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <
> > > > >> >>> saikat.maitra@gmail.com
> > > > >> >>> > >
> > > > >> >>> > > > wrote:
> > > > >> >>> > > >
> > > > >> >>> > > >> Hi Emmanouil,
> > > > >> >>> > > >>
> > > > >> >>> > > >> Can you please take a looks at dev utils, if this is
> > > > something
> > > > >> >>> you are
> > > > >> >>> > > >> looking for
> > > > >> >>> > > >>
> > > > https://github.com/apache/ignite/tree/master/modules/dev-utils
> > > > >> ?
> > > > >> >>> > > >>
> > > > >> >>> > > >> IMO, if you can release Pub/Sub server in maven and
> then
> > > use
> > > > >> it as
> > > > >> >>> > > >> dependency, that would be great.
> > > > >> >>> > > >>
> > > > >> >>> > > >> Regards,
> > > > >> >>> > > >> Saikat
> > > > >> >>> > > >>
> > > > >> >>> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <
> > > > >> >>> > saikat.maitra@gmail.com>
> > > > >> >>> > > >> wrote:
> > > > >> >>> > > >>
> > > > >> >>> > > >> > Hi Emmanouil,
> > > > >> >>> > > >> >
> > > > >> >>> > > >> > The master branch has been initialized. I have
> raised a
> > > PR
> > > > to
> > > > >> >>> > migrate
> > > > >> >>> > > >> > Flink module to ignite-extensions.
> > > > >> >>> > > >> >
> > > > >> >>> > > >> > PR
> https://github.com/apache/ignite-extensions/pull/1
> > > > >> >>> > > >> > Jira :
> > > https://issues.apache.org/jira/browse/IGNITE-12356
> > > > >> >>> > > >> >
> > > > >> >>> > > >> > Regards,
> > > > >> >>> > > >> > Saikat
> > > > >> >>> > > >> >
> > > > >> >>> > > >> >
> > > > >> >>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil
> Gkatziouras <
> > > > >> >>> > > >> > gkatzioura@gmail.com> wrote:
> > > > >> >>> > > >> >
> > > > >> >>> > > >> >> Hi all!
> > > > >> >>> > > >> >>
> > > > >> >>> > > >> >> This is an update. The integration is tested with
> > mocks
> > > > >> based
> > > > >> >>> on
> > > > >> >>> > the
> > > > >> >>> > > >> >> original Ignite repo [1].
> > > > >> >>> > > >> >> I am currently waiting for the first commits on the
> > new
> > > > >> >>> integration
> > > > >> >>> > > >> >> project
> > > > >> >>> > > >> >> to take place in order to create my pull request
> > > > accordingly
> > > > >> >>> to the
> > > > >> >>> > > >> >> initial
> > > > >> >>> > > >> >> commits.
> > > > >> >>> > > >> >> In order to simulate the Pub/Sub server in a test I
> > > used a
> > > > >> test
> > > > >> >>> > > server
> > > > >> >>> > > >> >> implementation [2]. Unfortunately they don't have
> this
> > > > util
> > > > >> >>> > uploaded
> > > > >> >>> > > >> in a
> > > > >> >>> > > >> >> maven repo, thus it needs to be hosted on a maven
> repo
> > > > >> (Apache
> > > > >> >>> > > License
> > > > >> >>> > > >> >> 2.0). Is there a repo for utilities that Ignite uses
> > for
> > > > the
> > > > >> >>> > > >> test/build?
> > > > >> >>> > > >> >> I have also created a ticket [3]
> > > > >> >>> > > >> >>
> > > > >> >>> > > >> >> Kind regards
> > > > >> >>> > > >> >>
> > > > >> >>> > > >> >> [1]
> > > > >> >>> >
> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > > > >> >>> > > >> >> [2]
> > > > >> >>> https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> > > > >> >>> > > >> >> [3]
> > https://issues.apache.org/jira/browse/IGNITE-12262
> > > > >> >>> > > >> >>
> > > > >> >>> > > >> >> *Emmanouil Gkatziouras*
> > > > >> >>> > > >> >> https://egkatzioura.com/ |
> > > > >> >>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > >> >>> > > >> >> https://github.com/gkatzioura
> > > > >> >>> > > >> >>
> > > > >> >>> > > >> >>
> > > > >> >>> > > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <
> > > > >> >>> > saikat.maitra@gmail.com
> > > > >> >>> > > >
> > > > >> >>> > > >> >> wrote:
> > > > >> >>> > > >> >>
> > > > >> >>> > > >> >> > Hello Ilya,
> > > > >> >>> > > >> >> >
> > > > >> >>> > > >> >> > Thank you for your email. Yes, before we remove
> any
> > > > >> >>> integration
> > > > >> >>> > we
> > > > >> >>> > > >> will
> > > > >> >>> > > >> >> > move them to another repository and release.
> > > > >> >>> > > >> >> >
> > > > >> >>> > > >> >> > We are thinking we will start with Pub/Sub
> Streamer
> > > > >> >>> > implementation
> > > > >> >>> > > >> and
> > > > >> >>> > > >> >> > continue to move other integration as mentioned in
> > the
> > > > >> >>> following
> > > > >> >>> > > doc.
> > > > >> >>> > > >> >> >
> > > > >> >>> > > >> >> >
> > > > >> >>> > > >> >> >
> > > > >> >>> > > >> >>
> > > > >> >>> > > >>
> > > > >> >>> > >
> > > > >> >>> >
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > >> >>> > > >> >> >
> > > > >> >>> > > >> >> > Regards,
> > > > >> >>> > > >> >> > Saikat
> > > > >> >>> > > >> >> >
> > > > >> >>> > > >> >> >
> > > > >> >>> > > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
> > > > >> >>> > > >> >> > ilya.kasnacheev@gmail.com>
> > > > >> >>> > > >> >> > wrote:
> > > > >> >>> > > >> >> >
> > > > >> >>> > > >> >> > > Hello!
> > > > >> >>> > > >> >> > >
> > > > >> >>> > > >> >> > > My take on this: we want to offload some of
> > > > >> integrations,
> > > > >> >>> but
> > > > >> >>> > > >> please
> > > > >> >>> > > >> >> > commit
> > > > >> >>> > > >> >> > > and release them to another project first, and
> > only
> > > > >> then we
> > > > >> >>> > will
> > > > >> >>> > > >> >> decide
> > > > >> >>> > > >> >> > to
> > > > >> >>> > > >> >> > > remove them from our repository.
> > > > >> >>> > > >> >> > >
> > > > >> >>> > > >> >> > > Please also make sure they are available with
> same
> > > > >> artifact
> > > > >> >>> > name
> > > > >> >>> > > >> and
> > > > >> >>> > > >> >> API.
> > > > >> >>> > > >> >> > > If this has to be changed, let's wait for AI 3.0
> > > > >> >>> > > >> >> > >
> > > > >> >>> > > >> >> > > Regards,
> > > > >> >>> > > >> >> > > --
> > > > >> >>> > > >> >> > > Ilya Kasnacheev
> > > > >> >>> > > >> >> > >
> > > > >> >>> > > >> >> > >
> > > > >> >>> > > >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <
> > > > >> >>> > > >> saikat.maitra@gmail.com>:
> > > > >> >>> > > >> >> > >
> > > > >> >>> > > >> >> > > > Hello Emmanouil,
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > > > As discussed earlier, I discussed with Apache
> > > Bahir
> > > > >> >>> community
> > > > >> >>> > > and
> > > > >> >>> > > >> >> they
> > > > >> >>> > > >> >> > > are
> > > > >> >>> > > >> >> > > > interested to have Apache Ignite extensions as
> > > part
> > > > of
> > > > >> >>> Apache
> > > > >> >>> > > >> Bahir
> > > > >> >>> > > >> >> > > > project.
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > > > Can you please share Pub/Sub streamer
> > > implementation
> > > > >> >>> details
> > > > >> >>> > > with
> > > > >> >>> > > >> >> > Apache
> > > > >> >>> > > >> >> > > > Bahir community and request for feedback.
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > > > https://bahir.apache.org/
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > > > I have also requested for contributor access
> in
> > > Jira
> > > > >> for
> > > > >> >>> > Apache
> > > > >> >>> > > >> >> Bahir
> > > > >> >>> > > >> >> > > > project so that I can create issues and assign
> > to
> > > > >> myself.
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > > > I can help with code reviews as well.
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > > > Here is the email thread for reference
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > >
> > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > > > Regards,
> > > > >> >>> > > >> >> > > > Saikat
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra
> <
> > > > >> >>> > > >> >> saikat.maitra@gmail.com
> > > > >> >>> > > >> >> > >
> > > > >> >>> > > >> >> > > > wrote:
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > > > > Hello,
> > > > >> >>> > > >> >> > > > >
> > > > >> >>> > > >> >> > > > > As discussed I have created following
> > discussion
> > > > >> >>> threads on
> > > > >> >>> > > our
> > > > >> >>> > > >> >> > > migration
> > > > >> >>> > > >> >> > > > > proposals for Apache Ignite extensions:
> > > > >> >>> > > >> >> > > > >
> > > > >> >>> > > >> >> > > > >
> > > > >> >>> > > >> >> > > > >
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > >
> > > > >> >>> > > >> >> >
> > > > >> >>> > > >> >>
> > > > >> >>> > > >>
> > > > >> >>> > >
> > > > >> >>> >
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> > > > >> >>> > > >> >> > > > >
> > > > >> >>> > > >> >> > > > >
> > > > >> >>> > > >>
> > > > >> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > > > >> >>> > > >> >> > > > >
> > > > >> >>> > > >> >> > > > > I will share update as I hear more
> > information.
> > > > >> >>> > > >> >> > > > >
> > > > >> >>> > > >> >> > > > > Regards,
> > > > >> >>> > > >> >> > > > > Saikat
> > > > >> >>> > > >> >> > > > >
> > > > >> >>> > > >> >> > > > >
> > > > >> >>> > > >> >> > > > >
> > > > >> >>> > > >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat
> Maitra
> > <
> > > > >> >>> > > >> >> > saikat.maitra@gmail.com
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > > > > wrote:
> > > > >> >>> > > >> >> > > > >
> > > > >> >>> > > >> >> > > > >> Hello Denis,
> > > > >> >>> > > >> >> > > > >>
> > > > >> >>> > > >> >> > > > >> Sure, sounds good. I will create a separate
> > > > >> discussion
> > > > >> >>> > > thread
> > > > >> >>> > > >> to
> > > > >> >>> > > >> >> get
> > > > >> >>> > > >> >> > > > >> community feedback on whether we
> > > > >> >>> > > >> >> > > > >> should join the Bahir or kick-off "Ignite
> > > > >> Extensions"
> > > > >> >>> > > project.
> > > > >> >>> > > >> >> > > > >>
> > > > >> >>> > > >> >> > > > >> Regards,
> > > > >> >>> > > >> >> > > > >> Saikat
> > > > >> >>> > > >> >> > > > >>
> > > > >> >>> > > >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis
> Magda <
> > > > >> >>> > > >> dmagda@apache.org>
> > > > >> >>> > > >> >> > > wrote:
> > > > >> >>> > > >> >> > > > >>
> > > > >> >>> > > >> >> > > > >>> Folks,
> > > > >> >>> > > >> >> > > > >>>
> > > > >> >>> > > >> >> > > > >>> The concept of Apache Bahir is precisely
> > what
> > > we
> > > > >> >>> need!
> > > > >> >>> > > >> Saikat,
> > > > >> >>> > > >> >> > thanks
> > > > >> >>> > > >> >> > > > for
> > > > >> >>> > > >> >> > > > >>> doing the research. Why I believe the *
> > idea*
> > > > >> fits us
> > > > >> >>> > well:
> > > > >> >>> > > >> >> > > > >>>
> > > > >> >>> > > >> >> > > > >>>    - All integrations can be stored in
> > > separate
> > > > >> >>> Github
> > > > >> >>> > > >> >> repositories
> > > > >> >>> > > >> >> > > and
> > > > >> >>> > > >> >> > > > >>>    have their dev lifecycles. We've not
> > > obliged
> > > > to
> > > > >> >>> couple
> > > > >> >>> > > all
> > > > >> >>> > > >> >> the
> > > > >> >>> > > >> >> > > > >>> integrations
> > > > >> >>> > > >> >> > > > >>>    in a single repo. For instance, Spark
> can
> > > be
> > > > >> >>> located
> > > > >> >>> > in
> > > > >> >>> > > a
> > > > >> >>> > > >> >> > > dedicated
> > > > >> >>> > > >> >> > > > >>> repo
> > > > >> >>> > > >> >> > > > >>>    while streaming integrations might be
> in
> > a
> > > > >> single
> > > > >> >>> one.
> > > > >> >>> > > >> It's
> > > > >> >>> > > >> >> up
> > > > >> >>> > > >> >> > to
> > > > >> >>> > > >> >> > > > us.
> > > > >> >>> > > >> >> > > > >>>    - All the repositories and integrations
> > > > belong
> > > > >> to
> > > > >> >>> ASF.
> > > > >> >>> > > >> We're
> > > > >> >>> > > >> >> not
> > > > >> >>> > > >> >> > > > >>> dumping
> > > > >> >>> > > >> >> > > > >>>    them on Github but rather keep
> supporting
> > > and
> > > > >> >>> > developing
> > > > >> >>> > > >> in
> > > > >> >>> > > >> >> > > > >>> accordance with
> > > > >> >>> > > >> >> > > > >>>    ASF vision and practices.
> > > > >> >>> > > >> >> > > > >>>    - There is a way to reward contributors
> > via
> > > > >> >>> > > committership
> > > > >> >>> > > >> and
> > > > >> >>> > > >> >> > PMC
> > > > >> >>> > > >> >> > > > >>>    membership. You won't get this if a
> > project
> > > > is
> > > > >> >>> just
> > > > >> >>> > one
> > > > >> >>> > > of
> > > > >> >>> > > >> >> the
> > > > >> >>> > > >> >> > > > >>> millions of
> > > > >> >>> > > >> >> > > > >>>    Github projects.
> > > > >> >>> > > >> >> > > > >>>
> > > > >> >>> > > >> >> > > > >>> Saikat, as Ignite PMC member, could you
> > please
> > > > >> >>> kick-off a
> > > > >> >>> > > >> >> separate
> > > > >> >>> > > >> >> > > > >>> dev-list
> > > > >> >>> > > >> >> > > > >>> discussion to involve more community
> members
> > > and
> > > > >> >>> > referring
> > > > >> >>> > > to
> > > > >> >>> > > >> >> this
> > > > >> >>> > > >> >> > > > >>> thread?
> > > > >> >>> > > >> >> > > > >>> I think the primary question the community
> > > needs
> > > > >> to
> > > > >> >>> > answer
> > > > >> >>> > > >> >> whether
> > > > >> >>> > > >> >> > we
> > > > >> >>> > > >> >> > > > >>> should join the Bahir or kick-off "Ignite
> > > > >> Extensions"
> > > > >> >>> > > >> project?
> > > > >> >>> > > >> >> > > > >>>
> > > > >> >>> > > >> >> > > > >>> -
> > > > >> >>> > > >> >> > > > >>> Denis
> > > > >> >>> > > >> >> > > > >>>
> > > > >> >>> > > >> >> > > > >>>
> > > > >> >>> > > >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey
> > > Zinoviev
> > > > <
> > > > >> >>> > > >> >> > > > zaleslaw.sin@gmail.com>
> > > > >> >>> > > >> >> > > > >>> wrote:
> > > > >> >>> > > >> >> > > > >>>
> > > > >> >>> > > >> >> > > > >>> > Maybe we could move all our Streaming
> > > > >> Integrations
> > > > >> >>> > there,
> > > > >> >>> > > >> but
> > > > >> >>> > > >> >> > what
> > > > >> >>> > > >> >> > > is
> > > > >> >>> > > >> >> > > > >>> about
> > > > >> >>> > > >> >> > > > >>> > maintaining and committer permissions to
> > the
> > > > new
> > > > >> >>> > > >> repositories?
> > > > >> >>> > > >> >> > > > >>> > I see the list of the committers and PMC
> > > > members
> > > > >> >>> there
> > > > >> >>> > > >> >> > > > >>> >
> > https://bahir.apache.org/community-members/
> > > > >> >>> > > >> >> > > > >>> >
> > > > >> >>> > > >> >> > > > >>> > Could somebody from Ignite community be
> > > added
> > > > to
> > > > >> >>> this
> > > > >> >>> > > list
> > > > >> >>> > > >> as
> > > > >> >>> > > >> >> a
> > > > >> >>> > > >> >> > > > >>> > PMC/committer to maintain Ignite
> > > integrations?
> > > > >> >>> > > >> >> > > > >>> > If yes, I'd happy to join this project
> and
> > > > >> >>> collaborate
> > > > >> >>> > > with
> > > > >> >>> > > >> >> these
> > > > >> >>> > > >> >> > > > guys
> > > > >> >>> > > >> >> > > > >>> > together
> > > > >> >>> > > >> >> > > > >>> > If no, and we should start from the zero
> > > level
> > > > >> with
> > > > >> >>> > > >> external
> > > > >> >>> > > >> >> PRs
> > > > >> >>> > > >> >> > -
> > > > >> >>> > > >> >> > > > >>> hmmm,
> > > > >> >>> > > >> >> > > > >>> > it's better to have our own external
> > > > repository
> > > > >> >>> with
> > > > >> >>> > > >> >> ApacheBahirr
> > > > >> >>> > > >> >> > > > >>> ideology.
> > > > >> >>> > > >> >> > > > >>> >
> > > > >> >>> > > >> >> > > > >>> > Also, I agree, that all connectors could
> > be
> > > > >> moved
> > > > >> >>> there
> > > > >> >>> > > (in
> > > > >> >>> > > >> >> > > > >>> ApacheBahirr
> > > > >> >>> > > >> >> > > > >>> > like repository), but not all modules
> from
> > > the
> > > > >> page
> > > > >> >>> > > >> >> > > > >>> >
> > > > >> >>> > > >> >> > > > >>> >
> > > > >> >>> > > >> >> > > > >>>
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > >
> > > > >> >>> > > >> >> >
> > > > >> >>> > > >> >>
> > > > >> >>> > > >>
> > > > >> >>> > >
> > > > >> >>> >
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > >> >>> > > >> >> > > > >>> > because
> > > > >> >>> > > >> >> > > > >>> > they use different parts of core modules
> > and
> > > > >> could
> > > > >> >>> be
> > > > >> >>> > > >> >> developed
> > > > >> >>> > > >> >> > > with
> > > > >> >>> > > >> >> > > > >>> > different velocity.
> > > > >> >>> > > >> >> > > > >>> >
> > > > >> >>> > > >> >> > > > >>> > In reality, before creation of new
> > > > repositories
> > > > >> we
> > > > >> >>> need
> > > > >> >>> > > >> wide
> > > > >> >>> > > >> >> > > > discussion
> > > > >> >>> > > >> >> > > > >>> > based on the document mentioned above.
> > > > >> >>> > > >> >> > > > >>> >
> > > > >> >>> > > >> >> > > > >>> > P.S. Of course, we could experiment in
> the
> > > > next
> > > > >> >>> release
> > > > >> >>> > > 2.8
> > > > >> >>> > > >> >> with
> > > > >> >>> > > >> >> > > one
> > > > >> >>> > > >> >> > > > >>> or two
> > > > >> >>> > > >> >> > > > >>> > integrations like twitter or ZeroMQ
> > > > >> >>> > > >> >> > > > >>> > Also, I'd like Denis Magda idea for the
> > > > separate
> > > > >> >>> > > >> repositories
> > > > >> >>> > > >> >> in
> > > > >> >>> > > >> >> > > > Apache
> > > > >> >>> > > >> >> > > > >>> > Github to maintain them by anybody.
> > > > >> >>> > > >> >> > > > >>> >
> > > > >> >>> > > >> >> > > > >>> >
> > > > >> >>> > > >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat
> > Maitra <
> > > > >> >>> > > >> >> > > saikat.maitra@gmail.com
> > > > >> >>> > > >> >> > > > >:
> > > > >> >>> > > >> >> > > > >>> >
> > > > >> >>> > > >> >> > > > >>> > > Hi Denis, Emmanouil
> > > > >> >>> > > >> >> > > > >>> > >
> > > > >> >>> > > >> >> > > > >>> > > Thank you for your email. I think
> > > creating a
> > > > >> >>> separate
> > > > >> >>> > > >> >> > integration
> > > > >> >>> > > >> >> > > > >>> project
> > > > >> >>> > > >> >> > > > >>> > > in github and also proposing it as an
> > > apache
> > > > >> >>> > incubator
> > > > >> >>> > > >> >> project
> > > > >> >>> > > >> >> > > will
> > > > >> >>> > > >> >> > > > >>> be a
> > > > >> >>> > > >> >> > > > >>> > > good move. The other separate
> > integration
> > > > >> modules
> > > > >> >>> > > >> >> > > > >>> > >
> > > > >> >>> > > >> >> > > > >>> > >
> > > > >> >>> > > >> >> > > > >>> >
> > > > >> >>> > > >> >> > > > >>>
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > >
> > > > >> >>> > > >> >> >
> > > > >> >>> > > >> >>
> > > > >> >>> > > >>
> > > > >> >>> > >
> > > > >> >>> >
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > >> >>> > > >> >> > > > >>> > > can
> > > > >> >>> > > >> >> > > > >>> > > be moved to this new apache incubator
> > > > project.
> > > > >> >>> > > >> >> > > > >>> > >
> > > > >> >>> > > >> >> > > > >>> > > There are similar integration
> available
> > > for
> > > > >> >>> Flink and
> > > > >> >>> > > >> Spark
> > > > >> >>> > > >> >> in
> > > > >> >>> > > >> >> > > > Apache
> > > > >> >>> > > >> >> > > > >>> > Bahir
> > > > >> >>> > > >> >> > > > >>> > > https://bahir.apache.org/
> > > > >> >>> > > >> >> > > > >>> > >
> > > > >> >>> > > >> >> > > > >>> > > Do you think we should reach out to
> > Apache
> > > > >> Bahir
> > > > >> >>> > > >> community
> > > > >> >>> > > >> >> > with a
> > > > >> >>> > > >> >> > > > >>> > proposal
> > > > >> >>> > > >> >> > > > >>> > > or we should plan to create a separate
> > > > Apache
> > > > >> >>> > Incubator
> > > > >> >>> > > >> >> > project?
> > > > >> >>> > > >> >> > > > >>> > >
> > > > >> >>> > > >> >> > > > >>> > > Regards,
> > > > >> >>> > > >> >> > > > >>> > > Saikat
> > > > >> >>> > > >> >> > > > >>> > >
> > > > >> >>> > > >> >> > > > >>> > >
> > > > >> >>> > > >> >> > > > >>> > >
> > > > >> >>> > > >> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM
> > Emmanouil
> > > > >> >>> > Gkatziouras <
> > > > >> >>> > > >> >> > > > >>> > > gkatzioura@gmail.com>
> > > > >> >>> > > >> >> > > > >>> > > wrote:
> > > > >> >>> > > >> >> > > > >>> > >
> > > > >> >>> > > >> >> > > > >>> > > > Hi all!
> > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> >>> > > >> >> > > > >>> > > > Based on the discussions most
> probably
> > > > this
> > > > >> is
> > > > >> >>> > going
> > > > >> >>> > > to
> > > > >> >>> > > >> >> be on
> > > > >> >>> > > >> >> > > > >>> another
> > > > >> >>> > > >> >> > > > >>> > > > GitHub repo. Therefore I will
> prepare
> > a
> > > > >> >>> standalone
> > > > >> >>> > > >> project
> > > > >> >>> > > >> >> > with
> > > > >> >>> > > >> >> > > > the
> > > > >> >>> > > >> >> > > > >>> > > Pub/Sub
> > > > >> >>> > > >> >> > > > >>> > > > module and once the repository is
> > > created
> > > > a
> > > > >> >>> pull
> > > > >> >>> > > >> request
> > > > >> >>> > > >> >> > shall
> > > > >> >>> > > >> >> > > be
> > > > >> >>> > > >> >> > > > >>> > issued.
> > > > >> >>> > > >> >> > > > >>> > > > If there is anything else I can do
> in
> > > > order
> > > > >> to
> > > > >> >>> > > >> facilitate
> > > > >> >>> > > >> >> > this
> > > > >> >>> > > >> >> > > > >>> please
> > > > >> >>> > > >> >> > > > >>> > let
> > > > >> >>> > > >> >> > > > >>> > > > me know.
> > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> >>> > > >> >> > > > >>> > > > If a ticket is created for this
> > issue, I
> > > > >> shall
> > > > >> >>> be
> > > > >> >>> > > able
> > > > >> >>> > > >> to
> > > > >> >>> > > >> >> > > create
> > > > >> >>> > > >> >> > > > a
> > > > >> >>> > > >> >> > > > >>> pull
> > > > >> >>> > > >> >> > > > >>> > > > request and thus a build will take
> > place
> > > > on
> > > > >> >>> > ignite's
> > > > >> >>> > > >> CI.
> > > > >> >>> > > >> >> > > > >>> > > > If everyone is aligned with adding a
> > > > Pub/Sub
> > > > >> >>> > > >> >> implementation
> > > > >> >>> > > >> >> > > may I
> > > > >> >>> > > >> >> > > > >>> > create
> > > > >> >>> > > >> >> > > > >>> > > a
> > > > >> >>> > > >> >> > > > >>> > > > ticket on JIRA?
> > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> >>> > > >> >> > > > >>> > > > My next question has to do on how
> > Ignite
> > > > >> >>> configures
> > > > >> >>> > > the
> > > > >> >>> > > >> >> build
> > > > >> >>> > > >> >> > > > jobs
> > > > >> >>> > > >> >> > > > >>> in
> > > > >> >>> > > >> >> > > > >>> > > Team
> > > > >> >>> > > >> >> > > > >>> > > > City. The unit tests for my
> > > implementation
> > > > >> >>> need a
> > > > >> >>> > > local
> > > > >> >>> > > >> >> > Pub/Sub
> > > > >> >>> > > >> >> > > > >>> server
> > > > >> >>> > > >> >> > > > >>> > up
> > > > >> >>> > > >> >> > > > >>> > > > and running (they provide one for
> > > testing
> > > > >> >>> > purposes).
> > > > >> >>> > > >> Will
> > > > >> >>> > > >> >> > this
> > > > >> >>> > > >> >> > > > take
> > > > >> >>> > > >> >> > > > >>> > > effect
> > > > >> >>> > > >> >> > > > >>> > > > in a form of a build script inside
> my
> > > > >> >>> > implementation
> > > > >> >>> > > or
> > > > >> >>> > > >> >> is it
> > > > >> >>> > > >> >> > > > >>> something
> > > > >> >>> > > >> >> > > > >>> > > > that should be configured on Team
> > City?
> > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> >>> > > >> >> > > > >>> > > > Kind regards,
> > > > >> >>> > > >> >> > > > >>> > > > Emmanouil
> > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> >>> > > >> >> > > > >>> > > > *Emmanouil Gkatziouras*
> > > > >> >>> > > >> >> > > > >>> > > > https://egkatzioura.com/ |
> > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > >> >>> > > >> >> > > > >>> > > > https://github.com/gkatzioura
> > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> >>> > > >> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis
> > > Magda
> > > > <
> > > > >> >>> > > >> >> dmagda@apache.org
> > > > >> >>> > > >> >> > >
> > > > >> >>> > > >> >> > > > >>> wrote:
> > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> >>> > > >> >> > > > >>> > > > > Emmanouil, Saikat,
> > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > After contemplating on this, let
> me
> > > > >> propose
> > > > >> >>> > another
> > > > >> >>> > > >> way
> > > > >> >>> > > >> >> > with
> > > > >> >>> > > >> >> > > > this
> > > > >> >>> > > >> >> > > > >>> > > > > particular integration that is
> > aligned
> > > > >> with
> > > > >> >>> our
> > > > >> >>> > > >> >> > > modularization
> > > > >> >>> > > >> >> > > > >>> > > endeavor.
> > > > >> >>> > > >> >> > > > >>> > > > > The modularization [1] defines the
> > > > Ignite
> > > > >> >>> core,
> > > > >> >>> > > >> Ignite
> > > > >> >>> > > >> >> > > modules,
> > > > >> >>> > > >> >> > > > >>> and
> > > > >> >>> > > >> >> > > > >>> > > > > independent integrations. The
> first
> > > two
> > > > >> are
> > > > >> >>> to be
> > > > >> >>> > > >> >> located
> > > > >> >>> > > >> >> > in
> > > > >> >>> > > >> >> > > > >>> Ignite
> > > > >> >>> > > >> >> > > > >>> > > > > repositories and to be
> > tested/updated
> > > by
> > > > >> the
> > > > >> >>> > > >> community
> > > > >> >>> > > >> >> for
> > > > >> >>> > > >> >> > > > every
> > > > >> >>> > > >> >> > > > >>> > > release.
> > > > >> >>> > > >> >> > > > >>> > > > > The modules are a subset of
> existing
> > > > >> >>> integrations
> > > > >> >>> > > >> that
> > > > >> >>> > > >> >> are
> > > > >> >>> > > >> >> > > > >>> adopted
> > > > >> >>> > > >> >> > > > >>> > > widely
> > > > >> >>> > > >> >> > > > >>> > > > > and require community attention.
> > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > As for the independent
> integrations
> > > [1],
> > > > >> >>> those
> > > > >> >>> > are
> > > > >> >>> > > >> >> supposed
> > > > >> >>> > > >> >> > > to
> > > > >> >>> > > >> >> > > > be
> > > > >> >>> > > >> >> > > > >>> > > located
> > > > >> >>> > > >> >> > > > >>> > > > > in a separate GitHub repo. I would
> > > > advise
> > > > >> us
> > > > >> >>> to
> > > > >> >>> > > start
> > > > >> >>> > > >> >> > > > >>> implementing
> > > > >> >>> > > >> >> > > > >>> > this
> > > > >> >>> > > >> >> > > > >>> > > > > concept with the Pub/Sub
> > integration.
> > > > Even
> > > > >> >>> though
> > > > >> >>> > > it
> > > > >> >>> > > >> >> will
> > > > >> >>> > > >> >> > be
> > > > >> >>> > > >> >> > > > >>> located
> > > > >> >>> > > >> >> > > > >>> > > in a
> > > > >> >>> > > >> >> > > > >>> > > > > separate Github repo, it will be
> > > listed
> > > > >> among
> > > > >> >>> > > >> officially
> > > > >> >>> > > >> >> > > > >>> available
> > > > >> >>> > > >> >> > > > >>> > > > > integrations on the Ignite website
> > > > >> featuring
> > > > >> >>> > > >> Emmanouil
> > > > >> >>> > > >> >> as a
> > > > >> >>> > > >> >> > > > >>> > > contributor.
> > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > What do you think?
> > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > [1]
> > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> >>> > > >> >> > > > >>> > >
> > > > >> >>> > > >> >> > > > >>> >
> > > > >> >>> > > >> >> > > > >>>
> > > > >> >>> > > >> >> > > >
> > > > >> >>> > > >> >> > >
> > > > >> >>> > > >> >> >
> > > > >> >>> > > >> >>
> > > > >> >>> > > >>
> > > > >> >>> > >
> > > > >> >>> >
> > > > >> >>>
> > > > >>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > -
> > > > >> >>> > > >> >> > > > >>> > > > > Denis
> > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM
> > Saikat
> > > > >> >>> Maitra <
> > > > >> >>> > > >> >> > > > >>> > saikat.maitra@gmail.com
> > > > >> >>> > > >> >> > > > >>> > > >
> > > > >> >>> > > >> >> > > > >>> > > > > wrote:
> > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > Hi Emmanouil,
> > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > The changes looks good to me. I
> > > wanted
> > > > >> to
> > > > >> >>> check
> > > > >> >>> > > if
> > > > >> >>> > > >> you
> > > > >> >>> > > >> >> > are
> > > > >> >>> > > >> >> > > > also
> > > > >> >>> > > >> >> > > > >>> > able
> > > > >> >>> > > >> >> > > > >>> > > to
> > > > >> >>> > > >> >> > > > >>> > > > > > validate that the once data is
> > added
> > > > to
> > > > >> >>> Ignite
> > > > >> >>> > > >> cluster
> > > > >> >>> > > >> >> > you
> > > > >> >>> > > >> >> > > > are
> > > > >> >>> > > >> >> > > > >>> also
> > > > >> >>> > > >> >> > > > >>> > > > able
> > > > >> >>> > > >> >> > > > >>> > > > > to
> > > > >> >>> > > >> >> > > > >>> > > > > > access it using another Ignite
> > > client
> > > > or
> > > > >> >>> using
> > > > >> >>> > > rest
> > > > >> >>> > > >> >> > > > endpoints.
> > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > We use Teamcity for CI process,
> > the
> > > > >> >>> details are
> > > > >> >>> > > as
> > > > >> >>> > > >> >> > follows
> > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > Trigger validation of those test
> > > > suites
> > > > >> >>> that
> > > > >> >>> > have
> > > > >> >>> > > >> been
> > > > >> >>> > > >> >> > > > >>> affected by
> > > > >> >>> > > >> >> > > > >>> > > your
> > > > >> >>> > > >> >> > > > >>> > > > > > changes on TeamCity <
> > > > >> >>> > > http://ci.ignite.apache.org/>:
> > > > >> >>> > > >> <
> > > > >> >>> > > >> >> > > > >>> > > > > http://204.14.53.153/
> > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > >    - Locate a test suite you
> have
> > to
> > > > >> check,
> > > > >> >>> > press
> > > > >> >>> > > >> >> button
> > > > >> >>> > > >> >> > > > named
> > > > >> >>> > > >> >> > > > >>> > "..."
> > > > >> >>> > > >> >> > > > >>> > > > that
> > > > >> >>> > > >> >> > > > >>> > > > > >    is located on the left of
> "Run"
> > > > >> button.
> > > > >> >>> "Run
> > > > >> >>> > > >> custom
> > > > >> >>> > > >> >> > > build"
> > > > >> >>> > > >> >> > > > >>> > dialog
> > > > >> >>> > > >> >> > > > >>> > > > will
> > > > >> >>> > > >> >> > > > >>> > > > > >    appear;
> > > > >> >>> > > >> >> > > > >>> > > > > >    - Go to "Changes" tab and
> > choose
> > > > >> >>> > > >> >> > > > >>> > "pull/<pull-request-number>/head"
> > > > >> >>> > > >> >> > > > >>> > > > in
> > > > >> >>> > > >> >> > > > >>> > > > > >    "Build branch" dropdown list;
> > > > >> >>> > > >> >> > > > >>> > > > > >    - Press "Run build" button
> and
> > > > >> monitor
> > > > >> >>> tests
> > > > >> >>> > > >> >> results.
> > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > @Roman rshtykh@yahoo.com - can
> > you
> > > > also
> > > > >> >>> please
> > > > >> >>> > > >> take a
> > > > >> >>> > > >> >> > > look?
> > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > Regards,
> > > > >> >>> > > >> >> > > > >>> > > > > > Saikat
> > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20 PM
> > > > >> Emmanouil
> > > > >> >>> > > >> Gkatziouras
> > > > >> >>> > > >> >> <
> > > > >> >>> > > >> >> > > > >>> > > > > > gkatzioura@gmail.com>
> > > > >> >>> > > >> >> > > > >>> > > > > > wrote:
> > > > >> >>> > > >> >> > > > >>> > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > Hi @Saikat Maitra
> > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > I have my implementation on an
> > > > Ignite
> > > > >> >>> fork on
> > > > >> >>> > > my
> > > > >> >>> > > >> >> > personal
> > > > >> >>> > > >> >> > > > >>> GitHub
> > > > >> >>> > > >> >> > > > >>> > > > > account
> > > > >> >>> > > >> >> > > > >>> > > > > > > [1]
> > > > >> >>> > > >> >> > > > >>> > > > > > > It is based on the Kafka
> > > > >> implementation
> > > > >> >>> which
> > > > >> >>> > > is
> > > > >> >>> > > >> of
> > > > >> >>> > > >> >> a
> > > > >> >>> > > >> >> > > > polling
> > > > >> >>> > > >> >> > > > >>> > > nature.
> > > > >> >>> > > >> >> > > > >>> > > > > > > GCP does not provide a service
> > > like
> > > > >> >>> Kafka.
> > > > >> >>> > > >> Instead
> > > > >> >>> > > >> >> it
> > > > >> >>> > > >> >> > > > >>> provides us
> > > > >> >>> > > >> >> > > > >>> > > > with
> > > > >> >>> > > >> >> > > > >>> > > > > > > Pub/Sub which is just like
> > Kafka.
> > > > >> >>> > > >> >> > > > >>> > > > > > > Therefore using the Kafka
> > streamer
> > > > as
> > > > >> a
> > > > >> >>> > > guideline
> > > > >> >>> > > >> >> made
> > > > >> >>> > > >> >> > > more
> > > > >> >>> > > >> >> > > > >>> sense
> > > > >> >>> > > >> >> > > > >>> > > to
> > > > >> >>> > > >> >> > > > >>> > > > > me.
> > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > Apart from the implementation
> I
> > > have
> > > > >> also
> > > > >> >>> > > added a
> > > > >> >>> > > >> >> unit
> > > > >> >>> > > >> >> > > > test.
> > > > >> >>> > > >> >> > > > >>> > > > > > > It definitely needs some
> > polishing
> > > > >> and a
> > > > >> >>> > double
> > > > >> >>> > > >> >> check
> > > > >> >>> > > >> >> > on
> > > > >> >>> > > >> >> > > my
> > > > >> >>> > > >> >> > > > >>> side
> > > > >> >>> > > >> >> > > > >>> > to
> > > > >> >>> > > >> >> > > > >>> > > > > make
> > > > >> >>> > > >> >> > > > >>> > > > > > > the review smooth.
> > > > >> >>> > > >> >> > > > >>> > > > > > > For the tests a local Pub/Sub
> > [2]
> > > > >> server
> > > > >> >>> has
> > > > >> >>> > to
> > > > >> >>> > > >> be
> > > > >> >>> > > >> >> > setup
> > > > >> >>> > > >> >> > > > >>> however
> > > > >> >>> > > >> >> > > > >>> > I
> > > > >> >>> > > >> >> > > > >>> > > am
> > > > >> >>> > > >> >> > > > >>> > > > > not
> > > > >> >>> > > >> >> > > > >>> > > > > > > sure which is the best way to
> do
> > > so
> > > > >> with
> > > > >> >>> > > regards
> > > > >> >>> > > >> to
> > > > >> >>> > > >> >> > your
> > > > >> >>> > > >> >> > > > >>> CI/CD.
> > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > [1]
> > > > >> >>> > > >> >> > > > >>> > >
> > > > >> >>> > > >> >> >
> > > > >> >>>
> https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > > > >> >>> > > >> >> > > > >>> > > > > > > [2]
> > > > >> >>> > > >> >> > > > >>>
> > > > >> >>> > https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
> > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > *Emmanouil Gkatziouras*
> > > > >> >>> > > >> >> > > > >>> > > > > > > https://egkatzioura.com/ |
> > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > > >> >>> > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > >> >>> > > >> >> > > > >>> > > > > > > https://github.com/gkatzioura
> > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > On Sat, 12 Oct 2019 at 02:11,
> > > Saikat
> > > > >> >>> Maitra <
> > > > >> >>> > > >> >> > > > >>> > > saikat.maitra@gmail.com
> > > > >> >>> > > >> >> > > > >>> > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > wrote:
> > > > >> >>> > > >> >> > > > >>> > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > > Hello Emmanouil,
> > > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > > Can you please share more
> info
> > > > about
> > > > >> >>> the
> > > > >> >>> > > >> >> > > implementation?
> > > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > > We have similar
> implementation
> > > for
> > > > >> data
> > > > >> >>> > > >> streamers
> > > > >> >>> > > >> >> > with
> > > > >> >>> > > >> >> > > > >>> Kafka
> > > > >> >>> > > >> >> > > > >>> > and
> > > > >> >>> > > >> >> > > > >>> > > > > > RocketMQ
> > > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > > >> >>> > > >> >> >
> > > https://apacheignite-mix.readme.io/docs/kafka-streamer
> > > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > > >> >>> > > >> >> > > >
> > > > >> >>> https://apacheignite-mix.readme.io/docs/rocketmq-streamer
> > > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > > Please review and share if
> you
> > > > have
> > > > >> any
> > > > >> >>> > > >> questions.
> > > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > > Regards,
> > > > >> >>> > > >> >> > > > >>> > > > > > > > Saikat
> > > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > > On Fri, Oct 11, 2019 at 4:56
> > PM
> > > > >> >>> Emmanouil
> > > > >> >>> > > >> >> > Gkatziouras <
> > > > >> >>> > > >> >> > > > >>> > > > > > > > gkatzioura@gmail.com>
> > > > >> >>> > > >> >> > > > >>> > > > > > > > wrote:
> > > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > > > Hi all,
> > > > >> >>> > > >> >> > > > >>> > > > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > > > By mentioning Pub/Sub I
> mean
> > > > >> Google
> > > > >> >>> > Cloud's
> > > > >> >>> > > >> >> Pub/Sub
> > > > >> >>> > > >> >> > > [1]
> > > > >> >>> > > >> >> > > > >>> > > > > > > > > Yes it is indeed cloud
> > > specific.
> > > > >> It
> > > > >> >>> is
> > > > >> >>> > the
> > > > >> >>> > > >> >> > messaging
> > > > >> >>> > > >> >> > > > >>> service
> > > > >> >>> > > >> >> > > > >>> > > > > provided
> > > > >> >>> > > >> >> > > > >>> > > > > > > by
> > > > >> >>> > > >> >> > > > >>> > > > > > > > > GCP.
> > > > >> >>> > > >> >> > > > >>> > > > > > > > > I have made the
> integration
> > > with
> > > > >> it,
> > > > >> >>> but
> > > > >> >>> > I
> > > > >> >>> > > >> >> haven't
> > > > >> >>> > > >> >> > > > >>> published
> > > > >> >>> > > >> >> > > > >>> > it
> > > > >> >>> > > >> >> > > > >>> > > > yet
> > > > >> >>> > > >> >> > > > >>> > > > > > (I
> > > > >> >>> > > >> >> > > > >>> > > > > > > am
> > > > >> >>> > > >> >> > > > >>> > > > > > > > > waiting for the
> > > > discussion/ticket
> > > > >> and
> > > > >> >>> > > proceed
> > > > >> >>> > > >> >> with
> > > > >> >>> > > >> >> > > the
> > > > >> >>> > > >> >> > > > >>> > > > > contribution).
> > > > >> >>> > > >> >> > > > >>> > > > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > > > Kind regards,
> > > > >> >>> > > >> >> > > > >>> > > > > > > > > Emmanouil
> > > > >> >>> > > >> >> > > > >>> > > > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > > > [1]
> > > > >> >>> > > >> >> https://cloud.google.com/pubsub/docs/overview
> > > > >> >>> > > >> >> > > > >>> > > > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > > > *Emmanouil Gkatziouras*
> > > > >> >>> > > >> >> > > > >>> > > > > > > > > https://egkatzioura.com/
> |
> > > > >> >>> > > >> >> > > > >>> > > > > > > > >
> > > > >> >>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > >> >>> > > >> >> > > > >>> > > > > > > > >
> > https://github.com/gkatzioura
> > > > >> >>> > > >> >> > > > >>> > > > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > > >
> > > > >> >>> > > >> >> > > > >>> > > > > > > > > On Fri, 11 Oct 2019 at
> > 21:45,
> > > > >> Denis
> > > > >> >>> > Magda <
> > > > >> >>> > > >> >> > > > >>> dmagda@apache.org
> > > > >> >>> > > >> >> > > > >>> > >
> > > > >> >>> > > >>
>

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra <sa...@gmail.com>]

Hi,

We will need to add test suites for tests to be executed in build. We pass
them as TestSuites param. I had added Flink sink and source testsuites.

Can you please review and confirm.

Regards
Saikat

On Wed, 27 Nov 2019 at 6:00 AM, Emmanouil Gkatziouras <gk...@gmail.com>
wrote:

> Hi all!
>
> I was successful building and running my pull requests. It seems that the
> tests do not run both for Flink and Pub/Sub [1]
> If there is something I cannot do to make them discoverable please let me
> know.
>
> [1]
>
> https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4796722&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=283
>
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>
>
> On Wed, 27 Nov 2019 at 04:57, Saikat Maitra <sa...@gmail.com>
> wrote:
>
> > Hi Emmanouil,
> >
> > I have added you as contributor in Ignite Extensions project. Can you
> > please check and confirm if you are able to see the project and execute
> > build on your pull request.
> >
> > Regards,
> > Saikat
> >
> > On Tue, Nov 26, 2019 at 12:49 PM Emmanouil Gkatziouras <
> > gkatzioura@gmail.com>
> > wrote:
> >
> > > Hi all!
> > >
> > > If someone could give me read access to the Job of Ignite Extensions on
> > > Team City will greatly help me (username gkatzioura).
> > > I suppose the builds get triggered automatically on a pull request.
> > >
> > > Kind regards,
> > > Emmanouil
> > >
> > > *Emmanouil Gkatziouras*
> > > https://egkatzioura.com/ |
> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > https://github.com/gkatzioura
> > >
> > >
> > > On Sun, 24 Nov 2019 at 21:11, Emmanouil Gkatziouras <
> > gkatzioura@gmail.com>
> > > wrote:
> > >
> > > > Hi Saikat!
> > > >
> > > > I just rebased with the flink branch. Unfortunately I do not have
> read
> > > > access to the team city link you provided and thus evaluate the tests
> > for
> > > > my pull request.
> > > > I guess it has to do with ignite extensions being new.
> > > > My username is gkatzioura and email the one I am using on this email
> > > >
> > > > Thank you for your time.
> > > >
> > > >
> > > > *Emmanouil Gkatziouras*
> > > > https://egkatzioura.com/ |
> > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > https://github.com/gkatzioura
> > > >
> > > >
> > > > On Sun, 24 Nov 2019 at 18:03, Saikat Maitra <saikat.maitra@gmail.com
> >
> > > > wrote:
> > > >
> > > >> Hi Emmanouil,
> > > >>
> > > >> The latest build on teamcity has passed on Flink pull request.
> > > >>
> > > >>
> > > >>
> > >
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4788928&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> > > >>
> > > >> you should be able to take the changes I made in my PR and run build
> > on
> > > >> your pull request.
> > > >>
> > > >> Please let me know if you have any questions.
> > > >>
> > > >> Regards,
> > > >> Saikat
> > > >>
> > > >> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra <
> > saikat.maitra@gmail.com
> > > >
> > > >> wrote:
> > > >>
> > > >> > Hi Emmanouil,
> > > >> >
> > > >> > I have fixed the Flink module testsuites and have setup new
> project
> > in
> > > >> > teamcity.
> > > >> >
> > > >> >
> > > >> >
> > > >>
> > >
> >
> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview
> > > >> >
> > > >> > I am looking into setting up the build.
> > > >> >
> > > >> > Regards,
> > > >> > Saikat
> > > >> >
> > > >> > On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra <
> > > saikat.maitra@gmail.com>
> > > >> > wrote:
> > > >> >
> > > >> >> Hi Emmanouil,
> > > >> >>
> > > >> >> Thank you for your email. Yes, the plan is once the Flink PR is
> > > merged
> > > >> it
> > > >> >> will provide the Licence, parent POMs etc and you can rebase from
> > > >> master
> > > >> >> branch and apply the changes on top of it.
> > > >> >>
> > > >> >> I am getting some issues with test failures in local with
> > > >> >> GridTestProperties as the test.properties is not present in this
> > new
> > > >> >> project but should be available from dependencies.
> > > >> >>
> > > >> >> Once I address this issue, I will go ahead and merge the changes
> > and
> > > >> then
> > > >> >> we can take it from there.
> > > >> >>
> > > >> >> Regards,
> > > >> >> Saikat
> > > >> >>
> > > >> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras <
> > > >> >> gkatzioura@gmail.com> wrote:
> > > >> >>
> > > >> >>> Hi all,
> > > >> >>>
> > > >> >>> I made my first pull request [1]. Since this project is brand
> new
> > > (no
> > > >> >>> parent poms, licensing), I reckoned it was best to use Saikat's
> > > >> branch on
> > > >> >>> Flink.
> > > >> >>> I suppose the Flink branch will be merged first. If not please
> > give
> > > me
> > > >> >>> guidelines on how I should proceed next.
> > > >> >>>
> > > >> >>> Kind regards
> > > >> >>> Emmanouil
> > > >> >>>
> > > >> >>> [1] https://github.com/apache/ignite-extensions/pull/2
> > > >> >>>
> > > >> >>> *Emmanouil Gkatziouras*
> > > >> >>> https://egkatzioura.com/ |
> > > >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> >>> https://github.com/gkatzioura
> > > >> >>>
> > > >> >>>
> > > >> >>> On Fri, 22 Nov 2019 at 20:55, Denis Magda <dm...@apache.org>
> > > wrote:
> > > >> >>>
> > > >> >>> > Awesome, ping us whenever you're ready!
> > > >> >>> >
> > > >> >>> > -
> > > >> >>> > Denis
> > > >> >>> >
> > > >> >>> >
> > > >> >>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <
> > > >> >>> > gkatzioura@gmail.com>
> > > >> >>> > wrote:
> > > >> >>> >
> > > >> >>> > > Hi all!
> > > >> >>> > >
> > > >> >>> > > I am sorry for being late on that. I was trying to refactor
> > the
> > > >> test
> > > >> >>> in
> > > >> >>> > > order not to be in need of any external tools or spinning
> up a
> > > >> >>> server.
> > > >> >>> > > I did forked the new repo and indeed my changes there, so a
> > pull
> > > >> >>> request
> > > >> >>> > is
> > > >> >>> > > a matter of time!
> > > >> >>> > >
> > > >> >>> > > Kind regards
> > > >> >>> > >
> > > >> >>> > > *Emmanouil Gkatziouras*
> > > >> >>> > > https://egkatzioura.com/ |
> > > >> >>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> >>> > > https://github.com/gkatzioura
> > > >> >>> > >
> > > >> >>> > >
> > > >> >>> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda <
> dmagda@apache.org>
> > > >> wrote:
> > > >> >>> > >
> > > >> >>> > > > Hi Emmanouil,
> > > >> >>> > > >
> > > >> >>> > > > Do you have any questions or need any support from the
> > > >> community?
> > > >> >>> > > >
> > > >> >>> > > > -
> > > >> >>> > > > Denis
> > > >> >>> > > >
> > > >> >>> > > >
> > > >> >>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <
> > > >> >>> saikat.maitra@gmail.com
> > > >> >>> > >
> > > >> >>> > > > wrote:
> > > >> >>> > > >
> > > >> >>> > > >> Hi Emmanouil,
> > > >> >>> > > >>
> > > >> >>> > > >> Can you please take a looks at dev utils, if this is
> > > something
> > > >> >>> you are
> > > >> >>> > > >> looking for
> > > >> >>> > > >>
> > > https://github.com/apache/ignite/tree/master/modules/dev-utils
> > > >> ?
> > > >> >>> > > >>
> > > >> >>> > > >> IMO, if you can release Pub/Sub server in maven and then
> > use
> > > >> it as
> > > >> >>> > > >> dependency, that would be great.
> > > >> >>> > > >>
> > > >> >>> > > >> Regards,
> > > >> >>> > > >> Saikat
> > > >> >>> > > >>
> > > >> >>> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <
> > > >> >>> > saikat.maitra@gmail.com>
> > > >> >>> > > >> wrote:
> > > >> >>> > > >>
> > > >> >>> > > >> > Hi Emmanouil,
> > > >> >>> > > >> >
> > > >> >>> > > >> > The master branch has been initialized. I have raised a
> > PR
> > > to
> > > >> >>> > migrate
> > > >> >>> > > >> > Flink module to ignite-extensions.
> > > >> >>> > > >> >
> > > >> >>> > > >> > PR https://github.com/apache/ignite-extensions/pull/1
> > > >> >>> > > >> > Jira :
> > https://issues.apache.org/jira/browse/IGNITE-12356
> > > >> >>> > > >> >
> > > >> >>> > > >> > Regards,
> > > >> >>> > > >> > Saikat
> > > >> >>> > > >> >
> > > >> >>> > > >> >
> > > >> >>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
> > > >> >>> > > >> > gkatzioura@gmail.com> wrote:
> > > >> >>> > > >> >
> > > >> >>> > > >> >> Hi all!
> > > >> >>> > > >> >>
> > > >> >>> > > >> >> This is an update. The integration is tested with
> mocks
> > > >> based
> > > >> >>> on
> > > >> >>> > the
> > > >> >>> > > >> >> original Ignite repo [1].
> > > >> >>> > > >> >> I am currently waiting for the first commits on the
> new
> > > >> >>> integration
> > > >> >>> > > >> >> project
> > > >> >>> > > >> >> to take place in order to create my pull request
> > > accordingly
> > > >> >>> to the
> > > >> >>> > > >> >> initial
> > > >> >>> > > >> >> commits.
> > > >> >>> > > >> >> In order to simulate the Pub/Sub server in a test I
> > used a
> > > >> test
> > > >> >>> > > server
> > > >> >>> > > >> >> implementation [2]. Unfortunately they don't have this
> > > util
> > > >> >>> > uploaded
> > > >> >>> > > >> in a
> > > >> >>> > > >> >> maven repo, thus it needs to be hosted on a maven repo
> > > >> (Apache
> > > >> >>> > > License
> > > >> >>> > > >> >> 2.0). Is there a repo for utilities that Ignite uses
> for
> > > the
> > > >> >>> > > >> test/build?
> > > >> >>> > > >> >> I have also created a ticket [3]
> > > >> >>> > > >> >>
> > > >> >>> > > >> >> Kind regards
> > > >> >>> > > >> >>
> > > >> >>> > > >> >> [1]
> > > >> >>> >
> https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > > >> >>> > > >> >> [2]
> > > >> >>> https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> > > >> >>> > > >> >> [3]
> https://issues.apache.org/jira/browse/IGNITE-12262
> > > >> >>> > > >> >>
> > > >> >>> > > >> >> *Emmanouil Gkatziouras*
> > > >> >>> > > >> >> https://egkatzioura.com/ |
> > > >> >>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> >>> > > >> >> https://github.com/gkatzioura
> > > >> >>> > > >> >>
> > > >> >>> > > >> >>
> > > >> >>> > > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <
> > > >> >>> > saikat.maitra@gmail.com
> > > >> >>> > > >
> > > >> >>> > > >> >> wrote:
> > > >> >>> > > >> >>
> > > >> >>> > > >> >> > Hello Ilya,
> > > >> >>> > > >> >> >
> > > >> >>> > > >> >> > Thank you for your email. Yes, before we remove any
> > > >> >>> integration
> > > >> >>> > we
> > > >> >>> > > >> will
> > > >> >>> > > >> >> > move them to another repository and release.
> > > >> >>> > > >> >> >
> > > >> >>> > > >> >> > We are thinking we will start with Pub/Sub Streamer
> > > >> >>> > implementation
> > > >> >>> > > >> and
> > > >> >>> > > >> >> > continue to move other integration as mentioned in
> the
> > > >> >>> following
> > > >> >>> > > doc.
> > > >> >>> > > >> >> >
> > > >> >>> > > >> >> >
> > > >> >>> > > >> >> >
> > > >> >>> > > >> >>
> > > >> >>> > > >>
> > > >> >>> > >
> > > >> >>> >
> > > >> >>>
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > >> >>> > > >> >> >
> > > >> >>> > > >> >> > Regards,
> > > >> >>> > > >> >> > Saikat
> > > >> >>> > > >> >> >
> > > >> >>> > > >> >> >
> > > >> >>> > > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
> > > >> >>> > > >> >> > ilya.kasnacheev@gmail.com>
> > > >> >>> > > >> >> > wrote:
> > > >> >>> > > >> >> >
> > > >> >>> > > >> >> > > Hello!
> > > >> >>> > > >> >> > >
> > > >> >>> > > >> >> > > My take on this: we want to offload some of
> > > >> integrations,
> > > >> >>> but
> > > >> >>> > > >> please
> > > >> >>> > > >> >> > commit
> > > >> >>> > > >> >> > > and release them to another project first, and
> only
> > > >> then we
> > > >> >>> > will
> > > >> >>> > > >> >> decide
> > > >> >>> > > >> >> > to
> > > >> >>> > > >> >> > > remove them from our repository.
> > > >> >>> > > >> >> > >
> > > >> >>> > > >> >> > > Please also make sure they are available with same
> > > >> artifact
> > > >> >>> > name
> > > >> >>> > > >> and
> > > >> >>> > > >> >> API.
> > > >> >>> > > >> >> > > If this has to be changed, let's wait for AI 3.0
> > > >> >>> > > >> >> > >
> > > >> >>> > > >> >> > > Regards,
> > > >> >>> > > >> >> > > --
> > > >> >>> > > >> >> > > Ilya Kasnacheev
> > > >> >>> > > >> >> > >
> > > >> >>> > > >> >> > >
> > > >> >>> > > >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <
> > > >> >>> > > >> saikat.maitra@gmail.com>:
> > > >> >>> > > >> >> > >
> > > >> >>> > > >> >> > > > Hello Emmanouil,
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > > > As discussed earlier, I discussed with Apache
> > Bahir
> > > >> >>> community
> > > >> >>> > > and
> > > >> >>> > > >> >> they
> > > >> >>> > > >> >> > > are
> > > >> >>> > > >> >> > > > interested to have Apache Ignite extensions as
> > part
> > > of
> > > >> >>> Apache
> > > >> >>> > > >> Bahir
> > > >> >>> > > >> >> > > > project.
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > > > Can you please share Pub/Sub streamer
> > implementation
> > > >> >>> details
> > > >> >>> > > with
> > > >> >>> > > >> >> > Apache
> > > >> >>> > > >> >> > > > Bahir community and request for feedback.
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > > > https://bahir.apache.org/
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > > > I have also requested for contributor access in
> > Jira
> > > >> for
> > > >> >>> > Apache
> > > >> >>> > > >> >> Bahir
> > > >> >>> > > >> >> > > > project so that I can create issues and assign
> to
> > > >> myself.
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > > > I can help with code reviews as well.
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > > > Here is the email thread for reference
> > > >> >>> > > >> >> > > >
> > > >> >>> > >
> > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > > > Regards,
> > > >> >>> > > >> >> > > > Saikat
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <
> > > >> >>> > > >> >> saikat.maitra@gmail.com
> > > >> >>> > > >> >> > >
> > > >> >>> > > >> >> > > > wrote:
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > > > > Hello,
> > > >> >>> > > >> >> > > > >
> > > >> >>> > > >> >> > > > > As discussed I have created following
> discussion
> > > >> >>> threads on
> > > >> >>> > > our
> > > >> >>> > > >> >> > > migration
> > > >> >>> > > >> >> > > > > proposals for Apache Ignite extensions:
> > > >> >>> > > >> >> > > > >
> > > >> >>> > > >> >> > > > >
> > > >> >>> > > >> >> > > > >
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > >
> > > >> >>> > > >> >> >
> > > >> >>> > > >> >>
> > > >> >>> > > >>
> > > >> >>> > >
> > > >> >>> >
> > > >> >>>
> > > >>
> > >
> >
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> > > >> >>> > > >> >> > > > >
> > > >> >>> > > >> >> > > > >
> > > >> >>> > > >>
> > > >> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > > >> >>> > > >> >> > > > >
> > > >> >>> > > >> >> > > > > I will share update as I hear more
> information.
> > > >> >>> > > >> >> > > > >
> > > >> >>> > > >> >> > > > > Regards,
> > > >> >>> > > >> >> > > > > Saikat
> > > >> >>> > > >> >> > > > >
> > > >> >>> > > >> >> > > > >
> > > >> >>> > > >> >> > > > >
> > > >> >>> > > >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra
> <
> > > >> >>> > > >> >> > saikat.maitra@gmail.com
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > > > > wrote:
> > > >> >>> > > >> >> > > > >
> > > >> >>> > > >> >> > > > >> Hello Denis,
> > > >> >>> > > >> >> > > > >>
> > > >> >>> > > >> >> > > > >> Sure, sounds good. I will create a separate
> > > >> discussion
> > > >> >>> > > thread
> > > >> >>> > > >> to
> > > >> >>> > > >> >> get
> > > >> >>> > > >> >> > > > >> community feedback on whether we
> > > >> >>> > > >> >> > > > >> should join the Bahir or kick-off "Ignite
> > > >> Extensions"
> > > >> >>> > > project.
> > > >> >>> > > >> >> > > > >>
> > > >> >>> > > >> >> > > > >> Regards,
> > > >> >>> > > >> >> > > > >> Saikat
> > > >> >>> > > >> >> > > > >>
> > > >> >>> > > >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda <
> > > >> >>> > > >> dmagda@apache.org>
> > > >> >>> > > >> >> > > wrote:
> > > >> >>> > > >> >> > > > >>
> > > >> >>> > > >> >> > > > >>> Folks,
> > > >> >>> > > >> >> > > > >>>
> > > >> >>> > > >> >> > > > >>> The concept of Apache Bahir is precisely
> what
> > we
> > > >> >>> need!
> > > >> >>> > > >> Saikat,
> > > >> >>> > > >> >> > thanks
> > > >> >>> > > >> >> > > > for
> > > >> >>> > > >> >> > > > >>> doing the research. Why I believe the *
> idea*
> > > >> fits us
> > > >> >>> > well:
> > > >> >>> > > >> >> > > > >>>
> > > >> >>> > > >> >> > > > >>>    - All integrations can be stored in
> > separate
> > > >> >>> Github
> > > >> >>> > > >> >> repositories
> > > >> >>> > > >> >> > > and
> > > >> >>> > > >> >> > > > >>>    have their dev lifecycles. We've not
> > obliged
> > > to
> > > >> >>> couple
> > > >> >>> > > all
> > > >> >>> > > >> >> the
> > > >> >>> > > >> >> > > > >>> integrations
> > > >> >>> > > >> >> > > > >>>    in a single repo. For instance, Spark can
> > be
> > > >> >>> located
> > > >> >>> > in
> > > >> >>> > > a
> > > >> >>> > > >> >> > > dedicated
> > > >> >>> > > >> >> > > > >>> repo
> > > >> >>> > > >> >> > > > >>>    while streaming integrations might be in
> a
> > > >> single
> > > >> >>> one.
> > > >> >>> > > >> It's
> > > >> >>> > > >> >> up
> > > >> >>> > > >> >> > to
> > > >> >>> > > >> >> > > > us.
> > > >> >>> > > >> >> > > > >>>    - All the repositories and integrations
> > > belong
> > > >> to
> > > >> >>> ASF.
> > > >> >>> > > >> We're
> > > >> >>> > > >> >> not
> > > >> >>> > > >> >> > > > >>> dumping
> > > >> >>> > > >> >> > > > >>>    them on Github but rather keep supporting
> > and
> > > >> >>> > developing
> > > >> >>> > > >> in
> > > >> >>> > > >> >> > > > >>> accordance with
> > > >> >>> > > >> >> > > > >>>    ASF vision and practices.
> > > >> >>> > > >> >> > > > >>>    - There is a way to reward contributors
> via
> > > >> >>> > > committership
> > > >> >>> > > >> and
> > > >> >>> > > >> >> > PMC
> > > >> >>> > > >> >> > > > >>>    membership. You won't get this if a
> project
> > > is
> > > >> >>> just
> > > >> >>> > one
> > > >> >>> > > of
> > > >> >>> > > >> >> the
> > > >> >>> > > >> >> > > > >>> millions of
> > > >> >>> > > >> >> > > > >>>    Github projects.
> > > >> >>> > > >> >> > > > >>>
> > > >> >>> > > >> >> > > > >>> Saikat, as Ignite PMC member, could you
> please
> > > >> >>> kick-off a
> > > >> >>> > > >> >> separate
> > > >> >>> > > >> >> > > > >>> dev-list
> > > >> >>> > > >> >> > > > >>> discussion to involve more community members
> > and
> > > >> >>> > referring
> > > >> >>> > > to
> > > >> >>> > > >> >> this
> > > >> >>> > > >> >> > > > >>> thread?
> > > >> >>> > > >> >> > > > >>> I think the primary question the community
> > needs
> > > >> to
> > > >> >>> > answer
> > > >> >>> > > >> >> whether
> > > >> >>> > > >> >> > we
> > > >> >>> > > >> >> > > > >>> should join the Bahir or kick-off "Ignite
> > > >> Extensions"
> > > >> >>> > > >> project?
> > > >> >>> > > >> >> > > > >>>
> > > >> >>> > > >> >> > > > >>> -
> > > >> >>> > > >> >> > > > >>> Denis
> > > >> >>> > > >> >> > > > >>>
> > > >> >>> > > >> >> > > > >>>
> > > >> >>> > > >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey
> > Zinoviev
> > > <
> > > >> >>> > > >> >> > > > zaleslaw.sin@gmail.com>
> > > >> >>> > > >> >> > > > >>> wrote:
> > > >> >>> > > >> >> > > > >>>
> > > >> >>> > > >> >> > > > >>> > Maybe we could move all our Streaming
> > > >> Integrations
> > > >> >>> > there,
> > > >> >>> > > >> but
> > > >> >>> > > >> >> > what
> > > >> >>> > > >> >> > > is
> > > >> >>> > > >> >> > > > >>> about
> > > >> >>> > > >> >> > > > >>> > maintaining and committer permissions to
> the
> > > new
> > > >> >>> > > >> repositories?
> > > >> >>> > > >> >> > > > >>> > I see the list of the committers and PMC
> > > members
> > > >> >>> there
> > > >> >>> > > >> >> > > > >>> >
> https://bahir.apache.org/community-members/
> > > >> >>> > > >> >> > > > >>> >
> > > >> >>> > > >> >> > > > >>> > Could somebody from Ignite community be
> > added
> > > to
> > > >> >>> this
> > > >> >>> > > list
> > > >> >>> > > >> as
> > > >> >>> > > >> >> a
> > > >> >>> > > >> >> > > > >>> > PMC/committer to maintain Ignite
> > integrations?
> > > >> >>> > > >> >> > > > >>> > If yes, I'd happy to join this project and
> > > >> >>> collaborate
> > > >> >>> > > with
> > > >> >>> > > >> >> these
> > > >> >>> > > >> >> > > > guys
> > > >> >>> > > >> >> > > > >>> > together
> > > >> >>> > > >> >> > > > >>> > If no, and we should start from the zero
> > level
> > > >> with
> > > >> >>> > > >> external
> > > >> >>> > > >> >> PRs
> > > >> >>> > > >> >> > -
> > > >> >>> > > >> >> > > > >>> hmmm,
> > > >> >>> > > >> >> > > > >>> > it's better to have our own external
> > > repository
> > > >> >>> with
> > > >> >>> > > >> >> ApacheBahirr
> > > >> >>> > > >> >> > > > >>> ideology.
> > > >> >>> > > >> >> > > > >>> >
> > > >> >>> > > >> >> > > > >>> > Also, I agree, that all connectors could
> be
> > > >> moved
> > > >> >>> there
> > > >> >>> > > (in
> > > >> >>> > > >> >> > > > >>> ApacheBahirr
> > > >> >>> > > >> >> > > > >>> > like repository), but not all modules from
> > the
> > > >> page
> > > >> >>> > > >> >> > > > >>> >
> > > >> >>> > > >> >> > > > >>> >
> > > >> >>> > > >> >> > > > >>>
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > >
> > > >> >>> > > >> >> >
> > > >> >>> > > >> >>
> > > >> >>> > > >>
> > > >> >>> > >
> > > >> >>> >
> > > >> >>>
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > >> >>> > > >> >> > > > >>> > because
> > > >> >>> > > >> >> > > > >>> > they use different parts of core modules
> and
> > > >> could
> > > >> >>> be
> > > >> >>> > > >> >> developed
> > > >> >>> > > >> >> > > with
> > > >> >>> > > >> >> > > > >>> > different velocity.
> > > >> >>> > > >> >> > > > >>> >
> > > >> >>> > > >> >> > > > >>> > In reality, before creation of new
> > > repositories
> > > >> we
> > > >> >>> need
> > > >> >>> > > >> wide
> > > >> >>> > > >> >> > > > discussion
> > > >> >>> > > >> >> > > > >>> > based on the document mentioned above.
> > > >> >>> > > >> >> > > > >>> >
> > > >> >>> > > >> >> > > > >>> > P.S. Of course, we could experiment in the
> > > next
> > > >> >>> release
> > > >> >>> > > 2.8
> > > >> >>> > > >> >> with
> > > >> >>> > > >> >> > > one
> > > >> >>> > > >> >> > > > >>> or two
> > > >> >>> > > >> >> > > > >>> > integrations like twitter or ZeroMQ
> > > >> >>> > > >> >> > > > >>> > Also, I'd like Denis Magda idea for the
> > > separate
> > > >> >>> > > >> repositories
> > > >> >>> > > >> >> in
> > > >> >>> > > >> >> > > > Apache
> > > >> >>> > > >> >> > > > >>> > Github to maintain them by anybody.
> > > >> >>> > > >> >> > > > >>> >
> > > >> >>> > > >> >> > > > >>> >
> > > >> >>> > > >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat
> Maitra <
> > > >> >>> > > >> >> > > saikat.maitra@gmail.com
> > > >> >>> > > >> >> > > > >:
> > > >> >>> > > >> >> > > > >>> >
> > > >> >>> > > >> >> > > > >>> > > Hi Denis, Emmanouil
> > > >> >>> > > >> >> > > > >>> > >
> > > >> >>> > > >> >> > > > >>> > > Thank you for your email. I think
> > creating a
> > > >> >>> separate
> > > >> >>> > > >> >> > integration
> > > >> >>> > > >> >> > > > >>> project
> > > >> >>> > > >> >> > > > >>> > > in github and also proposing it as an
> > apache
> > > >> >>> > incubator
> > > >> >>> > > >> >> project
> > > >> >>> > > >> >> > > will
> > > >> >>> > > >> >> > > > >>> be a
> > > >> >>> > > >> >> > > > >>> > > good move. The other separate
> integration
> > > >> modules
> > > >> >>> > > >> >> > > > >>> > >
> > > >> >>> > > >> >> > > > >>> > >
> > > >> >>> > > >> >> > > > >>> >
> > > >> >>> > > >> >> > > > >>>
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > >
> > > >> >>> > > >> >> >
> > > >> >>> > > >> >>
> > > >> >>> > > >>
> > > >> >>> > >
> > > >> >>> >
> > > >> >>>
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > >> >>> > > >> >> > > > >>> > > can
> > > >> >>> > > >> >> > > > >>> > > be moved to this new apache incubator
> > > project.
> > > >> >>> > > >> >> > > > >>> > >
> > > >> >>> > > >> >> > > > >>> > > There are similar integration available
> > for
> > > >> >>> Flink and
> > > >> >>> > > >> Spark
> > > >> >>> > > >> >> in
> > > >> >>> > > >> >> > > > Apache
> > > >> >>> > > >> >> > > > >>> > Bahir
> > > >> >>> > > >> >> > > > >>> > > https://bahir.apache.org/
> > > >> >>> > > >> >> > > > >>> > >
> > > >> >>> > > >> >> > > > >>> > > Do you think we should reach out to
> Apache
> > > >> Bahir
> > > >> >>> > > >> community
> > > >> >>> > > >> >> > with a
> > > >> >>> > > >> >> > > > >>> > proposal
> > > >> >>> > > >> >> > > > >>> > > or we should plan to create a separate
> > > Apache
> > > >> >>> > Incubator
> > > >> >>> > > >> >> > project?
> > > >> >>> > > >> >> > > > >>> > >
> > > >> >>> > > >> >> > > > >>> > > Regards,
> > > >> >>> > > >> >> > > > >>> > > Saikat
> > > >> >>> > > >> >> > > > >>> > >
> > > >> >>> > > >> >> > > > >>> > >
> > > >> >>> > > >> >> > > > >>> > >
> > > >> >>> > > >> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM
> Emmanouil
> > > >> >>> > Gkatziouras <
> > > >> >>> > > >> >> > > > >>> > > gkatzioura@gmail.com>
> > > >> >>> > > >> >> > > > >>> > > wrote:
> > > >> >>> > > >> >> > > > >>> > >
> > > >> >>> > > >> >> > > > >>> > > > Hi all!
> > > >> >>> > > >> >> > > > >>> > > >
> > > >> >>> > > >> >> > > > >>> > > > Based on the discussions most probably
> > > this
> > > >> is
> > > >> >>> > going
> > > >> >>> > > to
> > > >> >>> > > >> >> be on
> > > >> >>> > > >> >> > > > >>> another
> > > >> >>> > > >> >> > > > >>> > > > GitHub repo. Therefore I will prepare
> a
> > > >> >>> standalone
> > > >> >>> > > >> project
> > > >> >>> > > >> >> > with
> > > >> >>> > > >> >> > > > the
> > > >> >>> > > >> >> > > > >>> > > Pub/Sub
> > > >> >>> > > >> >> > > > >>> > > > module and once the repository is
> > created
> > > a
> > > >> >>> pull
> > > >> >>> > > >> request
> > > >> >>> > > >> >> > shall
> > > >> >>> > > >> >> > > be
> > > >> >>> > > >> >> > > > >>> > issued.
> > > >> >>> > > >> >> > > > >>> > > > If there is anything else I can do in
> > > order
> > > >> to
> > > >> >>> > > >> facilitate
> > > >> >>> > > >> >> > this
> > > >> >>> > > >> >> > > > >>> please
> > > >> >>> > > >> >> > > > >>> > let
> > > >> >>> > > >> >> > > > >>> > > > me know.
> > > >> >>> > > >> >> > > > >>> > > >
> > > >> >>> > > >> >> > > > >>> > > > If a ticket is created for this
> issue, I
> > > >> shall
> > > >> >>> be
> > > >> >>> > > able
> > > >> >>> > > >> to
> > > >> >>> > > >> >> > > create
> > > >> >>> > > >> >> > > > a
> > > >> >>> > > >> >> > > > >>> pull
> > > >> >>> > > >> >> > > > >>> > > > request and thus a build will take
> place
> > > on
> > > >> >>> > ignite's
> > > >> >>> > > >> CI.
> > > >> >>> > > >> >> > > > >>> > > > If everyone is aligned with adding a
> > > Pub/Sub
> > > >> >>> > > >> >> implementation
> > > >> >>> > > >> >> > > may I
> > > >> >>> > > >> >> > > > >>> > create
> > > >> >>> > > >> >> > > > >>> > > a
> > > >> >>> > > >> >> > > > >>> > > > ticket on JIRA?
> > > >> >>> > > >> >> > > > >>> > > >
> > > >> >>> > > >> >> > > > >>> > > > My next question has to do on how
> Ignite
> > > >> >>> configures
> > > >> >>> > > the
> > > >> >>> > > >> >> build
> > > >> >>> > > >> >> > > > jobs
> > > >> >>> > > >> >> > > > >>> in
> > > >> >>> > > >> >> > > > >>> > > Team
> > > >> >>> > > >> >> > > > >>> > > > City. The unit tests for my
> > implementation
> > > >> >>> need a
> > > >> >>> > > local
> > > >> >>> > > >> >> > Pub/Sub
> > > >> >>> > > >> >> > > > >>> server
> > > >> >>> > > >> >> > > > >>> > up
> > > >> >>> > > >> >> > > > >>> > > > and running (they provide one for
> > testing
> > > >> >>> > purposes).
> > > >> >>> > > >> Will
> > > >> >>> > > >> >> > this
> > > >> >>> > > >> >> > > > take
> > > >> >>> > > >> >> > > > >>> > > effect
> > > >> >>> > > >> >> > > > >>> > > > in a form of a build script inside my
> > > >> >>> > implementation
> > > >> >>> > > or
> > > >> >>> > > >> >> is it
> > > >> >>> > > >> >> > > > >>> something
> > > >> >>> > > >> >> > > > >>> > > > that should be configured on Team
> City?
> > > >> >>> > > >> >> > > > >>> > > >
> > > >> >>> > > >> >> > > > >>> > > > Kind regards,
> > > >> >>> > > >> >> > > > >>> > > > Emmanouil
> > > >> >>> > > >> >> > > > >>> > > >
> > > >> >>> > > >> >> > > > >>> > > >
> > > >> >>> > > >> >> > > > >>> > > > *Emmanouil Gkatziouras*
> > > >> >>> > > >> >> > > > >>> > > > https://egkatzioura.com/ |
> > > >> >>> > > >> >> > > > >>> > > >
> > > >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> >>> > > >> >> > > > >>> > > > https://github.com/gkatzioura
> > > >> >>> > > >> >> > > > >>> > > >
> > > >> >>> > > >> >> > > > >>> > > >
> > > >> >>> > > >> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis
> > Magda
> > > <
> > > >> >>> > > >> >> dmagda@apache.org
> > > >> >>> > > >> >> > >
> > > >> >>> > > >> >> > > > >>> wrote:
> > > >> >>> > > >> >> > > > >>> > > >
> > > >> >>> > > >> >> > > > >>> > > > > Emmanouil, Saikat,
> > > >> >>> > > >> >> > > > >>> > > > >
> > > >> >>> > > >> >> > > > >>> > > > > After contemplating on this, let me
> > > >> propose
> > > >> >>> > another
> > > >> >>> > > >> way
> > > >> >>> > > >> >> > with
> > > >> >>> > > >> >> > > > this
> > > >> >>> > > >> >> > > > >>> > > > > particular integration that is
> aligned
> > > >> with
> > > >> >>> our
> > > >> >>> > > >> >> > > modularization
> > > >> >>> > > >> >> > > > >>> > > endeavor.
> > > >> >>> > > >> >> > > > >>> > > > > The modularization [1] defines the
> > > Ignite
> > > >> >>> core,
> > > >> >>> > > >> Ignite
> > > >> >>> > > >> >> > > modules,
> > > >> >>> > > >> >> > > > >>> and
> > > >> >>> > > >> >> > > > >>> > > > > independent integrations. The first
> > two
> > > >> are
> > > >> >>> to be
> > > >> >>> > > >> >> located
> > > >> >>> > > >> >> > in
> > > >> >>> > > >> >> > > > >>> Ignite
> > > >> >>> > > >> >> > > > >>> > > > > repositories and to be
> tested/updated
> > by
> > > >> the
> > > >> >>> > > >> community
> > > >> >>> > > >> >> for
> > > >> >>> > > >> >> > > > every
> > > >> >>> > > >> >> > > > >>> > > release.
> > > >> >>> > > >> >> > > > >>> > > > > The modules are a subset of existing
> > > >> >>> integrations
> > > >> >>> > > >> that
> > > >> >>> > > >> >> are
> > > >> >>> > > >> >> > > > >>> adopted
> > > >> >>> > > >> >> > > > >>> > > widely
> > > >> >>> > > >> >> > > > >>> > > > > and require community attention.
> > > >> >>> > > >> >> > > > >>> > > > >
> > > >> >>> > > >> >> > > > >>> > > > > As for the independent integrations
> > [1],
> > > >> >>> those
> > > >> >>> > are
> > > >> >>> > > >> >> supposed
> > > >> >>> > > >> >> > > to
> > > >> >>> > > >> >> > > > be
> > > >> >>> > > >> >> > > > >>> > > located
> > > >> >>> > > >> >> > > > >>> > > > > in a separate GitHub repo. I would
> > > advise
> > > >> us
> > > >> >>> to
> > > >> >>> > > start
> > > >> >>> > > >> >> > > > >>> implementing
> > > >> >>> > > >> >> > > > >>> > this
> > > >> >>> > > >> >> > > > >>> > > > > concept with the Pub/Sub
> integration.
> > > Even
> > > >> >>> though
> > > >> >>> > > it
> > > >> >>> > > >> >> will
> > > >> >>> > > >> >> > be
> > > >> >>> > > >> >> > > > >>> located
> > > >> >>> > > >> >> > > > >>> > > in a
> > > >> >>> > > >> >> > > > >>> > > > > separate Github repo, it will be
> > listed
> > > >> among
> > > >> >>> > > >> officially
> > > >> >>> > > >> >> > > > >>> available
> > > >> >>> > > >> >> > > > >>> > > > > integrations on the Ignite website
> > > >> featuring
> > > >> >>> > > >> Emmanouil
> > > >> >>> > > >> >> as a
> > > >> >>> > > >> >> > > > >>> > > contributor.
> > > >> >>> > > >> >> > > > >>> > > > >
> > > >> >>> > > >> >> > > > >>> > > > > What do you think?
> > > >> >>> > > >> >> > > > >>> > > > >
> > > >> >>> > > >> >> > > > >>> > > > >
> > > >> >>> > > >> >> > > > >>> > > > > [1]
> > > >> >>> > > >> >> > > > >>> > > > >
> > > >> >>> > > >> >> > > > >>> > > > >
> > > >> >>> > > >> >> > > > >>> > > >
> > > >> >>> > > >> >> > > > >>> > >
> > > >> >>> > > >> >> > > > >>> >
> > > >> >>> > > >> >> > > > >>>
> > > >> >>> > > >> >> > > >
> > > >> >>> > > >> >> > >
> > > >> >>> > > >> >> >
> > > >> >>> > > >> >>
> > > >> >>> > > >>
> > > >> >>> > >
> > > >> >>> >
> > > >> >>>
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > >> >>> > > >> >> > > > >>> > > > >
> > > >> >>> > > >> >> > > > >>> > > > >
> > > >> >>> > > >> >> > > > >>> > > > >
> > > >> >>> > > >> >> > > > >>> > > > > -
> > > >> >>> > > >> >> > > > >>> > > > > Denis
> > > >> >>> > > >> >> > > > >>> > > > >
> > > >> >>> > > >> >> > > > >>> > > > >
> > > >> >>> > > >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM
> Saikat
> > > >> >>> Maitra <
> > > >> >>> > > >> >> > > > >>> > saikat.maitra@gmail.com
> > > >> >>> > > >> >> > > > >>> > > >
> > > >> >>> > > >> >> > > > >>> > > > > wrote:
> > > >> >>> > > >> >> > > > >>> > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > Hi Emmanouil,
> > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > The changes looks good to me. I
> > wanted
> > > >> to
> > > >> >>> check
> > > >> >>> > > if
> > > >> >>> > > >> you
> > > >> >>> > > >> >> > are
> > > >> >>> > > >> >> > > > also
> > > >> >>> > > >> >> > > > >>> > able
> > > >> >>> > > >> >> > > > >>> > > to
> > > >> >>> > > >> >> > > > >>> > > > > > validate that the once data is
> added
> > > to
> > > >> >>> Ignite
> > > >> >>> > > >> cluster
> > > >> >>> > > >> >> > you
> > > >> >>> > > >> >> > > > are
> > > >> >>> > > >> >> > > > >>> also
> > > >> >>> > > >> >> > > > >>> > > > able
> > > >> >>> > > >> >> > > > >>> > > > > to
> > > >> >>> > > >> >> > > > >>> > > > > > access it using another Ignite
> > client
> > > or
> > > >> >>> using
> > > >> >>> > > rest
> > > >> >>> > > >> >> > > > endpoints.
> > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > We use Teamcity for CI process,
> the
> > > >> >>> details are
> > > >> >>> > > as
> > > >> >>> > > >> >> > follows
> > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > Trigger validation of those test
> > > suites
> > > >> >>> that
> > > >> >>> > have
> > > >> >>> > > >> been
> > > >> >>> > > >> >> > > > >>> affected by
> > > >> >>> > > >> >> > > > >>> > > your
> > > >> >>> > > >> >> > > > >>> > > > > > changes on TeamCity <
> > > >> >>> > > http://ci.ignite.apache.org/>:
> > > >> >>> > > >> <
> > > >> >>> > > >> >> > > > >>> > > > > http://204.14.53.153/
> > > >> >>> > > >> >> > > > >>> > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > >    - Locate a test suite you have
> to
> > > >> check,
> > > >> >>> > press
> > > >> >>> > > >> >> button
> > > >> >>> > > >> >> > > > named
> > > >> >>> > > >> >> > > > >>> > "..."
> > > >> >>> > > >> >> > > > >>> > > > that
> > > >> >>> > > >> >> > > > >>> > > > > >    is located on the left of "Run"
> > > >> button.
> > > >> >>> "Run
> > > >> >>> > > >> custom
> > > >> >>> > > >> >> > > build"
> > > >> >>> > > >> >> > > > >>> > dialog
> > > >> >>> > > >> >> > > > >>> > > > will
> > > >> >>> > > >> >> > > > >>> > > > > >    appear;
> > > >> >>> > > >> >> > > > >>> > > > > >    - Go to "Changes" tab and
> choose
> > > >> >>> > > >> >> > > > >>> > "pull/<pull-request-number>/head"
> > > >> >>> > > >> >> > > > >>> > > > in
> > > >> >>> > > >> >> > > > >>> > > > > >    "Build branch" dropdown list;
> > > >> >>> > > >> >> > > > >>> > > > > >    - Press "Run build" button and
> > > >> monitor
> > > >> >>> tests
> > > >> >>> > > >> >> results.
> > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > @Roman rshtykh@yahoo.com - can
> you
> > > also
> > > >> >>> please
> > > >> >>> > > >> take a
> > > >> >>> > > >> >> > > look?
> > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > Regards,
> > > >> >>> > > >> >> > > > >>> > > > > > Saikat
> > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20 PM
> > > >> Emmanouil
> > > >> >>> > > >> Gkatziouras
> > > >> >>> > > >> >> <
> > > >> >>> > > >> >> > > > >>> > > > > > gkatzioura@gmail.com>
> > > >> >>> > > >> >> > > > >>> > > > > > wrote:
> > > >> >>> > > >> >> > > > >>> > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > Hi @Saikat Maitra
> > > >> >>> > > >> >> > > > >>> > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > I have my implementation on an
> > > Ignite
> > > >> >>> fork on
> > > >> >>> > > my
> > > >> >>> > > >> >> > personal
> > > >> >>> > > >> >> > > > >>> GitHub
> > > >> >>> > > >> >> > > > >>> > > > > account
> > > >> >>> > > >> >> > > > >>> > > > > > > [1]
> > > >> >>> > > >> >> > > > >>> > > > > > > It is based on the Kafka
> > > >> implementation
> > > >> >>> which
> > > >> >>> > > is
> > > >> >>> > > >> of
> > > >> >>> > > >> >> a
> > > >> >>> > > >> >> > > > polling
> > > >> >>> > > >> >> > > > >>> > > nature.
> > > >> >>> > > >> >> > > > >>> > > > > > > GCP does not provide a service
> > like
> > > >> >>> Kafka.
> > > >> >>> > > >> Instead
> > > >> >>> > > >> >> it
> > > >> >>> > > >> >> > > > >>> provides us
> > > >> >>> > > >> >> > > > >>> > > > with
> > > >> >>> > > >> >> > > > >>> > > > > > > Pub/Sub which is just like
> Kafka.
> > > >> >>> > > >> >> > > > >>> > > > > > > Therefore using the Kafka
> streamer
> > > as
> > > >> a
> > > >> >>> > > guideline
> > > >> >>> > > >> >> made
> > > >> >>> > > >> >> > > more
> > > >> >>> > > >> >> > > > >>> sense
> > > >> >>> > > >> >> > > > >>> > > to
> > > >> >>> > > >> >> > > > >>> > > > > me.
> > > >> >>> > > >> >> > > > >>> > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > Apart from the implementation I
> > have
> > > >> also
> > > >> >>> > > added a
> > > >> >>> > > >> >> unit
> > > >> >>> > > >> >> > > > test.
> > > >> >>> > > >> >> > > > >>> > > > > > > It definitely needs some
> polishing
> > > >> and a
> > > >> >>> > double
> > > >> >>> > > >> >> check
> > > >> >>> > > >> >> > on
> > > >> >>> > > >> >> > > my
> > > >> >>> > > >> >> > > > >>> side
> > > >> >>> > > >> >> > > > >>> > to
> > > >> >>> > > >> >> > > > >>> > > > > make
> > > >> >>> > > >> >> > > > >>> > > > > > > the review smooth.
> > > >> >>> > > >> >> > > > >>> > > > > > > For the tests a local Pub/Sub
> [2]
> > > >> server
> > > >> >>> has
> > > >> >>> > to
> > > >> >>> > > >> be
> > > >> >>> > > >> >> > setup
> > > >> >>> > > >> >> > > > >>> however
> > > >> >>> > > >> >> > > > >>> > I
> > > >> >>> > > >> >> > > > >>> > > am
> > > >> >>> > > >> >> > > > >>> > > > > not
> > > >> >>> > > >> >> > > > >>> > > > > > > sure which is the best way to do
> > so
> > > >> with
> > > >> >>> > > regards
> > > >> >>> > > >> to
> > > >> >>> > > >> >> > your
> > > >> >>> > > >> >> > > > >>> CI/CD.
> > > >> >>> > > >> >> > > > >>> > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > [1]
> > > >> >>> > > >> >> > > > >>> > >
> > > >> >>> > > >> >> >
> > > >> >>> https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > > >> >>> > > >> >> > > > >>> > > > > > > [2]
> > > >> >>> > > >> >> > > > >>>
> > > >> >>> > https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
> > > >> >>> > > >> >> > > > >>> > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > *Emmanouil Gkatziouras*
> > > >> >>> > > >> >> > > > >>> > > > > > > https://egkatzioura.com/ |
> > > >> >>> > > >> >> > > > >>> > > > > > >
> > > >> >>> > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> >>> > > >> >> > > > >>> > > > > > > https://github.com/gkatzioura
> > > >> >>> > > >> >> > > > >>> > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > On Sat, 12 Oct 2019 at 02:11,
> > Saikat
> > > >> >>> Maitra <
> > > >> >>> > > >> >> > > > >>> > > saikat.maitra@gmail.com
> > > >> >>> > > >> >> > > > >>> > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > wrote:
> > > >> >>> > > >> >> > > > >>> > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > > Hello Emmanouil,
> > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > > Can you please share more info
> > > about
> > > >> >>> the
> > > >> >>> > > >> >> > > implementation?
> > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > > We have similar implementation
> > for
> > > >> data
> > > >> >>> > > >> streamers
> > > >> >>> > > >> >> > with
> > > >> >>> > > >> >> > > > >>> Kafka
> > > >> >>> > > >> >> > > > >>> > and
> > > >> >>> > > >> >> > > > >>> > > > > > RocketMQ
> > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > >> >>> > > >> >> >
> > https://apacheignite-mix.readme.io/docs/kafka-streamer
> > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > >> >>> > > >> >> > > >
> > > >> >>> https://apacheignite-mix.readme.io/docs/rocketmq-streamer
> > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > > Please review and share if you
> > > have
> > > >> any
> > > >> >>> > > >> questions.
> > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > > Regards,
> > > >> >>> > > >> >> > > > >>> > > > > > > > Saikat
> > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > > On Fri, Oct 11, 2019 at 4:56
> PM
> > > >> >>> Emmanouil
> > > >> >>> > > >> >> > Gkatziouras <
> > > >> >>> > > >> >> > > > >>> > > > > > > > gkatzioura@gmail.com>
> > > >> >>> > > >> >> > > > >>> > > > > > > > wrote:
> > > >> >>> > > >> >> > > > >>> > > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > > > Hi all,
> > > >> >>> > > >> >> > > > >>> > > > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > > > By mentioning Pub/Sub I mean
> > > >> Google
> > > >> >>> > Cloud's
> > > >> >>> > > >> >> Pub/Sub
> > > >> >>> > > >> >> > > [1]
> > > >> >>> > > >> >> > > > >>> > > > > > > > > Yes it is indeed cloud
> > specific.
> > > >> It
> > > >> >>> is
> > > >> >>> > the
> > > >> >>> > > >> >> > messaging
> > > >> >>> > > >> >> > > > >>> service
> > > >> >>> > > >> >> > > > >>> > > > > provided
> > > >> >>> > > >> >> > > > >>> > > > > > > by
> > > >> >>> > > >> >> > > > >>> > > > > > > > > GCP.
> > > >> >>> > > >> >> > > > >>> > > > > > > > > I have made the integration
> > with
> > > >> it,
> > > >> >>> but
> > > >> >>> > I
> > > >> >>> > > >> >> haven't
> > > >> >>> > > >> >> > > > >>> published
> > > >> >>> > > >> >> > > > >>> > it
> > > >> >>> > > >> >> > > > >>> > > > yet
> > > >> >>> > > >> >> > > > >>> > > > > > (I
> > > >> >>> > > >> >> > > > >>> > > > > > > am
> > > >> >>> > > >> >> > > > >>> > > > > > > > > waiting for the
> > > discussion/ticket
> > > >> and
> > > >> >>> > > proceed
> > > >> >>> > > >> >> with
> > > >> >>> > > >> >> > > the
> > > >> >>> > > >> >> > > > >>> > > > > contribution).
> > > >> >>> > > >> >> > > > >>> > > > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > > > Kind regards,
> > > >> >>> > > >> >> > > > >>> > > > > > > > > Emmanouil
> > > >> >>> > > >> >> > > > >>> > > > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > > > [1]
> > > >> >>> > > >> >> https://cloud.google.com/pubsub/docs/overview
> > > >> >>> > > >> >> > > > >>> > > > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > > > *Emmanouil Gkatziouras*
> > > >> >>> > > >> >> > > > >>> > > > > > > > > https://egkatzioura.com/ |
> > > >> >>> > > >> >> > > > >>> > > > > > > > >
> > > >> >>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> >>> > > >> >> > > > >>> > > > > > > > >
> https://github.com/gkatzioura
> > > >> >>> > > >> >> > > > >>> > > > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > > >
> > > >> >>> > > >> >> > > > >>> > > > > > > > > On Fri, 11 Oct 2019 at
> 21:45,
> > > >> Denis
> > > >> >>> > Magda <
> > > >> >>> > > >> >> > > > >>> dmagda@apache.org
> > > >> >>> > > >> >> > > > >>> > >
> > > >> >>> > > >>

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Emmanouil Gkatziouras <gk...@gmail.com>]

Hi all!

I was successful building and running my pull requests. It seems that the
tests do not run both for Flink and Pub/Sub [1]
If there is something I cannot do to make them discoverable please let me
know.

[1]
https://ci.ignite.apache.org/viewLog.html?buildTypeId=IgniteExtensions_Build&buildId=4796722&tab=buildLog&logTab=tree&filter=debug&expand=all&_focus=283

*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura


On Wed, 27 Nov 2019 at 04:57, Saikat Maitra <sa...@gmail.com> wrote:

> Hi Emmanouil,
>
> I have added you as contributor in Ignite Extensions project. Can you
> please check and confirm if you are able to see the project and execute
> build on your pull request.
>
> Regards,
> Saikat
>
> On Tue, Nov 26, 2019 at 12:49 PM Emmanouil Gkatziouras <
> gkatzioura@gmail.com>
> wrote:
>
> > Hi all!
> >
> > If someone could give me read access to the Job of Ignite Extensions on
> > Team City will greatly help me (username gkatzioura).
> > I suppose the builds get triggered automatically on a pull request.
> >
> > Kind regards,
> > Emmanouil
> >
> > *Emmanouil Gkatziouras*
> > https://egkatzioura.com/ |
> > https://www.linkedin.com/in/gkatziourasemmanouil/
> > https://github.com/gkatzioura
> >
> >
> > On Sun, 24 Nov 2019 at 21:11, Emmanouil Gkatziouras <
> gkatzioura@gmail.com>
> > wrote:
> >
> > > Hi Saikat!
> > >
> > > I just rebased with the flink branch. Unfortunately I do not have read
> > > access to the team city link you provided and thus evaluate the tests
> for
> > > my pull request.
> > > I guess it has to do with ignite extensions being new.
> > > My username is gkatzioura and email the one I am using on this email
> > >
> > > Thank you for your time.
> > >
> > >
> > > *Emmanouil Gkatziouras*
> > > https://egkatzioura.com/ |
> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > https://github.com/gkatzioura
> > >
> > >
> > > On Sun, 24 Nov 2019 at 18:03, Saikat Maitra <sa...@gmail.com>
> > > wrote:
> > >
> > >> Hi Emmanouil,
> > >>
> > >> The latest build on teamcity has passed on Flink pull request.
> > >>
> > >>
> > >>
> >
> https://ci.ignite.apache.org/viewLog.html?buildId=4788928&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> > >>
> > >> you should be able to take the changes I made in my PR and run build
> on
> > >> your pull request.
> > >>
> > >> Please let me know if you have any questions.
> > >>
> > >> Regards,
> > >> Saikat
> > >>
> > >> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra <
> saikat.maitra@gmail.com
> > >
> > >> wrote:
> > >>
> > >> > Hi Emmanouil,
> > >> >
> > >> > I have fixed the Flink module testsuites and have setup new project
> in
> > >> > teamcity.
> > >> >
> > >> >
> > >> >
> > >>
> >
> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview
> > >> >
> > >> > I am looking into setting up the build.
> > >> >
> > >> > Regards,
> > >> > Saikat
> > >> >
> > >> > On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra <
> > saikat.maitra@gmail.com>
> > >> > wrote:
> > >> >
> > >> >> Hi Emmanouil,
> > >> >>
> > >> >> Thank you for your email. Yes, the plan is once the Flink PR is
> > merged
> > >> it
> > >> >> will provide the Licence, parent POMs etc and you can rebase from
> > >> master
> > >> >> branch and apply the changes on top of it.
> > >> >>
> > >> >> I am getting some issues with test failures in local with
> > >> >> GridTestProperties as the test.properties is not present in this
> new
> > >> >> project but should be available from dependencies.
> > >> >>
> > >> >> Once I address this issue, I will go ahead and merge the changes
> and
> > >> then
> > >> >> we can take it from there.
> > >> >>
> > >> >> Regards,
> > >> >> Saikat
> > >> >>
> > >> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras <
> > >> >> gkatzioura@gmail.com> wrote:
> > >> >>
> > >> >>> Hi all,
> > >> >>>
> > >> >>> I made my first pull request [1]. Since this project is brand new
> > (no
> > >> >>> parent poms, licensing), I reckoned it was best to use Saikat's
> > >> branch on
> > >> >>> Flink.
> > >> >>> I suppose the Flink branch will be merged first. If not please
> give
> > me
> > >> >>> guidelines on how I should proceed next.
> > >> >>>
> > >> >>> Kind regards
> > >> >>> Emmanouil
> > >> >>>
> > >> >>> [1] https://github.com/apache/ignite-extensions/pull/2
> > >> >>>
> > >> >>> *Emmanouil Gkatziouras*
> > >> >>> https://egkatzioura.com/ |
> > >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> >>> https://github.com/gkatzioura
> > >> >>>
> > >> >>>
> > >> >>> On Fri, 22 Nov 2019 at 20:55, Denis Magda <dm...@apache.org>
> > wrote:
> > >> >>>
> > >> >>> > Awesome, ping us whenever you're ready!
> > >> >>> >
> > >> >>> > -
> > >> >>> > Denis
> > >> >>> >
> > >> >>> >
> > >> >>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <
> > >> >>> > gkatzioura@gmail.com>
> > >> >>> > wrote:
> > >> >>> >
> > >> >>> > > Hi all!
> > >> >>> > >
> > >> >>> > > I am sorry for being late on that. I was trying to refactor
> the
> > >> test
> > >> >>> in
> > >> >>> > > order not to be in need of any external tools or spinning up a
> > >> >>> server.
> > >> >>> > > I did forked the new repo and indeed my changes there, so a
> pull
> > >> >>> request
> > >> >>> > is
> > >> >>> > > a matter of time!
> > >> >>> > >
> > >> >>> > > Kind regards
> > >> >>> > >
> > >> >>> > > *Emmanouil Gkatziouras*
> > >> >>> > > https://egkatzioura.com/ |
> > >> >>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> >>> > > https://github.com/gkatzioura
> > >> >>> > >
> > >> >>> > >
> > >> >>> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda <dm...@apache.org>
> > >> wrote:
> > >> >>> > >
> > >> >>> > > > Hi Emmanouil,
> > >> >>> > > >
> > >> >>> > > > Do you have any questions or need any support from the
> > >> community?
> > >> >>> > > >
> > >> >>> > > > -
> > >> >>> > > > Denis
> > >> >>> > > >
> > >> >>> > > >
> > >> >>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <
> > >> >>> saikat.maitra@gmail.com
> > >> >>> > >
> > >> >>> > > > wrote:
> > >> >>> > > >
> > >> >>> > > >> Hi Emmanouil,
> > >> >>> > > >>
> > >> >>> > > >> Can you please take a looks at dev utils, if this is
> > something
> > >> >>> you are
> > >> >>> > > >> looking for
> > >> >>> > > >>
> > https://github.com/apache/ignite/tree/master/modules/dev-utils
> > >> ?
> > >> >>> > > >>
> > >> >>> > > >> IMO, if you can release Pub/Sub server in maven and then
> use
> > >> it as
> > >> >>> > > >> dependency, that would be great.
> > >> >>> > > >>
> > >> >>> > > >> Regards,
> > >> >>> > > >> Saikat
> > >> >>> > > >>
> > >> >>> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <
> > >> >>> > saikat.maitra@gmail.com>
> > >> >>> > > >> wrote:
> > >> >>> > > >>
> > >> >>> > > >> > Hi Emmanouil,
> > >> >>> > > >> >
> > >> >>> > > >> > The master branch has been initialized. I have raised a
> PR
> > to
> > >> >>> > migrate
> > >> >>> > > >> > Flink module to ignite-extensions.
> > >> >>> > > >> >
> > >> >>> > > >> > PR https://github.com/apache/ignite-extensions/pull/1
> > >> >>> > > >> > Jira :
> https://issues.apache.org/jira/browse/IGNITE-12356
> > >> >>> > > >> >
> > >> >>> > > >> > Regards,
> > >> >>> > > >> > Saikat
> > >> >>> > > >> >
> > >> >>> > > >> >
> > >> >>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
> > >> >>> > > >> > gkatzioura@gmail.com> wrote:
> > >> >>> > > >> >
> > >> >>> > > >> >> Hi all!
> > >> >>> > > >> >>
> > >> >>> > > >> >> This is an update. The integration is tested with mocks
> > >> based
> > >> >>> on
> > >> >>> > the
> > >> >>> > > >> >> original Ignite repo [1].
> > >> >>> > > >> >> I am currently waiting for the first commits on the new
> > >> >>> integration
> > >> >>> > > >> >> project
> > >> >>> > > >> >> to take place in order to create my pull request
> > accordingly
> > >> >>> to the
> > >> >>> > > >> >> initial
> > >> >>> > > >> >> commits.
> > >> >>> > > >> >> In order to simulate the Pub/Sub server in a test I
> used a
> > >> test
> > >> >>> > > server
> > >> >>> > > >> >> implementation [2]. Unfortunately they don't have this
> > util
> > >> >>> > uploaded
> > >> >>> > > >> in a
> > >> >>> > > >> >> maven repo, thus it needs to be hosted on a maven repo
> > >> (Apache
> > >> >>> > > License
> > >> >>> > > >> >> 2.0). Is there a repo for utilities that Ignite uses for
> > the
> > >> >>> > > >> test/build?
> > >> >>> > > >> >> I have also created a ticket [3]
> > >> >>> > > >> >>
> > >> >>> > > >> >> Kind regards
> > >> >>> > > >> >>
> > >> >>> > > >> >> [1]
> > >> >>> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > >> >>> > > >> >> [2]
> > >> >>> https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> > >> >>> > > >> >> [3] https://issues.apache.org/jira/browse/IGNITE-12262
> > >> >>> > > >> >>
> > >> >>> > > >> >> *Emmanouil Gkatziouras*
> > >> >>> > > >> >> https://egkatzioura.com/ |
> > >> >>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> >>> > > >> >> https://github.com/gkatzioura
> > >> >>> > > >> >>
> > >> >>> > > >> >>
> > >> >>> > > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <
> > >> >>> > saikat.maitra@gmail.com
> > >> >>> > > >
> > >> >>> > > >> >> wrote:
> > >> >>> > > >> >>
> > >> >>> > > >> >> > Hello Ilya,
> > >> >>> > > >> >> >
> > >> >>> > > >> >> > Thank you for your email. Yes, before we remove any
> > >> >>> integration
> > >> >>> > we
> > >> >>> > > >> will
> > >> >>> > > >> >> > move them to another repository and release.
> > >> >>> > > >> >> >
> > >> >>> > > >> >> > We are thinking we will start with Pub/Sub Streamer
> > >> >>> > implementation
> > >> >>> > > >> and
> > >> >>> > > >> >> > continue to move other integration as mentioned in the
> > >> >>> following
> > >> >>> > > doc.
> > >> >>> > > >> >> >
> > >> >>> > > >> >> >
> > >> >>> > > >> >> >
> > >> >>> > > >> >>
> > >> >>> > > >>
> > >> >>> > >
> > >> >>> >
> > >> >>>
> > >>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > >> >>> > > >> >> >
> > >> >>> > > >> >> > Regards,
> > >> >>> > > >> >> > Saikat
> > >> >>> > > >> >> >
> > >> >>> > > >> >> >
> > >> >>> > > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
> > >> >>> > > >> >> > ilya.kasnacheev@gmail.com>
> > >> >>> > > >> >> > wrote:
> > >> >>> > > >> >> >
> > >> >>> > > >> >> > > Hello!
> > >> >>> > > >> >> > >
> > >> >>> > > >> >> > > My take on this: we want to offload some of
> > >> integrations,
> > >> >>> but
> > >> >>> > > >> please
> > >> >>> > > >> >> > commit
> > >> >>> > > >> >> > > and release them to another project first, and only
> > >> then we
> > >> >>> > will
> > >> >>> > > >> >> decide
> > >> >>> > > >> >> > to
> > >> >>> > > >> >> > > remove them from our repository.
> > >> >>> > > >> >> > >
> > >> >>> > > >> >> > > Please also make sure they are available with same
> > >> artifact
> > >> >>> > name
> > >> >>> > > >> and
> > >> >>> > > >> >> API.
> > >> >>> > > >> >> > > If this has to be changed, let's wait for AI 3.0
> > >> >>> > > >> >> > >
> > >> >>> > > >> >> > > Regards,
> > >> >>> > > >> >> > > --
> > >> >>> > > >> >> > > Ilya Kasnacheev
> > >> >>> > > >> >> > >
> > >> >>> > > >> >> > >
> > >> >>> > > >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <
> > >> >>> > > >> saikat.maitra@gmail.com>:
> > >> >>> > > >> >> > >
> > >> >>> > > >> >> > > > Hello Emmanouil,
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > > > As discussed earlier, I discussed with Apache
> Bahir
> > >> >>> community
> > >> >>> > > and
> > >> >>> > > >> >> they
> > >> >>> > > >> >> > > are
> > >> >>> > > >> >> > > > interested to have Apache Ignite extensions as
> part
> > of
> > >> >>> Apache
> > >> >>> > > >> Bahir
> > >> >>> > > >> >> > > > project.
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > > > Can you please share Pub/Sub streamer
> implementation
> > >> >>> details
> > >> >>> > > with
> > >> >>> > > >> >> > Apache
> > >> >>> > > >> >> > > > Bahir community and request for feedback.
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > > > https://bahir.apache.org/
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > > > I have also requested for contributor access in
> Jira
> > >> for
> > >> >>> > Apache
> > >> >>> > > >> >> Bahir
> > >> >>> > > >> >> > > > project so that I can create issues and assign to
> > >> myself.
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > > > I can help with code reviews as well.
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > > > Here is the email thread for reference
> > >> >>> > > >> >> > > >
> > >> >>> > >
> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > > > Regards,
> > >> >>> > > >> >> > > > Saikat
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <
> > >> >>> > > >> >> saikat.maitra@gmail.com
> > >> >>> > > >> >> > >
> > >> >>> > > >> >> > > > wrote:
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > > > > Hello,
> > >> >>> > > >> >> > > > >
> > >> >>> > > >> >> > > > > As discussed I have created following discussion
> > >> >>> threads on
> > >> >>> > > our
> > >> >>> > > >> >> > > migration
> > >> >>> > > >> >> > > > > proposals for Apache Ignite extensions:
> > >> >>> > > >> >> > > > >
> > >> >>> > > >> >> > > > >
> > >> >>> > > >> >> > > > >
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > >
> > >> >>> > > >> >> >
> > >> >>> > > >> >>
> > >> >>> > > >>
> > >> >>> > >
> > >> >>> >
> > >> >>>
> > >>
> >
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> > >> >>> > > >> >> > > > >
> > >> >>> > > >> >> > > > >
> > >> >>> > > >>
> > >> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > >> >>> > > >> >> > > > >
> > >> >>> > > >> >> > > > > I will share update as I hear more information.
> > >> >>> > > >> >> > > > >
> > >> >>> > > >> >> > > > > Regards,
> > >> >>> > > >> >> > > > > Saikat
> > >> >>> > > >> >> > > > >
> > >> >>> > > >> >> > > > >
> > >> >>> > > >> >> > > > >
> > >> >>> > > >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra <
> > >> >>> > > >> >> > saikat.maitra@gmail.com
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > > > > wrote:
> > >> >>> > > >> >> > > > >
> > >> >>> > > >> >> > > > >> Hello Denis,
> > >> >>> > > >> >> > > > >>
> > >> >>> > > >> >> > > > >> Sure, sounds good. I will create a separate
> > >> discussion
> > >> >>> > > thread
> > >> >>> > > >> to
> > >> >>> > > >> >> get
> > >> >>> > > >> >> > > > >> community feedback on whether we
> > >> >>> > > >> >> > > > >> should join the Bahir or kick-off "Ignite
> > >> Extensions"
> > >> >>> > > project.
> > >> >>> > > >> >> > > > >>
> > >> >>> > > >> >> > > > >> Regards,
> > >> >>> > > >> >> > > > >> Saikat
> > >> >>> > > >> >> > > > >>
> > >> >>> > > >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda <
> > >> >>> > > >> dmagda@apache.org>
> > >> >>> > > >> >> > > wrote:
> > >> >>> > > >> >> > > > >>
> > >> >>> > > >> >> > > > >>> Folks,
> > >> >>> > > >> >> > > > >>>
> > >> >>> > > >> >> > > > >>> The concept of Apache Bahir is precisely what
> we
> > >> >>> need!
> > >> >>> > > >> Saikat,
> > >> >>> > > >> >> > thanks
> > >> >>> > > >> >> > > > for
> > >> >>> > > >> >> > > > >>> doing the research. Why I believe the * idea*
> > >> fits us
> > >> >>> > well:
> > >> >>> > > >> >> > > > >>>
> > >> >>> > > >> >> > > > >>>    - All integrations can be stored in
> separate
> > >> >>> Github
> > >> >>> > > >> >> repositories
> > >> >>> > > >> >> > > and
> > >> >>> > > >> >> > > > >>>    have their dev lifecycles. We've not
> obliged
> > to
> > >> >>> couple
> > >> >>> > > all
> > >> >>> > > >> >> the
> > >> >>> > > >> >> > > > >>> integrations
> > >> >>> > > >> >> > > > >>>    in a single repo. For instance, Spark can
> be
> > >> >>> located
> > >> >>> > in
> > >> >>> > > a
> > >> >>> > > >> >> > > dedicated
> > >> >>> > > >> >> > > > >>> repo
> > >> >>> > > >> >> > > > >>>    while streaming integrations might be in a
> > >> single
> > >> >>> one.
> > >> >>> > > >> It's
> > >> >>> > > >> >> up
> > >> >>> > > >> >> > to
> > >> >>> > > >> >> > > > us.
> > >> >>> > > >> >> > > > >>>    - All the repositories and integrations
> > belong
> > >> to
> > >> >>> ASF.
> > >> >>> > > >> We're
> > >> >>> > > >> >> not
> > >> >>> > > >> >> > > > >>> dumping
> > >> >>> > > >> >> > > > >>>    them on Github but rather keep supporting
> and
> > >> >>> > developing
> > >> >>> > > >> in
> > >> >>> > > >> >> > > > >>> accordance with
> > >> >>> > > >> >> > > > >>>    ASF vision and practices.
> > >> >>> > > >> >> > > > >>>    - There is a way to reward contributors via
> > >> >>> > > committership
> > >> >>> > > >> and
> > >> >>> > > >> >> > PMC
> > >> >>> > > >> >> > > > >>>    membership. You won't get this if a project
> > is
> > >> >>> just
> > >> >>> > one
> > >> >>> > > of
> > >> >>> > > >> >> the
> > >> >>> > > >> >> > > > >>> millions of
> > >> >>> > > >> >> > > > >>>    Github projects.
> > >> >>> > > >> >> > > > >>>
> > >> >>> > > >> >> > > > >>> Saikat, as Ignite PMC member, could you please
> > >> >>> kick-off a
> > >> >>> > > >> >> separate
> > >> >>> > > >> >> > > > >>> dev-list
> > >> >>> > > >> >> > > > >>> discussion to involve more community members
> and
> > >> >>> > referring
> > >> >>> > > to
> > >> >>> > > >> >> this
> > >> >>> > > >> >> > > > >>> thread?
> > >> >>> > > >> >> > > > >>> I think the primary question the community
> needs
> > >> to
> > >> >>> > answer
> > >> >>> > > >> >> whether
> > >> >>> > > >> >> > we
> > >> >>> > > >> >> > > > >>> should join the Bahir or kick-off "Ignite
> > >> Extensions"
> > >> >>> > > >> project?
> > >> >>> > > >> >> > > > >>>
> > >> >>> > > >> >> > > > >>> -
> > >> >>> > > >> >> > > > >>> Denis
> > >> >>> > > >> >> > > > >>>
> > >> >>> > > >> >> > > > >>>
> > >> >>> > > >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey
> Zinoviev
> > <
> > >> >>> > > >> >> > > > zaleslaw.sin@gmail.com>
> > >> >>> > > >> >> > > > >>> wrote:
> > >> >>> > > >> >> > > > >>>
> > >> >>> > > >> >> > > > >>> > Maybe we could move all our Streaming
> > >> Integrations
> > >> >>> > there,
> > >> >>> > > >> but
> > >> >>> > > >> >> > what
> > >> >>> > > >> >> > > is
> > >> >>> > > >> >> > > > >>> about
> > >> >>> > > >> >> > > > >>> > maintaining and committer permissions to the
> > new
> > >> >>> > > >> repositories?
> > >> >>> > > >> >> > > > >>> > I see the list of the committers and PMC
> > members
> > >> >>> there
> > >> >>> > > >> >> > > > >>> > https://bahir.apache.org/community-members/
> > >> >>> > > >> >> > > > >>> >
> > >> >>> > > >> >> > > > >>> > Could somebody from Ignite community be
> added
> > to
> > >> >>> this
> > >> >>> > > list
> > >> >>> > > >> as
> > >> >>> > > >> >> a
> > >> >>> > > >> >> > > > >>> > PMC/committer to maintain Ignite
> integrations?
> > >> >>> > > >> >> > > > >>> > If yes, I'd happy to join this project and
> > >> >>> collaborate
> > >> >>> > > with
> > >> >>> > > >> >> these
> > >> >>> > > >> >> > > > guys
> > >> >>> > > >> >> > > > >>> > together
> > >> >>> > > >> >> > > > >>> > If no, and we should start from the zero
> level
> > >> with
> > >> >>> > > >> external
> > >> >>> > > >> >> PRs
> > >> >>> > > >> >> > -
> > >> >>> > > >> >> > > > >>> hmmm,
> > >> >>> > > >> >> > > > >>> > it's better to have our own external
> > repository
> > >> >>> with
> > >> >>> > > >> >> ApacheBahirr
> > >> >>> > > >> >> > > > >>> ideology.
> > >> >>> > > >> >> > > > >>> >
> > >> >>> > > >> >> > > > >>> > Also, I agree, that all connectors could be
> > >> moved
> > >> >>> there
> > >> >>> > > (in
> > >> >>> > > >> >> > > > >>> ApacheBahirr
> > >> >>> > > >> >> > > > >>> > like repository), but not all modules from
> the
> > >> page
> > >> >>> > > >> >> > > > >>> >
> > >> >>> > > >> >> > > > >>> >
> > >> >>> > > >> >> > > > >>>
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > >
> > >> >>> > > >> >> >
> > >> >>> > > >> >>
> > >> >>> > > >>
> > >> >>> > >
> > >> >>> >
> > >> >>>
> > >>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > >> >>> > > >> >> > > > >>> > because
> > >> >>> > > >> >> > > > >>> > they use different parts of core modules and
> > >> could
> > >> >>> be
> > >> >>> > > >> >> developed
> > >> >>> > > >> >> > > with
> > >> >>> > > >> >> > > > >>> > different velocity.
> > >> >>> > > >> >> > > > >>> >
> > >> >>> > > >> >> > > > >>> > In reality, before creation of new
> > repositories
> > >> we
> > >> >>> need
> > >> >>> > > >> wide
> > >> >>> > > >> >> > > > discussion
> > >> >>> > > >> >> > > > >>> > based on the document mentioned above.
> > >> >>> > > >> >> > > > >>> >
> > >> >>> > > >> >> > > > >>> > P.S. Of course, we could experiment in the
> > next
> > >> >>> release
> > >> >>> > > 2.8
> > >> >>> > > >> >> with
> > >> >>> > > >> >> > > one
> > >> >>> > > >> >> > > > >>> or two
> > >> >>> > > >> >> > > > >>> > integrations like twitter or ZeroMQ
> > >> >>> > > >> >> > > > >>> > Also, I'd like Denis Magda idea for the
> > separate
> > >> >>> > > >> repositories
> > >> >>> > > >> >> in
> > >> >>> > > >> >> > > > Apache
> > >> >>> > > >> >> > > > >>> > Github to maintain them by anybody.
> > >> >>> > > >> >> > > > >>> >
> > >> >>> > > >> >> > > > >>> >
> > >> >>> > > >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra <
> > >> >>> > > >> >> > > saikat.maitra@gmail.com
> > >> >>> > > >> >> > > > >:
> > >> >>> > > >> >> > > > >>> >
> > >> >>> > > >> >> > > > >>> > > Hi Denis, Emmanouil
> > >> >>> > > >> >> > > > >>> > >
> > >> >>> > > >> >> > > > >>> > > Thank you for your email. I think
> creating a
> > >> >>> separate
> > >> >>> > > >> >> > integration
> > >> >>> > > >> >> > > > >>> project
> > >> >>> > > >> >> > > > >>> > > in github and also proposing it as an
> apache
> > >> >>> > incubator
> > >> >>> > > >> >> project
> > >> >>> > > >> >> > > will
> > >> >>> > > >> >> > > > >>> be a
> > >> >>> > > >> >> > > > >>> > > good move. The other separate integration
> > >> modules
> > >> >>> > > >> >> > > > >>> > >
> > >> >>> > > >> >> > > > >>> > >
> > >> >>> > > >> >> > > > >>> >
> > >> >>> > > >> >> > > > >>>
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > >
> > >> >>> > > >> >> >
> > >> >>> > > >> >>
> > >> >>> > > >>
> > >> >>> > >
> > >> >>> >
> > >> >>>
> > >>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > >> >>> > > >> >> > > > >>> > > can
> > >> >>> > > >> >> > > > >>> > > be moved to this new apache incubator
> > project.
> > >> >>> > > >> >> > > > >>> > >
> > >> >>> > > >> >> > > > >>> > > There are similar integration available
> for
> > >> >>> Flink and
> > >> >>> > > >> Spark
> > >> >>> > > >> >> in
> > >> >>> > > >> >> > > > Apache
> > >> >>> > > >> >> > > > >>> > Bahir
> > >> >>> > > >> >> > > > >>> > > https://bahir.apache.org/
> > >> >>> > > >> >> > > > >>> > >
> > >> >>> > > >> >> > > > >>> > > Do you think we should reach out to Apache
> > >> Bahir
> > >> >>> > > >> community
> > >> >>> > > >> >> > with a
> > >> >>> > > >> >> > > > >>> > proposal
> > >> >>> > > >> >> > > > >>> > > or we should plan to create a separate
> > Apache
> > >> >>> > Incubator
> > >> >>> > > >> >> > project?
> > >> >>> > > >> >> > > > >>> > >
> > >> >>> > > >> >> > > > >>> > > Regards,
> > >> >>> > > >> >> > > > >>> > > Saikat
> > >> >>> > > >> >> > > > >>> > >
> > >> >>> > > >> >> > > > >>> > >
> > >> >>> > > >> >> > > > >>> > >
> > >> >>> > > >> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil
> > >> >>> > Gkatziouras <
> > >> >>> > > >> >> > > > >>> > > gkatzioura@gmail.com>
> > >> >>> > > >> >> > > > >>> > > wrote:
> > >> >>> > > >> >> > > > >>> > >
> > >> >>> > > >> >> > > > >>> > > > Hi all!
> > >> >>> > > >> >> > > > >>> > > >
> > >> >>> > > >> >> > > > >>> > > > Based on the discussions most probably
> > this
> > >> is
> > >> >>> > going
> > >> >>> > > to
> > >> >>> > > >> >> be on
> > >> >>> > > >> >> > > > >>> another
> > >> >>> > > >> >> > > > >>> > > > GitHub repo. Therefore I will prepare a
> > >> >>> standalone
> > >> >>> > > >> project
> > >> >>> > > >> >> > with
> > >> >>> > > >> >> > > > the
> > >> >>> > > >> >> > > > >>> > > Pub/Sub
> > >> >>> > > >> >> > > > >>> > > > module and once the repository is
> created
> > a
> > >> >>> pull
> > >> >>> > > >> request
> > >> >>> > > >> >> > shall
> > >> >>> > > >> >> > > be
> > >> >>> > > >> >> > > > >>> > issued.
> > >> >>> > > >> >> > > > >>> > > > If there is anything else I can do in
> > order
> > >> to
> > >> >>> > > >> facilitate
> > >> >>> > > >> >> > this
> > >> >>> > > >> >> > > > >>> please
> > >> >>> > > >> >> > > > >>> > let
> > >> >>> > > >> >> > > > >>> > > > me know.
> > >> >>> > > >> >> > > > >>> > > >
> > >> >>> > > >> >> > > > >>> > > > If a ticket is created for this issue, I
> > >> shall
> > >> >>> be
> > >> >>> > > able
> > >> >>> > > >> to
> > >> >>> > > >> >> > > create
> > >> >>> > > >> >> > > > a
> > >> >>> > > >> >> > > > >>> pull
> > >> >>> > > >> >> > > > >>> > > > request and thus a build will take place
> > on
> > >> >>> > ignite's
> > >> >>> > > >> CI.
> > >> >>> > > >> >> > > > >>> > > > If everyone is aligned with adding a
> > Pub/Sub
> > >> >>> > > >> >> implementation
> > >> >>> > > >> >> > > may I
> > >> >>> > > >> >> > > > >>> > create
> > >> >>> > > >> >> > > > >>> > > a
> > >> >>> > > >> >> > > > >>> > > > ticket on JIRA?
> > >> >>> > > >> >> > > > >>> > > >
> > >> >>> > > >> >> > > > >>> > > > My next question has to do on how Ignite
> > >> >>> configures
> > >> >>> > > the
> > >> >>> > > >> >> build
> > >> >>> > > >> >> > > > jobs
> > >> >>> > > >> >> > > > >>> in
> > >> >>> > > >> >> > > > >>> > > Team
> > >> >>> > > >> >> > > > >>> > > > City. The unit tests for my
> implementation
> > >> >>> need a
> > >> >>> > > local
> > >> >>> > > >> >> > Pub/Sub
> > >> >>> > > >> >> > > > >>> server
> > >> >>> > > >> >> > > > >>> > up
> > >> >>> > > >> >> > > > >>> > > > and running (they provide one for
> testing
> > >> >>> > purposes).
> > >> >>> > > >> Will
> > >> >>> > > >> >> > this
> > >> >>> > > >> >> > > > take
> > >> >>> > > >> >> > > > >>> > > effect
> > >> >>> > > >> >> > > > >>> > > > in a form of a build script inside my
> > >> >>> > implementation
> > >> >>> > > or
> > >> >>> > > >> >> is it
> > >> >>> > > >> >> > > > >>> something
> > >> >>> > > >> >> > > > >>> > > > that should be configured on Team City?
> > >> >>> > > >> >> > > > >>> > > >
> > >> >>> > > >> >> > > > >>> > > > Kind regards,
> > >> >>> > > >> >> > > > >>> > > > Emmanouil
> > >> >>> > > >> >> > > > >>> > > >
> > >> >>> > > >> >> > > > >>> > > >
> > >> >>> > > >> >> > > > >>> > > > *Emmanouil Gkatziouras*
> > >> >>> > > >> >> > > > >>> > > > https://egkatzioura.com/ |
> > >> >>> > > >> >> > > > >>> > > >
> > >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> >>> > > >> >> > > > >>> > > > https://github.com/gkatzioura
> > >> >>> > > >> >> > > > >>> > > >
> > >> >>> > > >> >> > > > >>> > > >
> > >> >>> > > >> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis
> Magda
> > <
> > >> >>> > > >> >> dmagda@apache.org
> > >> >>> > > >> >> > >
> > >> >>> > > >> >> > > > >>> wrote:
> > >> >>> > > >> >> > > > >>> > > >
> > >> >>> > > >> >> > > > >>> > > > > Emmanouil, Saikat,
> > >> >>> > > >> >> > > > >>> > > > >
> > >> >>> > > >> >> > > > >>> > > > > After contemplating on this, let me
> > >> propose
> > >> >>> > another
> > >> >>> > > >> way
> > >> >>> > > >> >> > with
> > >> >>> > > >> >> > > > this
> > >> >>> > > >> >> > > > >>> > > > > particular integration that is aligned
> > >> with
> > >> >>> our
> > >> >>> > > >> >> > > modularization
> > >> >>> > > >> >> > > > >>> > > endeavor.
> > >> >>> > > >> >> > > > >>> > > > > The modularization [1] defines the
> > Ignite
> > >> >>> core,
> > >> >>> > > >> Ignite
> > >> >>> > > >> >> > > modules,
> > >> >>> > > >> >> > > > >>> and
> > >> >>> > > >> >> > > > >>> > > > > independent integrations. The first
> two
> > >> are
> > >> >>> to be
> > >> >>> > > >> >> located
> > >> >>> > > >> >> > in
> > >> >>> > > >> >> > > > >>> Ignite
> > >> >>> > > >> >> > > > >>> > > > > repositories and to be tested/updated
> by
> > >> the
> > >> >>> > > >> community
> > >> >>> > > >> >> for
> > >> >>> > > >> >> > > > every
> > >> >>> > > >> >> > > > >>> > > release.
> > >> >>> > > >> >> > > > >>> > > > > The modules are a subset of existing
> > >> >>> integrations
> > >> >>> > > >> that
> > >> >>> > > >> >> are
> > >> >>> > > >> >> > > > >>> adopted
> > >> >>> > > >> >> > > > >>> > > widely
> > >> >>> > > >> >> > > > >>> > > > > and require community attention.
> > >> >>> > > >> >> > > > >>> > > > >
> > >> >>> > > >> >> > > > >>> > > > > As for the independent integrations
> [1],
> > >> >>> those
> > >> >>> > are
> > >> >>> > > >> >> supposed
> > >> >>> > > >> >> > > to
> > >> >>> > > >> >> > > > be
> > >> >>> > > >> >> > > > >>> > > located
> > >> >>> > > >> >> > > > >>> > > > > in a separate GitHub repo. I would
> > advise
> > >> us
> > >> >>> to
> > >> >>> > > start
> > >> >>> > > >> >> > > > >>> implementing
> > >> >>> > > >> >> > > > >>> > this
> > >> >>> > > >> >> > > > >>> > > > > concept with the Pub/Sub integration.
> > Even
> > >> >>> though
> > >> >>> > > it
> > >> >>> > > >> >> will
> > >> >>> > > >> >> > be
> > >> >>> > > >> >> > > > >>> located
> > >> >>> > > >> >> > > > >>> > > in a
> > >> >>> > > >> >> > > > >>> > > > > separate Github repo, it will be
> listed
> > >> among
> > >> >>> > > >> officially
> > >> >>> > > >> >> > > > >>> available
> > >> >>> > > >> >> > > > >>> > > > > integrations on the Ignite website
> > >> featuring
> > >> >>> > > >> Emmanouil
> > >> >>> > > >> >> as a
> > >> >>> > > >> >> > > > >>> > > contributor.
> > >> >>> > > >> >> > > > >>> > > > >
> > >> >>> > > >> >> > > > >>> > > > > What do you think?
> > >> >>> > > >> >> > > > >>> > > > >
> > >> >>> > > >> >> > > > >>> > > > >
> > >> >>> > > >> >> > > > >>> > > > > [1]
> > >> >>> > > >> >> > > > >>> > > > >
> > >> >>> > > >> >> > > > >>> > > > >
> > >> >>> > > >> >> > > > >>> > > >
> > >> >>> > > >> >> > > > >>> > >
> > >> >>> > > >> >> > > > >>> >
> > >> >>> > > >> >> > > > >>>
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > >
> > >> >>> > > >> >> >
> > >> >>> > > >> >>
> > >> >>> > > >>
> > >> >>> > >
> > >> >>> >
> > >> >>>
> > >>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > >> >>> > > >> >> > > > >>> > > > >
> > >> >>> > > >> >> > > > >>> > > > >
> > >> >>> > > >> >> > > > >>> > > > >
> > >> >>> > > >> >> > > > >>> > > > > -
> > >> >>> > > >> >> > > > >>> > > > > Denis
> > >> >>> > > >> >> > > > >>> > > > >
> > >> >>> > > >> >> > > > >>> > > > >
> > >> >>> > > >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM Saikat
> > >> >>> Maitra <
> > >> >>> > > >> >> > > > >>> > saikat.maitra@gmail.com
> > >> >>> > > >> >> > > > >>> > > >
> > >> >>> > > >> >> > > > >>> > > > > wrote:
> > >> >>> > > >> >> > > > >>> > > > >
> > >> >>> > > >> >> > > > >>> > > > > > Hi Emmanouil,
> > >> >>> > > >> >> > > > >>> > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > The changes looks good to me. I
> wanted
> > >> to
> > >> >>> check
> > >> >>> > > if
> > >> >>> > > >> you
> > >> >>> > > >> >> > are
> > >> >>> > > >> >> > > > also
> > >> >>> > > >> >> > > > >>> > able
> > >> >>> > > >> >> > > > >>> > > to
> > >> >>> > > >> >> > > > >>> > > > > > validate that the once data is added
> > to
> > >> >>> Ignite
> > >> >>> > > >> cluster
> > >> >>> > > >> >> > you
> > >> >>> > > >> >> > > > are
> > >> >>> > > >> >> > > > >>> also
> > >> >>> > > >> >> > > > >>> > > > able
> > >> >>> > > >> >> > > > >>> > > > > to
> > >> >>> > > >> >> > > > >>> > > > > > access it using another Ignite
> client
> > or
> > >> >>> using
> > >> >>> > > rest
> > >> >>> > > >> >> > > > endpoints.
> > >> >>> > > >> >> > > > >>> > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > We use Teamcity for CI process, the
> > >> >>> details are
> > >> >>> > > as
> > >> >>> > > >> >> > follows
> > >> >>> > > >> >> > > > >>> > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > Trigger validation of those test
> > suites
> > >> >>> that
> > >> >>> > have
> > >> >>> > > >> been
> > >> >>> > > >> >> > > > >>> affected by
> > >> >>> > > >> >> > > > >>> > > your
> > >> >>> > > >> >> > > > >>> > > > > > changes on TeamCity <
> > >> >>> > > http://ci.ignite.apache.org/>:
> > >> >>> > > >> <
> > >> >>> > > >> >> > > > >>> > > > > http://204.14.53.153/
> > >> >>> > > >> >> > > > >>> > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > >
> > >> >>> > > >> >> > > > >>> > > > > >    - Locate a test suite you have to
> > >> check,
> > >> >>> > press
> > >> >>> > > >> >> button
> > >> >>> > > >> >> > > > named
> > >> >>> > > >> >> > > > >>> > "..."
> > >> >>> > > >> >> > > > >>> > > > that
> > >> >>> > > >> >> > > > >>> > > > > >    is located on the left of "Run"
> > >> button.
> > >> >>> "Run
> > >> >>> > > >> custom
> > >> >>> > > >> >> > > build"
> > >> >>> > > >> >> > > > >>> > dialog
> > >> >>> > > >> >> > > > >>> > > > will
> > >> >>> > > >> >> > > > >>> > > > > >    appear;
> > >> >>> > > >> >> > > > >>> > > > > >    - Go to "Changes" tab and choose
> > >> >>> > > >> >> > > > >>> > "pull/<pull-request-number>/head"
> > >> >>> > > >> >> > > > >>> > > > in
> > >> >>> > > >> >> > > > >>> > > > > >    "Build branch" dropdown list;
> > >> >>> > > >> >> > > > >>> > > > > >    - Press "Run build" button and
> > >> monitor
> > >> >>> tests
> > >> >>> > > >> >> results.
> > >> >>> > > >> >> > > > >>> > > > > >
> > >> >>> > > >> >> > > > >>> > > > > >
> > >> >>> > > >> >> > > > >>> > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > @Roman rshtykh@yahoo.com - can you
> > also
> > >> >>> please
> > >> >>> > > >> take a
> > >> >>> > > >> >> > > look?
> > >> >>> > > >> >> > > > >>> > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > Regards,
> > >> >>> > > >> >> > > > >>> > > > > > Saikat
> > >> >>> > > >> >> > > > >>> > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20 PM
> > >> Emmanouil
> > >> >>> > > >> Gkatziouras
> > >> >>> > > >> >> <
> > >> >>> > > >> >> > > > >>> > > > > > gkatzioura@gmail.com>
> > >> >>> > > >> >> > > > >>> > > > > > wrote:
> > >> >>> > > >> >> > > > >>> > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > Hi @Saikat Maitra
> > >> >>> > > >> >> > > > >>> > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > I have my implementation on an
> > Ignite
> > >> >>> fork on
> > >> >>> > > my
> > >> >>> > > >> >> > personal
> > >> >>> > > >> >> > > > >>> GitHub
> > >> >>> > > >> >> > > > >>> > > > > account
> > >> >>> > > >> >> > > > >>> > > > > > > [1]
> > >> >>> > > >> >> > > > >>> > > > > > > It is based on the Kafka
> > >> implementation
> > >> >>> which
> > >> >>> > > is
> > >> >>> > > >> of
> > >> >>> > > >> >> a
> > >> >>> > > >> >> > > > polling
> > >> >>> > > >> >> > > > >>> > > nature.
> > >> >>> > > >> >> > > > >>> > > > > > > GCP does not provide a service
> like
> > >> >>> Kafka.
> > >> >>> > > >> Instead
> > >> >>> > > >> >> it
> > >> >>> > > >> >> > > > >>> provides us
> > >> >>> > > >> >> > > > >>> > > > with
> > >> >>> > > >> >> > > > >>> > > > > > > Pub/Sub which is just like Kafka.
> > >> >>> > > >> >> > > > >>> > > > > > > Therefore using the Kafka streamer
> > as
> > >> a
> > >> >>> > > guideline
> > >> >>> > > >> >> made
> > >> >>> > > >> >> > > more
> > >> >>> > > >> >> > > > >>> sense
> > >> >>> > > >> >> > > > >>> > > to
> > >> >>> > > >> >> > > > >>> > > > > me.
> > >> >>> > > >> >> > > > >>> > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > Apart from the implementation I
> have
> > >> also
> > >> >>> > > added a
> > >> >>> > > >> >> unit
> > >> >>> > > >> >> > > > test.
> > >> >>> > > >> >> > > > >>> > > > > > > It definitely needs some polishing
> > >> and a
> > >> >>> > double
> > >> >>> > > >> >> check
> > >> >>> > > >> >> > on
> > >> >>> > > >> >> > > my
> > >> >>> > > >> >> > > > >>> side
> > >> >>> > > >> >> > > > >>> > to
> > >> >>> > > >> >> > > > >>> > > > > make
> > >> >>> > > >> >> > > > >>> > > > > > > the review smooth.
> > >> >>> > > >> >> > > > >>> > > > > > > For the tests a local Pub/Sub [2]
> > >> server
> > >> >>> has
> > >> >>> > to
> > >> >>> > > >> be
> > >> >>> > > >> >> > setup
> > >> >>> > > >> >> > > > >>> however
> > >> >>> > > >> >> > > > >>> > I
> > >> >>> > > >> >> > > > >>> > > am
> > >> >>> > > >> >> > > > >>> > > > > not
> > >> >>> > > >> >> > > > >>> > > > > > > sure which is the best way to do
> so
> > >> with
> > >> >>> > > regards
> > >> >>> > > >> to
> > >> >>> > > >> >> > your
> > >> >>> > > >> >> > > > >>> CI/CD.
> > >> >>> > > >> >> > > > >>> > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > [1]
> > >> >>> > > >> >> > > > >>> > >
> > >> >>> > > >> >> >
> > >> >>> https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > >> >>> > > >> >> > > > >>> > > > > > > [2]
> > >> >>> > > >> >> > > > >>>
> > >> >>> > https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
> > >> >>> > > >> >> > > > >>> > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > *Emmanouil Gkatziouras*
> > >> >>> > > >> >> > > > >>> > > > > > > https://egkatzioura.com/ |
> > >> >>> > > >> >> > > > >>> > > > > > >
> > >> >>> > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> >>> > > >> >> > > > >>> > > > > > > https://github.com/gkatzioura
> > >> >>> > > >> >> > > > >>> > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > On Sat, 12 Oct 2019 at 02:11,
> Saikat
> > >> >>> Maitra <
> > >> >>> > > >> >> > > > >>> > > saikat.maitra@gmail.com
> > >> >>> > > >> >> > > > >>> > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > wrote:
> > >> >>> > > >> >> > > > >>> > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > Hello Emmanouil,
> > >> >>> > > >> >> > > > >>> > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > Can you please share more info
> > about
> > >> >>> the
> > >> >>> > > >> >> > > implementation?
> > >> >>> > > >> >> > > > >>> > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > We have similar implementation
> for
> > >> data
> > >> >>> > > >> streamers
> > >> >>> > > >> >> > with
> > >> >>> > > >> >> > > > >>> Kafka
> > >> >>> > > >> >> > > > >>> > and
> > >> >>> > > >> >> > > > >>> > > > > > RocketMQ
> > >> >>> > > >> >> > > > >>> > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > >
> > >> >>> > > >> >> >
> https://apacheignite-mix.readme.io/docs/kafka-streamer
> > >> >>> > > >> >> > > > >>> > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > >
> > >> >>> > > >> >> > > >
> > >> >>> https://apacheignite-mix.readme.io/docs/rocketmq-streamer
> > >> >>> > > >> >> > > > >>> > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > Please review and share if you
> > have
> > >> any
> > >> >>> > > >> questions.
> > >> >>> > > >> >> > > > >>> > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > Regards,
> > >> >>> > > >> >> > > > >>> > > > > > > > Saikat
> > >> >>> > > >> >> > > > >>> > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > On Fri, Oct 11, 2019 at 4:56 PM
> > >> >>> Emmanouil
> > >> >>> > > >> >> > Gkatziouras <
> > >> >>> > > >> >> > > > >>> > > > > > > > gkatzioura@gmail.com>
> > >> >>> > > >> >> > > > >>> > > > > > > > wrote:
> > >> >>> > > >> >> > > > >>> > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > Hi all,
> > >> >>> > > >> >> > > > >>> > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > By mentioning Pub/Sub I mean
> > >> Google
> > >> >>> > Cloud's
> > >> >>> > > >> >> Pub/Sub
> > >> >>> > > >> >> > > [1]
> > >> >>> > > >> >> > > > >>> > > > > > > > > Yes it is indeed cloud
> specific.
> > >> It
> > >> >>> is
> > >> >>> > the
> > >> >>> > > >> >> > messaging
> > >> >>> > > >> >> > > > >>> service
> > >> >>> > > >> >> > > > >>> > > > > provided
> > >> >>> > > >> >> > > > >>> > > > > > > by
> > >> >>> > > >> >> > > > >>> > > > > > > > > GCP.
> > >> >>> > > >> >> > > > >>> > > > > > > > > I have made the integration
> with
> > >> it,
> > >> >>> but
> > >> >>> > I
> > >> >>> > > >> >> haven't
> > >> >>> > > >> >> > > > >>> published
> > >> >>> > > >> >> > > > >>> > it
> > >> >>> > > >> >> > > > >>> > > > yet
> > >> >>> > > >> >> > > > >>> > > > > > (I
> > >> >>> > > >> >> > > > >>> > > > > > > am
> > >> >>> > > >> >> > > > >>> > > > > > > > > waiting for the
> > discussion/ticket
> > >> and
> > >> >>> > > proceed
> > >> >>> > > >> >> with
> > >> >>> > > >> >> > > the
> > >> >>> > > >> >> > > > >>> > > > > contribution).
> > >> >>> > > >> >> > > > >>> > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > Kind regards,
> > >> >>> > > >> >> > > > >>> > > > > > > > > Emmanouil
> > >> >>> > > >> >> > > > >>> > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > [1]
> > >> >>> > > >> >> https://cloud.google.com/pubsub/docs/overview
> > >> >>> > > >> >> > > > >>> > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > *Emmanouil Gkatziouras*
> > >> >>> > > >> >> > > > >>> > > > > > > > > https://egkatzioura.com/ |
> > >> >>> > > >> >> > > > >>> > > > > > > > >
> > >> >>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> >>> > > >> >> > > > >>> > > > > > > > > https://github.com/gkatzioura
> > >> >>> > > >> >> > > > >>> > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > On Fri, 11 Oct 2019 at 21:45,
> > >> Denis
> > >> >>> > Magda <
> > >> >>> > > >> >> > > > >>> dmagda@apache.org
> > >> >>> > > >> >> > > > >>> > >
> > >> >>> > > >> >> > > > >>> > > > > wrote:
> > >> >>> > > >> >> > > > >>> > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > > Hello Emmanouil,
> > >> >>> > > >> >> > > > >>> > > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > > Are you referring to any
> > >> specific
> > >> >>> > > >> >> implementation
> > >> >>> > > >> >> > > > >>> tailored
> > >> >>> > > >> >> > > > >>> > > for a
> > >> >>> > > >> >> > > > >>> > > > > > > cloud?
> > >> >>> > > >> >> > > > >>> > > > > > > > If
> > >> >>> > > >> >> > > > >>> > > > > > > > > > I'm not mistaken you were
> > >> >>> mentioning
> > >> >>> > > >> AWS/GCE.
> > >> >>> > > >> >> Is
> > >> >>> > > >> >> > > > there
> > >> >>> > > >> >> > > > >>> any
> > >> >>> > > >> >> > > > >>> > > > Github
> > >> >>> > > >> >> > > > >>> > > > > > > > source
> > >> >>> > > >> >> > > > >>> > > > > > > > > > code?
> > >> >>> > > >> >> > > > >>> > > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > > Here is a list of existing
> > >> >>> integrations
> > >> >>> > > >> with
> > >> >>> > > >> >> > > > streaming
> > >> >>> > > >> >> > > > >>> and
> > >> >>> > > >> >> > > > >>> > > > > pub/sub
> > >> >>> > > >> >> > > > >>> > > > > > > > > > frameworks:
> > >> >>> > > >> >> > > > >>> > > > > > > > > >
> > >> >>> > > >> >> https://apacheignite-mix.readme.io/docs/overview
> > >> >>> > > >> >> > > > >>> > > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > > with the source code in
> Ignite
> > >> >>> master:
> > >> >>> > > >> >> > > > >>> > > > > > > > > >
> > >> >>> > > >> >> > >
> https://github.com/apache/ignite/tree/master/modules
> > >> >>> > > >> >> > > > >>> > > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > > Check out those resources to
> > see
> > >> >>> what
> > >> >>> > are
> > >> >>> > > >> the
> > >> >>> > > >> >> > most
> > >> >>> > > >> >> > > > >>> suitable
> > >> >>> > > >> >> > > > >>> > > > > Ignite
> > >> >>> > > >> >> > > > >>> > > > > > > APIs
> > >> >>> > > >> >> > > > >>> > > > > > > > > for
> > >> >>> > > >> >> > > > >>> > > > > > > > > > such integrations.
> > >> >>> > > >> >> > > > >>> > > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > > @Saikat Maitra <
> > >> >>> > saikat.maitra@gmail.com>
> > >> >>> > > ,
> > >> >>> > > >> >> > @Roman
> > >> >>> > > >> >> > > <
> > >> >>> > > >> >> > > > >>> > > > > > rshtykh@yahoo.com
> > >> >>> > > >> >> > > > >>> > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > ,
> > >> >>> > > >> >> > > > >>> > > > > > > > > as
> > >> >>> > > >> >> > > > >>> > > > > > > > > > streaming technologies
> > >> >>> > > >> >> contributors/maintainers,
> > >> >>> > > >> >> > > > could
> > >> >>> > > >> >> > > > >>> you
> > >> >>> > > >> >> > > > >>> > > > please
> > >> >>> > > >> >> > > > >>> > > > > > > join
> > >> >>> > > >> >> > > > >>> > > > > > > > > this
> > >> >>> > > >> >> > > > >>> > > > > > > > > > thread?
> > >> >>> > > >> >> > > > >>> > > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > > -
> > >> >>> > > >> >> > > > >>> > > > > > > > > > Denis
> > >> >>> > > >> >> > > > >>> > > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > > On Thu, Oct 10, 2019 at 7:58
> > AM
> > >> >>> > Emmanouil
> > >> >>> > > >> >> > > > Gkatziouras <
> > >> >>> > > >> >> > > > >>> > > > > > > > > > gkatzioura@gmail.com>
> > >> >>> > > >> >> > > > >>> > > > > > > > > > wrote:
> > >> >>> > > >> >> > > > >>> > > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > > > Hello everyone,
> > >> >>> > > >> >> > > > >>> > > > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > > > I started using Ignite
> > lately.
> > >> >>> Part
> > >> >>> > of
> > >> >>> > > my
> > >> >>> > > >> >> work
> > >> >>> > > >> >> > > > >>> involves
> > >> >>> > > >> >> > > > >>> > > > > Pub/Sub,
> > >> >>> > > >> >> > > > >>> > > > > > > > thus I
> > >> >>> > > >> >> > > > >>> > > > > > > > > > > created a pub/sub
> streamer.
> > >> >>> > > >> >> > > > >>> > > > > > > > > > > Pub/Sub is pretty close to
> > >> Kafka.
> > >> >>> > > >> >> > > > >>> > > > > > > > > > > Here's how the streamer
> > works.
> > >> >>> Every
> > >> >>> > > node
> > >> >>> > > >> >> > creates
> > >> >>> > > >> >> > > > an
> > >> >>> > > >> >> > > > >>> > > instance
> > >> >>> > > >> >> > > > >>> > > > > of
> > >> >>> > > >> >> > > > >>> > > > > > a
> > >> >>> > > >> >> > > > >>> > > > > > > > > > Pub/Sub
> > >> >>> > > >> >> > > > >>> > > > > > > > > > > client.
> > >> >>> > > >> >> > > > >>> > > > > > > > > > > The clients subscribe to
> the
> > >> same
> > >> >>> > topic
> > >> >>> > > >> >> using
> > >> >>> > > >> >> > the
> > >> >>> > > >> >> > > > >>> same
> > >> >>> > > >> >> > > > >>> > > > > > subscription
> > >> >>> > > >> >> > > > >>> > > > > > > > > name.
> > >> >>> > > >> >> > > > >>> > > > > > > > > > > A message is being sent
> and
> > >> one
> > >> >>> of
> > >> >>> > the
> > >> >>> > > >> nodes
> > >> >>> > > >> >> > will
> > >> >>> > > >> >> > > > >>> receive
> > >> >>> > > >> >> > > > >>> > > the
> > >> >>> > > >> >> > > > >>> > > > > > > > message,
> > >> >>> > > >> >> > > > >>> > > > > > > > > > > process it and then
> forward
> > >> it to
> > >> >>> > > Ignite
> > >> >>> > > >> >> using
> > >> >>> > > >> >> > > the
> > >> >>> > > >> >> > > > >>> > > extractor
> > >> >>> > > >> >> > > > >>> > > > > > > > provided.
> > >> >>> > > >> >> > > > >>> > > > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > > > Any thoughts?
> > >> >>> > > >> >> > > > >>> > > > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > > > *Emmanouil Gkatziouras*
> > >> >>> > > >> >> > > > >>> > > > > > > > > > > https://egkatzioura.com/
> |
> > >> >>> > > >> >> > > > >>> > > > > > > > > > >
> > >> >>> > > >> >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> >>> > > >> >> > > > >>> > > > > > > > > > >
> > https://github.com/gkatzioura
> > >> >>> > > >> >> > > > >>> > > > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > > >
> > >> >>> > > >> >> > > > >>> > > > > >
> > >> >>> > > >> >> > > > >>> > > > >
> > >> >>> > > >> >> > > > >>> > > >
> > >> >>> > > >> >> > > > >>> > >
> > >> >>> > > >> >> > > > >>> >
> > >> >>> > > >> >> > > > >>>
> > >> >>> > > >> >> > > > >>
> > >> >>> > > >> >> > > >
> > >> >>> > > >> >> > >
> > >> >>> > > >> >> >
> > >> >>> > > >> >>
> > >> >>> > > >> >
> > >> >>> > > >>
> > >> >>> > > >
> > >> >>> > >
> > >> >>> >
> > >> >>>
> > >> >>
> > >>
> > >
> >
>

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra <sa...@gmail.com>]

Hi Emmanouil,

I have added you as contributor in Ignite Extensions project. Can you
please check and confirm if you are able to see the project and execute
build on your pull request.

Regards,
Saikat

On Tue, Nov 26, 2019 at 12:49 PM Emmanouil Gkatziouras <gk...@gmail.com>
wrote:

> Hi all!
>
> If someone could give me read access to the Job of Ignite Extensions on
> Team City will greatly help me (username gkatzioura).
> I suppose the builds get triggered automatically on a pull request.
>
> Kind regards,
> Emmanouil
>
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>
>
> On Sun, 24 Nov 2019 at 21:11, Emmanouil Gkatziouras <gk...@gmail.com>
> wrote:
>
> > Hi Saikat!
> >
> > I just rebased with the flink branch. Unfortunately I do not have read
> > access to the team city link you provided and thus evaluate the tests for
> > my pull request.
> > I guess it has to do with ignite extensions being new.
> > My username is gkatzioura and email the one I am using on this email
> >
> > Thank you for your time.
> >
> >
> > *Emmanouil Gkatziouras*
> > https://egkatzioura.com/ |
> > https://www.linkedin.com/in/gkatziourasemmanouil/
> > https://github.com/gkatzioura
> >
> >
> > On Sun, 24 Nov 2019 at 18:03, Saikat Maitra <sa...@gmail.com>
> > wrote:
> >
> >> Hi Emmanouil,
> >>
> >> The latest build on teamcity has passed on Flink pull request.
> >>
> >>
> >>
> https://ci.ignite.apache.org/viewLog.html?buildId=4788928&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
> >>
> >> you should be able to take the changes I made in my PR and run build on
> >> your pull request.
> >>
> >> Please let me know if you have any questions.
> >>
> >> Regards,
> >> Saikat
> >>
> >> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra <saikat.maitra@gmail.com
> >
> >> wrote:
> >>
> >> > Hi Emmanouil,
> >> >
> >> > I have fixed the Flink module testsuites and have setup new project in
> >> > teamcity.
> >> >
> >> >
> >> >
> >>
> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview
> >> >
> >> > I am looking into setting up the build.
> >> >
> >> > Regards,
> >> > Saikat
> >> >
> >> > On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra <
> saikat.maitra@gmail.com>
> >> > wrote:
> >> >
> >> >> Hi Emmanouil,
> >> >>
> >> >> Thank you for your email. Yes, the plan is once the Flink PR is
> merged
> >> it
> >> >> will provide the Licence, parent POMs etc and you can rebase from
> >> master
> >> >> branch and apply the changes on top of it.
> >> >>
> >> >> I am getting some issues with test failures in local with
> >> >> GridTestProperties as the test.properties is not present in this new
> >> >> project but should be available from dependencies.
> >> >>
> >> >> Once I address this issue, I will go ahead and merge the changes and
> >> then
> >> >> we can take it from there.
> >> >>
> >> >> Regards,
> >> >> Saikat
> >> >>
> >> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras <
> >> >> gkatzioura@gmail.com> wrote:
> >> >>
> >> >>> Hi all,
> >> >>>
> >> >>> I made my first pull request [1]. Since this project is brand new
> (no
> >> >>> parent poms, licensing), I reckoned it was best to use Saikat's
> >> branch on
> >> >>> Flink.
> >> >>> I suppose the Flink branch will be merged first. If not please give
> me
> >> >>> guidelines on how I should proceed next.
> >> >>>
> >> >>> Kind regards
> >> >>> Emmanouil
> >> >>>
> >> >>> [1] https://github.com/apache/ignite-extensions/pull/2
> >> >>>
> >> >>> *Emmanouil Gkatziouras*
> >> >>> https://egkatzioura.com/ |
> >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> >>> https://github.com/gkatzioura
> >> >>>
> >> >>>
> >> >>> On Fri, 22 Nov 2019 at 20:55, Denis Magda <dm...@apache.org>
> wrote:
> >> >>>
> >> >>> > Awesome, ping us whenever you're ready!
> >> >>> >
> >> >>> > -
> >> >>> > Denis
> >> >>> >
> >> >>> >
> >> >>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <
> >> >>> > gkatzioura@gmail.com>
> >> >>> > wrote:
> >> >>> >
> >> >>> > > Hi all!
> >> >>> > >
> >> >>> > > I am sorry for being late on that. I was trying to refactor the
> >> test
> >> >>> in
> >> >>> > > order not to be in need of any external tools or spinning up a
> >> >>> server.
> >> >>> > > I did forked the new repo and indeed my changes there, so a pull
> >> >>> request
> >> >>> > is
> >> >>> > > a matter of time!
> >> >>> > >
> >> >>> > > Kind regards
> >> >>> > >
> >> >>> > > *Emmanouil Gkatziouras*
> >> >>> > > https://egkatzioura.com/ |
> >> >>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >> >>> > > https://github.com/gkatzioura
> >> >>> > >
> >> >>> > >
> >> >>> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda <dm...@apache.org>
> >> wrote:
> >> >>> > >
> >> >>> > > > Hi Emmanouil,
> >> >>> > > >
> >> >>> > > > Do you have any questions or need any support from the
> >> community?
> >> >>> > > >
> >> >>> > > > -
> >> >>> > > > Denis
> >> >>> > > >
> >> >>> > > >
> >> >>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <
> >> >>> saikat.maitra@gmail.com
> >> >>> > >
> >> >>> > > > wrote:
> >> >>> > > >
> >> >>> > > >> Hi Emmanouil,
> >> >>> > > >>
> >> >>> > > >> Can you please take a looks at dev utils, if this is
> something
> >> >>> you are
> >> >>> > > >> looking for
> >> >>> > > >>
> https://github.com/apache/ignite/tree/master/modules/dev-utils
> >> ?
> >> >>> > > >>
> >> >>> > > >> IMO, if you can release Pub/Sub server in maven and then use
> >> it as
> >> >>> > > >> dependency, that would be great.
> >> >>> > > >>
> >> >>> > > >> Regards,
> >> >>> > > >> Saikat
> >> >>> > > >>
> >> >>> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <
> >> >>> > saikat.maitra@gmail.com>
> >> >>> > > >> wrote:
> >> >>> > > >>
> >> >>> > > >> > Hi Emmanouil,
> >> >>> > > >> >
> >> >>> > > >> > The master branch has been initialized. I have raised a PR
> to
> >> >>> > migrate
> >> >>> > > >> > Flink module to ignite-extensions.
> >> >>> > > >> >
> >> >>> > > >> > PR https://github.com/apache/ignite-extensions/pull/1
> >> >>> > > >> > Jira : https://issues.apache.org/jira/browse/IGNITE-12356
> >> >>> > > >> >
> >> >>> > > >> > Regards,
> >> >>> > > >> > Saikat
> >> >>> > > >> >
> >> >>> > > >> >
> >> >>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
> >> >>> > > >> > gkatzioura@gmail.com> wrote:
> >> >>> > > >> >
> >> >>> > > >> >> Hi all!
> >> >>> > > >> >>
> >> >>> > > >> >> This is an update. The integration is tested with mocks
> >> based
> >> >>> on
> >> >>> > the
> >> >>> > > >> >> original Ignite repo [1].
> >> >>> > > >> >> I am currently waiting for the first commits on the new
> >> >>> integration
> >> >>> > > >> >> project
> >> >>> > > >> >> to take place in order to create my pull request
> accordingly
> >> >>> to the
> >> >>> > > >> >> initial
> >> >>> > > >> >> commits.
> >> >>> > > >> >> In order to simulate the Pub/Sub server in a test I used a
> >> test
> >> >>> > > server
> >> >>> > > >> >> implementation [2]. Unfortunately they don't have this
> util
> >> >>> > uploaded
> >> >>> > > >> in a
> >> >>> > > >> >> maven repo, thus it needs to be hosted on a maven repo
> >> (Apache
> >> >>> > > License
> >> >>> > > >> >> 2.0). Is there a repo for utilities that Ignite uses for
> the
> >> >>> > > >> test/build?
> >> >>> > > >> >> I have also created a ticket [3]
> >> >>> > > >> >>
> >> >>> > > >> >> Kind regards
> >> >>> > > >> >>
> >> >>> > > >> >> [1]
> >> >>> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> >> >>> > > >> >> [2]
> >> >>> https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> >> >>> > > >> >> [3] https://issues.apache.org/jira/browse/IGNITE-12262
> >> >>> > > >> >>
> >> >>> > > >> >> *Emmanouil Gkatziouras*
> >> >>> > > >> >> https://egkatzioura.com/ |
> >> >>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> >>> > > >> >> https://github.com/gkatzioura
> >> >>> > > >> >>
> >> >>> > > >> >>
> >> >>> > > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <
> >> >>> > saikat.maitra@gmail.com
> >> >>> > > >
> >> >>> > > >> >> wrote:
> >> >>> > > >> >>
> >> >>> > > >> >> > Hello Ilya,
> >> >>> > > >> >> >
> >> >>> > > >> >> > Thank you for your email. Yes, before we remove any
> >> >>> integration
> >> >>> > we
> >> >>> > > >> will
> >> >>> > > >> >> > move them to another repository and release.
> >> >>> > > >> >> >
> >> >>> > > >> >> > We are thinking we will start with Pub/Sub Streamer
> >> >>> > implementation
> >> >>> > > >> and
> >> >>> > > >> >> > continue to move other integration as mentioned in the
> >> >>> following
> >> >>> > > doc.
> >> >>> > > >> >> >
> >> >>> > > >> >> >
> >> >>> > > >> >> >
> >> >>> > > >> >>
> >> >>> > > >>
> >> >>> > >
> >> >>> >
> >> >>>
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >> >>> > > >> >> >
> >> >>> > > >> >> > Regards,
> >> >>> > > >> >> > Saikat
> >> >>> > > >> >> >
> >> >>> > > >> >> >
> >> >>> > > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
> >> >>> > > >> >> > ilya.kasnacheev@gmail.com>
> >> >>> > > >> >> > wrote:
> >> >>> > > >> >> >
> >> >>> > > >> >> > > Hello!
> >> >>> > > >> >> > >
> >> >>> > > >> >> > > My take on this: we want to offload some of
> >> integrations,
> >> >>> but
> >> >>> > > >> please
> >> >>> > > >> >> > commit
> >> >>> > > >> >> > > and release them to another project first, and only
> >> then we
> >> >>> > will
> >> >>> > > >> >> decide
> >> >>> > > >> >> > to
> >> >>> > > >> >> > > remove them from our repository.
> >> >>> > > >> >> > >
> >> >>> > > >> >> > > Please also make sure they are available with same
> >> artifact
> >> >>> > name
> >> >>> > > >> and
> >> >>> > > >> >> API.
> >> >>> > > >> >> > > If this has to be changed, let's wait for AI 3.0
> >> >>> > > >> >> > >
> >> >>> > > >> >> > > Regards,
> >> >>> > > >> >> > > --
> >> >>> > > >> >> > > Ilya Kasnacheev
> >> >>> > > >> >> > >
> >> >>> > > >> >> > >
> >> >>> > > >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <
> >> >>> > > >> saikat.maitra@gmail.com>:
> >> >>> > > >> >> > >
> >> >>> > > >> >> > > > Hello Emmanouil,
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > > > As discussed earlier, I discussed with Apache Bahir
> >> >>> community
> >> >>> > > and
> >> >>> > > >> >> they
> >> >>> > > >> >> > > are
> >> >>> > > >> >> > > > interested to have Apache Ignite extensions as part
> of
> >> >>> Apache
> >> >>> > > >> Bahir
> >> >>> > > >> >> > > > project.
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > > > Can you please share Pub/Sub streamer implementation
> >> >>> details
> >> >>> > > with
> >> >>> > > >> >> > Apache
> >> >>> > > >> >> > > > Bahir community and request for feedback.
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > > > https://bahir.apache.org/
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > > > I have also requested for contributor access in Jira
> >> for
> >> >>> > Apache
> >> >>> > > >> >> Bahir
> >> >>> > > >> >> > > > project so that I can create issues and assign to
> >> myself.
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > > > I can help with code reviews as well.
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > > > Here is the email thread for reference
> >> >>> > > >> >> > > >
> >> >>> > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > > > Regards,
> >> >>> > > >> >> > > > Saikat
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <
> >> >>> > > >> >> saikat.maitra@gmail.com
> >> >>> > > >> >> > >
> >> >>> > > >> >> > > > wrote:
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > > > > Hello,
> >> >>> > > >> >> > > > >
> >> >>> > > >> >> > > > > As discussed I have created following discussion
> >> >>> threads on
> >> >>> > > our
> >> >>> > > >> >> > > migration
> >> >>> > > >> >> > > > > proposals for Apache Ignite extensions:
> >> >>> > > >> >> > > > >
> >> >>> > > >> >> > > > >
> >> >>> > > >> >> > > > >
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > >
> >> >>> > > >> >> >
> >> >>> > > >> >>
> >> >>> > > >>
> >> >>> > >
> >> >>> >
> >> >>>
> >>
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> >> >>> > > >> >> > > > >
> >> >>> > > >> >> > > > >
> >> >>> > > >>
> >> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> >> >>> > > >> >> > > > >
> >> >>> > > >> >> > > > > I will share update as I hear more information.
> >> >>> > > >> >> > > > >
> >> >>> > > >> >> > > > > Regards,
> >> >>> > > >> >> > > > > Saikat
> >> >>> > > >> >> > > > >
> >> >>> > > >> >> > > > >
> >> >>> > > >> >> > > > >
> >> >>> > > >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra <
> >> >>> > > >> >> > saikat.maitra@gmail.com
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > > > > wrote:
> >> >>> > > >> >> > > > >
> >> >>> > > >> >> > > > >> Hello Denis,
> >> >>> > > >> >> > > > >>
> >> >>> > > >> >> > > > >> Sure, sounds good. I will create a separate
> >> discussion
> >> >>> > > thread
> >> >>> > > >> to
> >> >>> > > >> >> get
> >> >>> > > >> >> > > > >> community feedback on whether we
> >> >>> > > >> >> > > > >> should join the Bahir or kick-off "Ignite
> >> Extensions"
> >> >>> > > project.
> >> >>> > > >> >> > > > >>
> >> >>> > > >> >> > > > >> Regards,
> >> >>> > > >> >> > > > >> Saikat
> >> >>> > > >> >> > > > >>
> >> >>> > > >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda <
> >> >>> > > >> dmagda@apache.org>
> >> >>> > > >> >> > > wrote:
> >> >>> > > >> >> > > > >>
> >> >>> > > >> >> > > > >>> Folks,
> >> >>> > > >> >> > > > >>>
> >> >>> > > >> >> > > > >>> The concept of Apache Bahir is precisely what we
> >> >>> need!
> >> >>> > > >> Saikat,
> >> >>> > > >> >> > thanks
> >> >>> > > >> >> > > > for
> >> >>> > > >> >> > > > >>> doing the research. Why I believe the * idea*
> >> fits us
> >> >>> > well:
> >> >>> > > >> >> > > > >>>
> >> >>> > > >> >> > > > >>>    - All integrations can be stored in separate
> >> >>> Github
> >> >>> > > >> >> repositories
> >> >>> > > >> >> > > and
> >> >>> > > >> >> > > > >>>    have their dev lifecycles. We've not obliged
> to
> >> >>> couple
> >> >>> > > all
> >> >>> > > >> >> the
> >> >>> > > >> >> > > > >>> integrations
> >> >>> > > >> >> > > > >>>    in a single repo. For instance, Spark can be
> >> >>> located
> >> >>> > in
> >> >>> > > a
> >> >>> > > >> >> > > dedicated
> >> >>> > > >> >> > > > >>> repo
> >> >>> > > >> >> > > > >>>    while streaming integrations might be in a
> >> single
> >> >>> one.
> >> >>> > > >> It's
> >> >>> > > >> >> up
> >> >>> > > >> >> > to
> >> >>> > > >> >> > > > us.
> >> >>> > > >> >> > > > >>>    - All the repositories and integrations
> belong
> >> to
> >> >>> ASF.
> >> >>> > > >> We're
> >> >>> > > >> >> not
> >> >>> > > >> >> > > > >>> dumping
> >> >>> > > >> >> > > > >>>    them on Github but rather keep supporting and
> >> >>> > developing
> >> >>> > > >> in
> >> >>> > > >> >> > > > >>> accordance with
> >> >>> > > >> >> > > > >>>    ASF vision and practices.
> >> >>> > > >> >> > > > >>>    - There is a way to reward contributors via
> >> >>> > > committership
> >> >>> > > >> and
> >> >>> > > >> >> > PMC
> >> >>> > > >> >> > > > >>>    membership. You won't get this if a project
> is
> >> >>> just
> >> >>> > one
> >> >>> > > of
> >> >>> > > >> >> the
> >> >>> > > >> >> > > > >>> millions of
> >> >>> > > >> >> > > > >>>    Github projects.
> >> >>> > > >> >> > > > >>>
> >> >>> > > >> >> > > > >>> Saikat, as Ignite PMC member, could you please
> >> >>> kick-off a
> >> >>> > > >> >> separate
> >> >>> > > >> >> > > > >>> dev-list
> >> >>> > > >> >> > > > >>> discussion to involve more community members and
> >> >>> > referring
> >> >>> > > to
> >> >>> > > >> >> this
> >> >>> > > >> >> > > > >>> thread?
> >> >>> > > >> >> > > > >>> I think the primary question the community needs
> >> to
> >> >>> > answer
> >> >>> > > >> >> whether
> >> >>> > > >> >> > we
> >> >>> > > >> >> > > > >>> should join the Bahir or kick-off "Ignite
> >> Extensions"
> >> >>> > > >> project?
> >> >>> > > >> >> > > > >>>
> >> >>> > > >> >> > > > >>> -
> >> >>> > > >> >> > > > >>> Denis
> >> >>> > > >> >> > > > >>>
> >> >>> > > >> >> > > > >>>
> >> >>> > > >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev
> <
> >> >>> > > >> >> > > > zaleslaw.sin@gmail.com>
> >> >>> > > >> >> > > > >>> wrote:
> >> >>> > > >> >> > > > >>>
> >> >>> > > >> >> > > > >>> > Maybe we could move all our Streaming
> >> Integrations
> >> >>> > there,
> >> >>> > > >> but
> >> >>> > > >> >> > what
> >> >>> > > >> >> > > is
> >> >>> > > >> >> > > > >>> about
> >> >>> > > >> >> > > > >>> > maintaining and committer permissions to the
> new
> >> >>> > > >> repositories?
> >> >>> > > >> >> > > > >>> > I see the list of the committers and PMC
> members
> >> >>> there
> >> >>> > > >> >> > > > >>> > https://bahir.apache.org/community-members/
> >> >>> > > >> >> > > > >>> >
> >> >>> > > >> >> > > > >>> > Could somebody from Ignite community be added
> to
> >> >>> this
> >> >>> > > list
> >> >>> > > >> as
> >> >>> > > >> >> a
> >> >>> > > >> >> > > > >>> > PMC/committer to maintain Ignite integrations?
> >> >>> > > >> >> > > > >>> > If yes, I'd happy to join this project and
> >> >>> collaborate
> >> >>> > > with
> >> >>> > > >> >> these
> >> >>> > > >> >> > > > guys
> >> >>> > > >> >> > > > >>> > together
> >> >>> > > >> >> > > > >>> > If no, and we should start from the zero level
> >> with
> >> >>> > > >> external
> >> >>> > > >> >> PRs
> >> >>> > > >> >> > -
> >> >>> > > >> >> > > > >>> hmmm,
> >> >>> > > >> >> > > > >>> > it's better to have our own external
> repository
> >> >>> with
> >> >>> > > >> >> ApacheBahirr
> >> >>> > > >> >> > > > >>> ideology.
> >> >>> > > >> >> > > > >>> >
> >> >>> > > >> >> > > > >>> > Also, I agree, that all connectors could be
> >> moved
> >> >>> there
> >> >>> > > (in
> >> >>> > > >> >> > > > >>> ApacheBahirr
> >> >>> > > >> >> > > > >>> > like repository), but not all modules from the
> >> page
> >> >>> > > >> >> > > > >>> >
> >> >>> > > >> >> > > > >>> >
> >> >>> > > >> >> > > > >>>
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > >
> >> >>> > > >> >> >
> >> >>> > > >> >>
> >> >>> > > >>
> >> >>> > >
> >> >>> >
> >> >>>
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >> >>> > > >> >> > > > >>> > because
> >> >>> > > >> >> > > > >>> > they use different parts of core modules and
> >> could
> >> >>> be
> >> >>> > > >> >> developed
> >> >>> > > >> >> > > with
> >> >>> > > >> >> > > > >>> > different velocity.
> >> >>> > > >> >> > > > >>> >
> >> >>> > > >> >> > > > >>> > In reality, before creation of new
> repositories
> >> we
> >> >>> need
> >> >>> > > >> wide
> >> >>> > > >> >> > > > discussion
> >> >>> > > >> >> > > > >>> > based on the document mentioned above.
> >> >>> > > >> >> > > > >>> >
> >> >>> > > >> >> > > > >>> > P.S. Of course, we could experiment in the
> next
> >> >>> release
> >> >>> > > 2.8
> >> >>> > > >> >> with
> >> >>> > > >> >> > > one
> >> >>> > > >> >> > > > >>> or two
> >> >>> > > >> >> > > > >>> > integrations like twitter or ZeroMQ
> >> >>> > > >> >> > > > >>> > Also, I'd like Denis Magda idea for the
> separate
> >> >>> > > >> repositories
> >> >>> > > >> >> in
> >> >>> > > >> >> > > > Apache
> >> >>> > > >> >> > > > >>> > Github to maintain them by anybody.
> >> >>> > > >> >> > > > >>> >
> >> >>> > > >> >> > > > >>> >
> >> >>> > > >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra <
> >> >>> > > >> >> > > saikat.maitra@gmail.com
> >> >>> > > >> >> > > > >:
> >> >>> > > >> >> > > > >>> >
> >> >>> > > >> >> > > > >>> > > Hi Denis, Emmanouil
> >> >>> > > >> >> > > > >>> > >
> >> >>> > > >> >> > > > >>> > > Thank you for your email. I think creating a
> >> >>> separate
> >> >>> > > >> >> > integration
> >> >>> > > >> >> > > > >>> project
> >> >>> > > >> >> > > > >>> > > in github and also proposing it as an apache
> >> >>> > incubator
> >> >>> > > >> >> project
> >> >>> > > >> >> > > will
> >> >>> > > >> >> > > > >>> be a
> >> >>> > > >> >> > > > >>> > > good move. The other separate integration
> >> modules
> >> >>> > > >> >> > > > >>> > >
> >> >>> > > >> >> > > > >>> > >
> >> >>> > > >> >> > > > >>> >
> >> >>> > > >> >> > > > >>>
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > >
> >> >>> > > >> >> >
> >> >>> > > >> >>
> >> >>> > > >>
> >> >>> > >
> >> >>> >
> >> >>>
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >> >>> > > >> >> > > > >>> > > can
> >> >>> > > >> >> > > > >>> > > be moved to this new apache incubator
> project.
> >> >>> > > >> >> > > > >>> > >
> >> >>> > > >> >> > > > >>> > > There are similar integration available for
> >> >>> Flink and
> >> >>> > > >> Spark
> >> >>> > > >> >> in
> >> >>> > > >> >> > > > Apache
> >> >>> > > >> >> > > > >>> > Bahir
> >> >>> > > >> >> > > > >>> > > https://bahir.apache.org/
> >> >>> > > >> >> > > > >>> > >
> >> >>> > > >> >> > > > >>> > > Do you think we should reach out to Apache
> >> Bahir
> >> >>> > > >> community
> >> >>> > > >> >> > with a
> >> >>> > > >> >> > > > >>> > proposal
> >> >>> > > >> >> > > > >>> > > or we should plan to create a separate
> Apache
> >> >>> > Incubator
> >> >>> > > >> >> > project?
> >> >>> > > >> >> > > > >>> > >
> >> >>> > > >> >> > > > >>> > > Regards,
> >> >>> > > >> >> > > > >>> > > Saikat
> >> >>> > > >> >> > > > >>> > >
> >> >>> > > >> >> > > > >>> > >
> >> >>> > > >> >> > > > >>> > >
> >> >>> > > >> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil
> >> >>> > Gkatziouras <
> >> >>> > > >> >> > > > >>> > > gkatzioura@gmail.com>
> >> >>> > > >> >> > > > >>> > > wrote:
> >> >>> > > >> >> > > > >>> > >
> >> >>> > > >> >> > > > >>> > > > Hi all!
> >> >>> > > >> >> > > > >>> > > >
> >> >>> > > >> >> > > > >>> > > > Based on the discussions most probably
> this
> >> is
> >> >>> > going
> >> >>> > > to
> >> >>> > > >> >> be on
> >> >>> > > >> >> > > > >>> another
> >> >>> > > >> >> > > > >>> > > > GitHub repo. Therefore I will prepare a
> >> >>> standalone
> >> >>> > > >> project
> >> >>> > > >> >> > with
> >> >>> > > >> >> > > > the
> >> >>> > > >> >> > > > >>> > > Pub/Sub
> >> >>> > > >> >> > > > >>> > > > module and once the repository is created
> a
> >> >>> pull
> >> >>> > > >> request
> >> >>> > > >> >> > shall
> >> >>> > > >> >> > > be
> >> >>> > > >> >> > > > >>> > issued.
> >> >>> > > >> >> > > > >>> > > > If there is anything else I can do in
> order
> >> to
> >> >>> > > >> facilitate
> >> >>> > > >> >> > this
> >> >>> > > >> >> > > > >>> please
> >> >>> > > >> >> > > > >>> > let
> >> >>> > > >> >> > > > >>> > > > me know.
> >> >>> > > >> >> > > > >>> > > >
> >> >>> > > >> >> > > > >>> > > > If a ticket is created for this issue, I
> >> shall
> >> >>> be
> >> >>> > > able
> >> >>> > > >> to
> >> >>> > > >> >> > > create
> >> >>> > > >> >> > > > a
> >> >>> > > >> >> > > > >>> pull
> >> >>> > > >> >> > > > >>> > > > request and thus a build will take place
> on
> >> >>> > ignite's
> >> >>> > > >> CI.
> >> >>> > > >> >> > > > >>> > > > If everyone is aligned with adding a
> Pub/Sub
> >> >>> > > >> >> implementation
> >> >>> > > >> >> > > may I
> >> >>> > > >> >> > > > >>> > create
> >> >>> > > >> >> > > > >>> > > a
> >> >>> > > >> >> > > > >>> > > > ticket on JIRA?
> >> >>> > > >> >> > > > >>> > > >
> >> >>> > > >> >> > > > >>> > > > My next question has to do on how Ignite
> >> >>> configures
> >> >>> > > the
> >> >>> > > >> >> build
> >> >>> > > >> >> > > > jobs
> >> >>> > > >> >> > > > >>> in
> >> >>> > > >> >> > > > >>> > > Team
> >> >>> > > >> >> > > > >>> > > > City. The unit tests for my implementation
> >> >>> need a
> >> >>> > > local
> >> >>> > > >> >> > Pub/Sub
> >> >>> > > >> >> > > > >>> server
> >> >>> > > >> >> > > > >>> > up
> >> >>> > > >> >> > > > >>> > > > and running (they provide one for testing
> >> >>> > purposes).
> >> >>> > > >> Will
> >> >>> > > >> >> > this
> >> >>> > > >> >> > > > take
> >> >>> > > >> >> > > > >>> > > effect
> >> >>> > > >> >> > > > >>> > > > in a form of a build script inside my
> >> >>> > implementation
> >> >>> > > or
> >> >>> > > >> >> is it
> >> >>> > > >> >> > > > >>> something
> >> >>> > > >> >> > > > >>> > > > that should be configured on Team City?
> >> >>> > > >> >> > > > >>> > > >
> >> >>> > > >> >> > > > >>> > > > Kind regards,
> >> >>> > > >> >> > > > >>> > > > Emmanouil
> >> >>> > > >> >> > > > >>> > > >
> >> >>> > > >> >> > > > >>> > > >
> >> >>> > > >> >> > > > >>> > > > *Emmanouil Gkatziouras*
> >> >>> > > >> >> > > > >>> > > > https://egkatzioura.com/ |
> >> >>> > > >> >> > > > >>> > > >
> >> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> >>> > > >> >> > > > >>> > > > https://github.com/gkatzioura
> >> >>> > > >> >> > > > >>> > > >
> >> >>> > > >> >> > > > >>> > > >
> >> >>> > > >> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis Magda
> <
> >> >>> > > >> >> dmagda@apache.org
> >> >>> > > >> >> > >
> >> >>> > > >> >> > > > >>> wrote:
> >> >>> > > >> >> > > > >>> > > >
> >> >>> > > >> >> > > > >>> > > > > Emmanouil, Saikat,
> >> >>> > > >> >> > > > >>> > > > >
> >> >>> > > >> >> > > > >>> > > > > After contemplating on this, let me
> >> propose
> >> >>> > another
> >> >>> > > >> way
> >> >>> > > >> >> > with
> >> >>> > > >> >> > > > this
> >> >>> > > >> >> > > > >>> > > > > particular integration that is aligned
> >> with
> >> >>> our
> >> >>> > > >> >> > > modularization
> >> >>> > > >> >> > > > >>> > > endeavor.
> >> >>> > > >> >> > > > >>> > > > > The modularization [1] defines the
> Ignite
> >> >>> core,
> >> >>> > > >> Ignite
> >> >>> > > >> >> > > modules,
> >> >>> > > >> >> > > > >>> and
> >> >>> > > >> >> > > > >>> > > > > independent integrations. The first two
> >> are
> >> >>> to be
> >> >>> > > >> >> located
> >> >>> > > >> >> > in
> >> >>> > > >> >> > > > >>> Ignite
> >> >>> > > >> >> > > > >>> > > > > repositories and to be tested/updated by
> >> the
> >> >>> > > >> community
> >> >>> > > >> >> for
> >> >>> > > >> >> > > > every
> >> >>> > > >> >> > > > >>> > > release.
> >> >>> > > >> >> > > > >>> > > > > The modules are a subset of existing
> >> >>> integrations
> >> >>> > > >> that
> >> >>> > > >> >> are
> >> >>> > > >> >> > > > >>> adopted
> >> >>> > > >> >> > > > >>> > > widely
> >> >>> > > >> >> > > > >>> > > > > and require community attention.
> >> >>> > > >> >> > > > >>> > > > >
> >> >>> > > >> >> > > > >>> > > > > As for the independent integrations [1],
> >> >>> those
> >> >>> > are
> >> >>> > > >> >> supposed
> >> >>> > > >> >> > > to
> >> >>> > > >> >> > > > be
> >> >>> > > >> >> > > > >>> > > located
> >> >>> > > >> >> > > > >>> > > > > in a separate GitHub repo. I would
> advise
> >> us
> >> >>> to
> >> >>> > > start
> >> >>> > > >> >> > > > >>> implementing
> >> >>> > > >> >> > > > >>> > this
> >> >>> > > >> >> > > > >>> > > > > concept with the Pub/Sub integration.
> Even
> >> >>> though
> >> >>> > > it
> >> >>> > > >> >> will
> >> >>> > > >> >> > be
> >> >>> > > >> >> > > > >>> located
> >> >>> > > >> >> > > > >>> > > in a
> >> >>> > > >> >> > > > >>> > > > > separate Github repo, it will be listed
> >> among
> >> >>> > > >> officially
> >> >>> > > >> >> > > > >>> available
> >> >>> > > >> >> > > > >>> > > > > integrations on the Ignite website
> >> featuring
> >> >>> > > >> Emmanouil
> >> >>> > > >> >> as a
> >> >>> > > >> >> > > > >>> > > contributor.
> >> >>> > > >> >> > > > >>> > > > >
> >> >>> > > >> >> > > > >>> > > > > What do you think?
> >> >>> > > >> >> > > > >>> > > > >
> >> >>> > > >> >> > > > >>> > > > >
> >> >>> > > >> >> > > > >>> > > > > [1]
> >> >>> > > >> >> > > > >>> > > > >
> >> >>> > > >> >> > > > >>> > > > >
> >> >>> > > >> >> > > > >>> > > >
> >> >>> > > >> >> > > > >>> > >
> >> >>> > > >> >> > > > >>> >
> >> >>> > > >> >> > > > >>>
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > >
> >> >>> > > >> >> >
> >> >>> > > >> >>
> >> >>> > > >>
> >> >>> > >
> >> >>> >
> >> >>>
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >> >>> > > >> >> > > > >>> > > > >
> >> >>> > > >> >> > > > >>> > > > >
> >> >>> > > >> >> > > > >>> > > > >
> >> >>> > > >> >> > > > >>> > > > > -
> >> >>> > > >> >> > > > >>> > > > > Denis
> >> >>> > > >> >> > > > >>> > > > >
> >> >>> > > >> >> > > > >>> > > > >
> >> >>> > > >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM Saikat
> >> >>> Maitra <
> >> >>> > > >> >> > > > >>> > saikat.maitra@gmail.com
> >> >>> > > >> >> > > > >>> > > >
> >> >>> > > >> >> > > > >>> > > > > wrote:
> >> >>> > > >> >> > > > >>> > > > >
> >> >>> > > >> >> > > > >>> > > > > > Hi Emmanouil,
> >> >>> > > >> >> > > > >>> > > > > >
> >> >>> > > >> >> > > > >>> > > > > > The changes looks good to me. I wanted
> >> to
> >> >>> check
> >> >>> > > if
> >> >>> > > >> you
> >> >>> > > >> >> > are
> >> >>> > > >> >> > > > also
> >> >>> > > >> >> > > > >>> > able
> >> >>> > > >> >> > > > >>> > > to
> >> >>> > > >> >> > > > >>> > > > > > validate that the once data is added
> to
> >> >>> Ignite
> >> >>> > > >> cluster
> >> >>> > > >> >> > you
> >> >>> > > >> >> > > > are
> >> >>> > > >> >> > > > >>> also
> >> >>> > > >> >> > > > >>> > > > able
> >> >>> > > >> >> > > > >>> > > > > to
> >> >>> > > >> >> > > > >>> > > > > > access it using another Ignite client
> or
> >> >>> using
> >> >>> > > rest
> >> >>> > > >> >> > > > endpoints.
> >> >>> > > >> >> > > > >>> > > > > >
> >> >>> > > >> >> > > > >>> > > > > > We use Teamcity for CI process, the
> >> >>> details are
> >> >>> > > as
> >> >>> > > >> >> > follows
> >> >>> > > >> >> > > > >>> > > > > >
> >> >>> > > >> >> > > > >>> > > > > > Trigger validation of those test
> suites
> >> >>> that
> >> >>> > have
> >> >>> > > >> been
> >> >>> > > >> >> > > > >>> affected by
> >> >>> > > >> >> > > > >>> > > your
> >> >>> > > >> >> > > > >>> > > > > > changes on TeamCity <
> >> >>> > > http://ci.ignite.apache.org/>:
> >> >>> > > >> <
> >> >>> > > >> >> > > > >>> > > > > http://204.14.53.153/
> >> >>> > > >> >> > > > >>> > > > > > >
> >> >>> > > >> >> > > > >>> > > > > >
> >> >>> > > >> >> > > > >>> > > > > >    - Locate a test suite you have to
> >> check,
> >> >>> > press
> >> >>> > > >> >> button
> >> >>> > > >> >> > > > named
> >> >>> > > >> >> > > > >>> > "..."
> >> >>> > > >> >> > > > >>> > > > that
> >> >>> > > >> >> > > > >>> > > > > >    is located on the left of "Run"
> >> button.
> >> >>> "Run
> >> >>> > > >> custom
> >> >>> > > >> >> > > build"
> >> >>> > > >> >> > > > >>> > dialog
> >> >>> > > >> >> > > > >>> > > > will
> >> >>> > > >> >> > > > >>> > > > > >    appear;
> >> >>> > > >> >> > > > >>> > > > > >    - Go to "Changes" tab and choose
> >> >>> > > >> >> > > > >>> > "pull/<pull-request-number>/head"
> >> >>> > > >> >> > > > >>> > > > in
> >> >>> > > >> >> > > > >>> > > > > >    "Build branch" dropdown list;
> >> >>> > > >> >> > > > >>> > > > > >    - Press "Run build" button and
> >> monitor
> >> >>> tests
> >> >>> > > >> >> results.
> >> >>> > > >> >> > > > >>> > > > > >
> >> >>> > > >> >> > > > >>> > > > > >
> >> >>> > > >> >> > > > >>> > > > > >
> >> >>> > > >> >> > > > >>> > > > > > @Roman rshtykh@yahoo.com - can you
> also
> >> >>> please
> >> >>> > > >> take a
> >> >>> > > >> >> > > look?
> >> >>> > > >> >> > > > >>> > > > > >
> >> >>> > > >> >> > > > >>> > > > > > Regards,
> >> >>> > > >> >> > > > >>> > > > > > Saikat
> >> >>> > > >> >> > > > >>> > > > > >
> >> >>> > > >> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20 PM
> >> Emmanouil
> >> >>> > > >> Gkatziouras
> >> >>> > > >> >> <
> >> >>> > > >> >> > > > >>> > > > > > gkatzioura@gmail.com>
> >> >>> > > >> >> > > > >>> > > > > > wrote:
> >> >>> > > >> >> > > > >>> > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > Hi @Saikat Maitra
> >> >>> > > >> >> > > > >>> > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > I have my implementation on an
> Ignite
> >> >>> fork on
> >> >>> > > my
> >> >>> > > >> >> > personal
> >> >>> > > >> >> > > > >>> GitHub
> >> >>> > > >> >> > > > >>> > > > > account
> >> >>> > > >> >> > > > >>> > > > > > > [1]
> >> >>> > > >> >> > > > >>> > > > > > > It is based on the Kafka
> >> implementation
> >> >>> which
> >> >>> > > is
> >> >>> > > >> of
> >> >>> > > >> >> a
> >> >>> > > >> >> > > > polling
> >> >>> > > >> >> > > > >>> > > nature.
> >> >>> > > >> >> > > > >>> > > > > > > GCP does not provide a service like
> >> >>> Kafka.
> >> >>> > > >> Instead
> >> >>> > > >> >> it
> >> >>> > > >> >> > > > >>> provides us
> >> >>> > > >> >> > > > >>> > > > with
> >> >>> > > >> >> > > > >>> > > > > > > Pub/Sub which is just like Kafka.
> >> >>> > > >> >> > > > >>> > > > > > > Therefore using the Kafka streamer
> as
> >> a
> >> >>> > > guideline
> >> >>> > > >> >> made
> >> >>> > > >> >> > > more
> >> >>> > > >> >> > > > >>> sense
> >> >>> > > >> >> > > > >>> > > to
> >> >>> > > >> >> > > > >>> > > > > me.
> >> >>> > > >> >> > > > >>> > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > Apart from the implementation I have
> >> also
> >> >>> > > added a
> >> >>> > > >> >> unit
> >> >>> > > >> >> > > > test.
> >> >>> > > >> >> > > > >>> > > > > > > It definitely needs some polishing
> >> and a
> >> >>> > double
> >> >>> > > >> >> check
> >> >>> > > >> >> > on
> >> >>> > > >> >> > > my
> >> >>> > > >> >> > > > >>> side
> >> >>> > > >> >> > > > >>> > to
> >> >>> > > >> >> > > > >>> > > > > make
> >> >>> > > >> >> > > > >>> > > > > > > the review smooth.
> >> >>> > > >> >> > > > >>> > > > > > > For the tests a local Pub/Sub [2]
> >> server
> >> >>> has
> >> >>> > to
> >> >>> > > >> be
> >> >>> > > >> >> > setup
> >> >>> > > >> >> > > > >>> however
> >> >>> > > >> >> > > > >>> > I
> >> >>> > > >> >> > > > >>> > > am
> >> >>> > > >> >> > > > >>> > > > > not
> >> >>> > > >> >> > > > >>> > > > > > > sure which is the best way to do so
> >> with
> >> >>> > > regards
> >> >>> > > >> to
> >> >>> > > >> >> > your
> >> >>> > > >> >> > > > >>> CI/CD.
> >> >>> > > >> >> > > > >>> > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > [1]
> >> >>> > > >> >> > > > >>> > >
> >> >>> > > >> >> >
> >> >>> https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> >> >>> > > >> >> > > > >>> > > > > > > [2]
> >> >>> > > >> >> > > > >>>
> >> >>> > https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
> >> >>> > > >> >> > > > >>> > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > *Emmanouil Gkatziouras*
> >> >>> > > >> >> > > > >>> > > > > > > https://egkatzioura.com/ |
> >> >>> > > >> >> > > > >>> > > > > > >
> >> >>> > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> >>> > > >> >> > > > >>> > > > > > > https://github.com/gkatzioura
> >> >>> > > >> >> > > > >>> > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > On Sat, 12 Oct 2019 at 02:11, Saikat
> >> >>> Maitra <
> >> >>> > > >> >> > > > >>> > > saikat.maitra@gmail.com
> >> >>> > > >> >> > > > >>> > > > >
> >> >>> > > >> >> > > > >>> > > > > > > wrote:
> >> >>> > > >> >> > > > >>> > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > Hello Emmanouil,
> >> >>> > > >> >> > > > >>> > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > Can you please share more info
> about
> >> >>> the
> >> >>> > > >> >> > > implementation?
> >> >>> > > >> >> > > > >>> > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > We have similar implementation for
> >> data
> >> >>> > > >> streamers
> >> >>> > > >> >> > with
> >> >>> > > >> >> > > > >>> Kafka
> >> >>> > > >> >> > > > >>> > and
> >> >>> > > >> >> > > > >>> > > > > > RocketMQ
> >> >>> > > >> >> > > > >>> > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > >
> >> >>> > > >> >> > https://apacheignite-mix.readme.io/docs/kafka-streamer
> >> >>> > > >> >> > > > >>> > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > >
> >> >>> > > >> >> > > >
> >> >>> https://apacheignite-mix.readme.io/docs/rocketmq-streamer
> >> >>> > > >> >> > > > >>> > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > Please review and share if you
> have
> >> any
> >> >>> > > >> questions.
> >> >>> > > >> >> > > > >>> > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > Regards,
> >> >>> > > >> >> > > > >>> > > > > > > > Saikat
> >> >>> > > >> >> > > > >>> > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > On Fri, Oct 11, 2019 at 4:56 PM
> >> >>> Emmanouil
> >> >>> > > >> >> > Gkatziouras <
> >> >>> > > >> >> > > > >>> > > > > > > > gkatzioura@gmail.com>
> >> >>> > > >> >> > > > >>> > > > > > > > wrote:
> >> >>> > > >> >> > > > >>> > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > Hi all,
> >> >>> > > >> >> > > > >>> > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > By mentioning Pub/Sub I mean
> >> Google
> >> >>> > Cloud's
> >> >>> > > >> >> Pub/Sub
> >> >>> > > >> >> > > [1]
> >> >>> > > >> >> > > > >>> > > > > > > > > Yes it is indeed cloud specific.
> >> It
> >> >>> is
> >> >>> > the
> >> >>> > > >> >> > messaging
> >> >>> > > >> >> > > > >>> service
> >> >>> > > >> >> > > > >>> > > > > provided
> >> >>> > > >> >> > > > >>> > > > > > > by
> >> >>> > > >> >> > > > >>> > > > > > > > > GCP.
> >> >>> > > >> >> > > > >>> > > > > > > > > I have made the integration with
> >> it,
> >> >>> but
> >> >>> > I
> >> >>> > > >> >> haven't
> >> >>> > > >> >> > > > >>> published
> >> >>> > > >> >> > > > >>> > it
> >> >>> > > >> >> > > > >>> > > > yet
> >> >>> > > >> >> > > > >>> > > > > > (I
> >> >>> > > >> >> > > > >>> > > > > > > am
> >> >>> > > >> >> > > > >>> > > > > > > > > waiting for the
> discussion/ticket
> >> and
> >> >>> > > proceed
> >> >>> > > >> >> with
> >> >>> > > >> >> > > the
> >> >>> > > >> >> > > > >>> > > > > contribution).
> >> >>> > > >> >> > > > >>> > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > Kind regards,
> >> >>> > > >> >> > > > >>> > > > > > > > > Emmanouil
> >> >>> > > >> >> > > > >>> > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > [1]
> >> >>> > > >> >> https://cloud.google.com/pubsub/docs/overview
> >> >>> > > >> >> > > > >>> > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > *Emmanouil Gkatziouras*
> >> >>> > > >> >> > > > >>> > > > > > > > > https://egkatzioura.com/ |
> >> >>> > > >> >> > > > >>> > > > > > > > >
> >> >>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> >>> > > >> >> > > > >>> > > > > > > > > https://github.com/gkatzioura
> >> >>> > > >> >> > > > >>> > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > On Fri, 11 Oct 2019 at 21:45,
> >> Denis
> >> >>> > Magda <
> >> >>> > > >> >> > > > >>> dmagda@apache.org
> >> >>> > > >> >> > > > >>> > >
> >> >>> > > >> >> > > > >>> > > > > wrote:
> >> >>> > > >> >> > > > >>> > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > > Hello Emmanouil,
> >> >>> > > >> >> > > > >>> > > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > > Are you referring to any
> >> specific
> >> >>> > > >> >> implementation
> >> >>> > > >> >> > > > >>> tailored
> >> >>> > > >> >> > > > >>> > > for a
> >> >>> > > >> >> > > > >>> > > > > > > cloud?
> >> >>> > > >> >> > > > >>> > > > > > > > If
> >> >>> > > >> >> > > > >>> > > > > > > > > > I'm not mistaken you were
> >> >>> mentioning
> >> >>> > > >> AWS/GCE.
> >> >>> > > >> >> Is
> >> >>> > > >> >> > > > there
> >> >>> > > >> >> > > > >>> any
> >> >>> > > >> >> > > > >>> > > > Github
> >> >>> > > >> >> > > > >>> > > > > > > > source
> >> >>> > > >> >> > > > >>> > > > > > > > > > code?
> >> >>> > > >> >> > > > >>> > > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > > Here is a list of existing
> >> >>> integrations
> >> >>> > > >> with
> >> >>> > > >> >> > > > streaming
> >> >>> > > >> >> > > > >>> and
> >> >>> > > >> >> > > > >>> > > > > pub/sub
> >> >>> > > >> >> > > > >>> > > > > > > > > > frameworks:
> >> >>> > > >> >> > > > >>> > > > > > > > > >
> >> >>> > > >> >> https://apacheignite-mix.readme.io/docs/overview
> >> >>> > > >> >> > > > >>> > > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > > with the source code in Ignite
> >> >>> master:
> >> >>> > > >> >> > > > >>> > > > > > > > > >
> >> >>> > > >> >> > > https://github.com/apache/ignite/tree/master/modules
> >> >>> > > >> >> > > > >>> > > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > > Check out those resources to
> see
> >> >>> what
> >> >>> > are
> >> >>> > > >> the
> >> >>> > > >> >> > most
> >> >>> > > >> >> > > > >>> suitable
> >> >>> > > >> >> > > > >>> > > > > Ignite
> >> >>> > > >> >> > > > >>> > > > > > > APIs
> >> >>> > > >> >> > > > >>> > > > > > > > > for
> >> >>> > > >> >> > > > >>> > > > > > > > > > such integrations.
> >> >>> > > >> >> > > > >>> > > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > > @Saikat Maitra <
> >> >>> > saikat.maitra@gmail.com>
> >> >>> > > ,
> >> >>> > > >> >> > @Roman
> >> >>> > > >> >> > > <
> >> >>> > > >> >> > > > >>> > > > > > rshtykh@yahoo.com
> >> >>> > > >> >> > > > >>> > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > ,
> >> >>> > > >> >> > > > >>> > > > > > > > > as
> >> >>> > > >> >> > > > >>> > > > > > > > > > streaming technologies
> >> >>> > > >> >> contributors/maintainers,
> >> >>> > > >> >> > > > could
> >> >>> > > >> >> > > > >>> you
> >> >>> > > >> >> > > > >>> > > > please
> >> >>> > > >> >> > > > >>> > > > > > > join
> >> >>> > > >> >> > > > >>> > > > > > > > > this
> >> >>> > > >> >> > > > >>> > > > > > > > > > thread?
> >> >>> > > >> >> > > > >>> > > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > > -
> >> >>> > > >> >> > > > >>> > > > > > > > > > Denis
> >> >>> > > >> >> > > > >>> > > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > > On Thu, Oct 10, 2019 at 7:58
> AM
> >> >>> > Emmanouil
> >> >>> > > >> >> > > > Gkatziouras <
> >> >>> > > >> >> > > > >>> > > > > > > > > > gkatzioura@gmail.com>
> >> >>> > > >> >> > > > >>> > > > > > > > > > wrote:
> >> >>> > > >> >> > > > >>> > > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > > > Hello everyone,
> >> >>> > > >> >> > > > >>> > > > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > > > I started using Ignite
> lately.
> >> >>> Part
> >> >>> > of
> >> >>> > > my
> >> >>> > > >> >> work
> >> >>> > > >> >> > > > >>> involves
> >> >>> > > >> >> > > > >>> > > > > Pub/Sub,
> >> >>> > > >> >> > > > >>> > > > > > > > thus I
> >> >>> > > >> >> > > > >>> > > > > > > > > > > created a pub/sub streamer.
> >> >>> > > >> >> > > > >>> > > > > > > > > > > Pub/Sub is pretty close to
> >> Kafka.
> >> >>> > > >> >> > > > >>> > > > > > > > > > > Here's how the streamer
> works.
> >> >>> Every
> >> >>> > > node
> >> >>> > > >> >> > creates
> >> >>> > > >> >> > > > an
> >> >>> > > >> >> > > > >>> > > instance
> >> >>> > > >> >> > > > >>> > > > > of
> >> >>> > > >> >> > > > >>> > > > > > a
> >> >>> > > >> >> > > > >>> > > > > > > > > > Pub/Sub
> >> >>> > > >> >> > > > >>> > > > > > > > > > > client.
> >> >>> > > >> >> > > > >>> > > > > > > > > > > The clients subscribe to the
> >> same
> >> >>> > topic
> >> >>> > > >> >> using
> >> >>> > > >> >> > the
> >> >>> > > >> >> > > > >>> same
> >> >>> > > >> >> > > > >>> > > > > > subscription
> >> >>> > > >> >> > > > >>> > > > > > > > > name.
> >> >>> > > >> >> > > > >>> > > > > > > > > > > A message is being sent and
> >> one
> >> >>> of
> >> >>> > the
> >> >>> > > >> nodes
> >> >>> > > >> >> > will
> >> >>> > > >> >> > > > >>> receive
> >> >>> > > >> >> > > > >>> > > the
> >> >>> > > >> >> > > > >>> > > > > > > > message,
> >> >>> > > >> >> > > > >>> > > > > > > > > > > process it and then forward
> >> it to
> >> >>> > > Ignite
> >> >>> > > >> >> using
> >> >>> > > >> >> > > the
> >> >>> > > >> >> > > > >>> > > extractor
> >> >>> > > >> >> > > > >>> > > > > > > > provided.
> >> >>> > > >> >> > > > >>> > > > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > > > Any thoughts?
> >> >>> > > >> >> > > > >>> > > > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > > > *Emmanouil Gkatziouras*
> >> >>> > > >> >> > > > >>> > > > > > > > > > > https://egkatzioura.com/ |
> >> >>> > > >> >> > > > >>> > > > > > > > > > >
> >> >>> > > >> >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >> >>> > > >> >> > > > >>> > > > > > > > > > >
> https://github.com/gkatzioura
> >> >>> > > >> >> > > > >>> > > > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > > >
> >> >>> > > >> >> > > > >>> > > > > > >
> >> >>> > > >> >> > > > >>> > > > > >
> >> >>> > > >> >> > > > >>> > > > >
> >> >>> > > >> >> > > > >>> > > >
> >> >>> > > >> >> > > > >>> > >
> >> >>> > > >> >> > > > >>> >
> >> >>> > > >> >> > > > >>>
> >> >>> > > >> >> > > > >>
> >> >>> > > >> >> > > >
> >> >>> > > >> >> > >
> >> >>> > > >> >> >
> >> >>> > > >> >>
> >> >>> > > >> >
> >> >>> > > >>
> >> >>> > > >
> >> >>> > >
> >> >>> >
> >> >>>
> >> >>
> >>
> >
>

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Emmanouil Gkatziouras <gk...@gmail.com>]

Hi all!

If someone could give me read access to the Job of Ignite Extensions on
Team City will greatly help me (username gkatzioura).
I suppose the builds get triggered automatically on a pull request.

Kind regards,
Emmanouil

*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura


On Sun, 24 Nov 2019 at 21:11, Emmanouil Gkatziouras <gk...@gmail.com>
wrote:

> Hi Saikat!
>
> I just rebased with the flink branch. Unfortunately I do not have read
> access to the team city link you provided and thus evaluate the tests for
> my pull request.
> I guess it has to do with ignite extensions being new.
> My username is gkatzioura and email the one I am using on this email
>
> Thank you for your time.
>
>
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>
>
> On Sun, 24 Nov 2019 at 18:03, Saikat Maitra <sa...@gmail.com>
> wrote:
>
>> Hi Emmanouil,
>>
>> The latest build on teamcity has passed on Flink pull request.
>>
>>
>> https://ci.ignite.apache.org/viewLog.html?buildId=4788928&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
>>
>> you should be able to take the changes I made in my PR and run build on
>> your pull request.
>>
>> Please let me know if you have any questions.
>>
>> Regards,
>> Saikat
>>
>> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra <sa...@gmail.com>
>> wrote:
>>
>> > Hi Emmanouil,
>> >
>> > I have fixed the Flink module testsuites and have setup new project in
>> > teamcity.
>> >
>> >
>> >
>> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview
>> >
>> > I am looking into setting up the build.
>> >
>> > Regards,
>> > Saikat
>> >
>> > On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra <sa...@gmail.com>
>> > wrote:
>> >
>> >> Hi Emmanouil,
>> >>
>> >> Thank you for your email. Yes, the plan is once the Flink PR is merged
>> it
>> >> will provide the Licence, parent POMs etc and you can rebase from
>> master
>> >> branch and apply the changes on top of it.
>> >>
>> >> I am getting some issues with test failures in local with
>> >> GridTestProperties as the test.properties is not present in this new
>> >> project but should be available from dependencies.
>> >>
>> >> Once I address this issue, I will go ahead and merge the changes and
>> then
>> >> we can take it from there.
>> >>
>> >> Regards,
>> >> Saikat
>> >>
>> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras <
>> >> gkatzioura@gmail.com> wrote:
>> >>
>> >>> Hi all,
>> >>>
>> >>> I made my first pull request [1]. Since this project is brand new (no
>> >>> parent poms, licensing), I reckoned it was best to use Saikat's
>> branch on
>> >>> Flink.
>> >>> I suppose the Flink branch will be merged first. If not please give me
>> >>> guidelines on how I should proceed next.
>> >>>
>> >>> Kind regards
>> >>> Emmanouil
>> >>>
>> >>> [1] https://github.com/apache/ignite-extensions/pull/2
>> >>>
>> >>> *Emmanouil Gkatziouras*
>> >>> https://egkatzioura.com/ |
>> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
>> >>> https://github.com/gkatzioura
>> >>>
>> >>>
>> >>> On Fri, 22 Nov 2019 at 20:55, Denis Magda <dm...@apache.org> wrote:
>> >>>
>> >>> > Awesome, ping us whenever you're ready!
>> >>> >
>> >>> > -
>> >>> > Denis
>> >>> >
>> >>> >
>> >>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <
>> >>> > gkatzioura@gmail.com>
>> >>> > wrote:
>> >>> >
>> >>> > > Hi all!
>> >>> > >
>> >>> > > I am sorry for being late on that. I was trying to refactor the
>> test
>> >>> in
>> >>> > > order not to be in need of any external tools or spinning up a
>> >>> server.
>> >>> > > I did forked the new repo and indeed my changes there, so a pull
>> >>> request
>> >>> > is
>> >>> > > a matter of time!
>> >>> > >
>> >>> > > Kind regards
>> >>> > >
>> >>> > > *Emmanouil Gkatziouras*
>> >>> > > https://egkatzioura.com/ |
>> >>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> >>> > > https://github.com/gkatzioura
>> >>> > >
>> >>> > >
>> >>> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda <dm...@apache.org>
>> wrote:
>> >>> > >
>> >>> > > > Hi Emmanouil,
>> >>> > > >
>> >>> > > > Do you have any questions or need any support from the
>> community?
>> >>> > > >
>> >>> > > > -
>> >>> > > > Denis
>> >>> > > >
>> >>> > > >
>> >>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <
>> >>> saikat.maitra@gmail.com
>> >>> > >
>> >>> > > > wrote:
>> >>> > > >
>> >>> > > >> Hi Emmanouil,
>> >>> > > >>
>> >>> > > >> Can you please take a looks at dev utils, if this is something
>> >>> you are
>> >>> > > >> looking for
>> >>> > > >> https://github.com/apache/ignite/tree/master/modules/dev-utils
>> ?
>> >>> > > >>
>> >>> > > >> IMO, if you can release Pub/Sub server in maven and then use
>> it as
>> >>> > > >> dependency, that would be great.
>> >>> > > >>
>> >>> > > >> Regards,
>> >>> > > >> Saikat
>> >>> > > >>
>> >>> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <
>> >>> > saikat.maitra@gmail.com>
>> >>> > > >> wrote:
>> >>> > > >>
>> >>> > > >> > Hi Emmanouil,
>> >>> > > >> >
>> >>> > > >> > The master branch has been initialized. I have raised a PR to
>> >>> > migrate
>> >>> > > >> > Flink module to ignite-extensions.
>> >>> > > >> >
>> >>> > > >> > PR https://github.com/apache/ignite-extensions/pull/1
>> >>> > > >> > Jira : https://issues.apache.org/jira/browse/IGNITE-12356
>> >>> > > >> >
>> >>> > > >> > Regards,
>> >>> > > >> > Saikat
>> >>> > > >> >
>> >>> > > >> >
>> >>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
>> >>> > > >> > gkatzioura@gmail.com> wrote:
>> >>> > > >> >
>> >>> > > >> >> Hi all!
>> >>> > > >> >>
>> >>> > > >> >> This is an update. The integration is tested with mocks
>> based
>> >>> on
>> >>> > the
>> >>> > > >> >> original Ignite repo [1].
>> >>> > > >> >> I am currently waiting for the first commits on the new
>> >>> integration
>> >>> > > >> >> project
>> >>> > > >> >> to take place in order to create my pull request accordingly
>> >>> to the
>> >>> > > >> >> initial
>> >>> > > >> >> commits.
>> >>> > > >> >> In order to simulate the Pub/Sub server in a test I used a
>> test
>> >>> > > server
>> >>> > > >> >> implementation [2]. Unfortunately they don't have this util
>> >>> > uploaded
>> >>> > > >> in a
>> >>> > > >> >> maven repo, thus it needs to be hosted on a maven repo
>> (Apache
>> >>> > > License
>> >>> > > >> >> 2.0). Is there a repo for utilities that Ignite uses for the
>> >>> > > >> test/build?
>> >>> > > >> >> I have also created a ticket [3]
>> >>> > > >> >>
>> >>> > > >> >> Kind regards
>> >>> > > >> >>
>> >>> > > >> >> [1]
>> >>> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
>> >>> > > >> >> [2]
>> >>> https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
>> >>> > > >> >> [3] https://issues.apache.org/jira/browse/IGNITE-12262
>> >>> > > >> >>
>> >>> > > >> >> *Emmanouil Gkatziouras*
>> >>> > > >> >> https://egkatzioura.com/ |
>> >>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
>> >>> > > >> >> https://github.com/gkatzioura
>> >>> > > >> >>
>> >>> > > >> >>
>> >>> > > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <
>> >>> > saikat.maitra@gmail.com
>> >>> > > >
>> >>> > > >> >> wrote:
>> >>> > > >> >>
>> >>> > > >> >> > Hello Ilya,
>> >>> > > >> >> >
>> >>> > > >> >> > Thank you for your email. Yes, before we remove any
>> >>> integration
>> >>> > we
>> >>> > > >> will
>> >>> > > >> >> > move them to another repository and release.
>> >>> > > >> >> >
>> >>> > > >> >> > We are thinking we will start with Pub/Sub Streamer
>> >>> > implementation
>> >>> > > >> and
>> >>> > > >> >> > continue to move other integration as mentioned in the
>> >>> following
>> >>> > > doc.
>> >>> > > >> >> >
>> >>> > > >> >> >
>> >>> > > >> >> >
>> >>> > > >> >>
>> >>> > > >>
>> >>> > >
>> >>> >
>> >>>
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> >>> > > >> >> >
>> >>> > > >> >> > Regards,
>> >>> > > >> >> > Saikat
>> >>> > > >> >> >
>> >>> > > >> >> >
>> >>> > > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
>> >>> > > >> >> > ilya.kasnacheev@gmail.com>
>> >>> > > >> >> > wrote:
>> >>> > > >> >> >
>> >>> > > >> >> > > Hello!
>> >>> > > >> >> > >
>> >>> > > >> >> > > My take on this: we want to offload some of
>> integrations,
>> >>> but
>> >>> > > >> please
>> >>> > > >> >> > commit
>> >>> > > >> >> > > and release them to another project first, and only
>> then we
>> >>> > will
>> >>> > > >> >> decide
>> >>> > > >> >> > to
>> >>> > > >> >> > > remove them from our repository.
>> >>> > > >> >> > >
>> >>> > > >> >> > > Please also make sure they are available with same
>> artifact
>> >>> > name
>> >>> > > >> and
>> >>> > > >> >> API.
>> >>> > > >> >> > > If this has to be changed, let's wait for AI 3.0
>> >>> > > >> >> > >
>> >>> > > >> >> > > Regards,
>> >>> > > >> >> > > --
>> >>> > > >> >> > > Ilya Kasnacheev
>> >>> > > >> >> > >
>> >>> > > >> >> > >
>> >>> > > >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <
>> >>> > > >> saikat.maitra@gmail.com>:
>> >>> > > >> >> > >
>> >>> > > >> >> > > > Hello Emmanouil,
>> >>> > > >> >> > > >
>> >>> > > >> >> > > > As discussed earlier, I discussed with Apache Bahir
>> >>> community
>> >>> > > and
>> >>> > > >> >> they
>> >>> > > >> >> > > are
>> >>> > > >> >> > > > interested to have Apache Ignite extensions as part of
>> >>> Apache
>> >>> > > >> Bahir
>> >>> > > >> >> > > > project.
>> >>> > > >> >> > > >
>> >>> > > >> >> > > > Can you please share Pub/Sub streamer implementation
>> >>> details
>> >>> > > with
>> >>> > > >> >> > Apache
>> >>> > > >> >> > > > Bahir community and request for feedback.
>> >>> > > >> >> > > >
>> >>> > > >> >> > > > https://bahir.apache.org/
>> >>> > > >> >> > > >
>> >>> > > >> >> > > > I have also requested for contributor access in Jira
>> for
>> >>> > Apache
>> >>> > > >> >> Bahir
>> >>> > > >> >> > > > project so that I can create issues and assign to
>> myself.
>> >>> > > >> >> > > >
>> >>> > > >> >> > > > I can help with code reviews as well.
>> >>> > > >> >> > > >
>> >>> > > >> >> > > > Here is the email thread for reference
>> >>> > > >> >> > > >
>> >>> > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
>> >>> > > >> >> > > >
>> >>> > > >> >> > > > Regards,
>> >>> > > >> >> > > > Saikat
>> >>> > > >> >> > > >
>> >>> > > >> >> > > >
>> >>> > > >> >> > > >
>> >>> > > >> >> > > >
>> >>> > > >> >> > > >
>> >>> > > >> >> > > >
>> >>> > > >> >> > > >
>> >>> > > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <
>> >>> > > >> >> saikat.maitra@gmail.com
>> >>> > > >> >> > >
>> >>> > > >> >> > > > wrote:
>> >>> > > >> >> > > >
>> >>> > > >> >> > > > > Hello,
>> >>> > > >> >> > > > >
>> >>> > > >> >> > > > > As discussed I have created following discussion
>> >>> threads on
>> >>> > > our
>> >>> > > >> >> > > migration
>> >>> > > >> >> > > > > proposals for Apache Ignite extensions:
>> >>> > > >> >> > > > >
>> >>> > > >> >> > > > >
>> >>> > > >> >> > > > >
>> >>> > > >> >> > > >
>> >>> > > >> >> > >
>> >>> > > >> >> >
>> >>> > > >> >>
>> >>> > > >>
>> >>> > >
>> >>> >
>> >>>
>> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
>> >>> > > >> >> > > > >
>> >>> > > >> >> > > > >
>> >>> > > >>
>> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
>> >>> > > >> >> > > > >
>> >>> > > >> >> > > > > I will share update as I hear more information.
>> >>> > > >> >> > > > >
>> >>> > > >> >> > > > > Regards,
>> >>> > > >> >> > > > > Saikat
>> >>> > > >> >> > > > >
>> >>> > > >> >> > > > >
>> >>> > > >> >> > > > >
>> >>> > > >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra <
>> >>> > > >> >> > saikat.maitra@gmail.com
>> >>> > > >> >> > > >
>> >>> > > >> >> > > > > wrote:
>> >>> > > >> >> > > > >
>> >>> > > >> >> > > > >> Hello Denis,
>> >>> > > >> >> > > > >>
>> >>> > > >> >> > > > >> Sure, sounds good. I will create a separate
>> discussion
>> >>> > > thread
>> >>> > > >> to
>> >>> > > >> >> get
>> >>> > > >> >> > > > >> community feedback on whether we
>> >>> > > >> >> > > > >> should join the Bahir or kick-off "Ignite
>> Extensions"
>> >>> > > project.
>> >>> > > >> >> > > > >>
>> >>> > > >> >> > > > >> Regards,
>> >>> > > >> >> > > > >> Saikat
>> >>> > > >> >> > > > >>
>> >>> > > >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda <
>> >>> > > >> dmagda@apache.org>
>> >>> > > >> >> > > wrote:
>> >>> > > >> >> > > > >>
>> >>> > > >> >> > > > >>> Folks,
>> >>> > > >> >> > > > >>>
>> >>> > > >> >> > > > >>> The concept of Apache Bahir is precisely what we
>> >>> need!
>> >>> > > >> Saikat,
>> >>> > > >> >> > thanks
>> >>> > > >> >> > > > for
>> >>> > > >> >> > > > >>> doing the research. Why I believe the * idea*
>> fits us
>> >>> > well:
>> >>> > > >> >> > > > >>>
>> >>> > > >> >> > > > >>>    - All integrations can be stored in separate
>> >>> Github
>> >>> > > >> >> repositories
>> >>> > > >> >> > > and
>> >>> > > >> >> > > > >>>    have their dev lifecycles. We've not obliged to
>> >>> couple
>> >>> > > all
>> >>> > > >> >> the
>> >>> > > >> >> > > > >>> integrations
>> >>> > > >> >> > > > >>>    in a single repo. For instance, Spark can be
>> >>> located
>> >>> > in
>> >>> > > a
>> >>> > > >> >> > > dedicated
>> >>> > > >> >> > > > >>> repo
>> >>> > > >> >> > > > >>>    while streaming integrations might be in a
>> single
>> >>> one.
>> >>> > > >> It's
>> >>> > > >> >> up
>> >>> > > >> >> > to
>> >>> > > >> >> > > > us.
>> >>> > > >> >> > > > >>>    - All the repositories and integrations belong
>> to
>> >>> ASF.
>> >>> > > >> We're
>> >>> > > >> >> not
>> >>> > > >> >> > > > >>> dumping
>> >>> > > >> >> > > > >>>    them on Github but rather keep supporting and
>> >>> > developing
>> >>> > > >> in
>> >>> > > >> >> > > > >>> accordance with
>> >>> > > >> >> > > > >>>    ASF vision and practices.
>> >>> > > >> >> > > > >>>    - There is a way to reward contributors via
>> >>> > > committership
>> >>> > > >> and
>> >>> > > >> >> > PMC
>> >>> > > >> >> > > > >>>    membership. You won't get this if a project is
>> >>> just
>> >>> > one
>> >>> > > of
>> >>> > > >> >> the
>> >>> > > >> >> > > > >>> millions of
>> >>> > > >> >> > > > >>>    Github projects.
>> >>> > > >> >> > > > >>>
>> >>> > > >> >> > > > >>> Saikat, as Ignite PMC member, could you please
>> >>> kick-off a
>> >>> > > >> >> separate
>> >>> > > >> >> > > > >>> dev-list
>> >>> > > >> >> > > > >>> discussion to involve more community members and
>> >>> > referring
>> >>> > > to
>> >>> > > >> >> this
>> >>> > > >> >> > > > >>> thread?
>> >>> > > >> >> > > > >>> I think the primary question the community needs
>> to
>> >>> > answer
>> >>> > > >> >> whether
>> >>> > > >> >> > we
>> >>> > > >> >> > > > >>> should join the Bahir or kick-off "Ignite
>> Extensions"
>> >>> > > >> project?
>> >>> > > >> >> > > > >>>
>> >>> > > >> >> > > > >>> -
>> >>> > > >> >> > > > >>> Denis
>> >>> > > >> >> > > > >>>
>> >>> > > >> >> > > > >>>
>> >>> > > >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev <
>> >>> > > >> >> > > > zaleslaw.sin@gmail.com>
>> >>> > > >> >> > > > >>> wrote:
>> >>> > > >> >> > > > >>>
>> >>> > > >> >> > > > >>> > Maybe we could move all our Streaming
>> Integrations
>> >>> > there,
>> >>> > > >> but
>> >>> > > >> >> > what
>> >>> > > >> >> > > is
>> >>> > > >> >> > > > >>> about
>> >>> > > >> >> > > > >>> > maintaining and committer permissions to the new
>> >>> > > >> repositories?
>> >>> > > >> >> > > > >>> > I see the list of the committers and PMC members
>> >>> there
>> >>> > > >> >> > > > >>> > https://bahir.apache.org/community-members/
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>> > Could somebody from Ignite community be added to
>> >>> this
>> >>> > > list
>> >>> > > >> as
>> >>> > > >> >> a
>> >>> > > >> >> > > > >>> > PMC/committer to maintain Ignite integrations?
>> >>> > > >> >> > > > >>> > If yes, I'd happy to join this project and
>> >>> collaborate
>> >>> > > with
>> >>> > > >> >> these
>> >>> > > >> >> > > > guys
>> >>> > > >> >> > > > >>> > together
>> >>> > > >> >> > > > >>> > If no, and we should start from the zero level
>> with
>> >>> > > >> external
>> >>> > > >> >> PRs
>> >>> > > >> >> > -
>> >>> > > >> >> > > > >>> hmmm,
>> >>> > > >> >> > > > >>> > it's better to have our own external repository
>> >>> with
>> >>> > > >> >> ApacheBahirr
>> >>> > > >> >> > > > >>> ideology.
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>> > Also, I agree, that all connectors could be
>> moved
>> >>> there
>> >>> > > (in
>> >>> > > >> >> > > > >>> ApacheBahirr
>> >>> > > >> >> > > > >>> > like repository), but not all modules from the
>> page
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>>
>> >>> > > >> >> > > >
>> >>> > > >> >> > >
>> >>> > > >> >> >
>> >>> > > >> >>
>> >>> > > >>
>> >>> > >
>> >>> >
>> >>>
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> >>> > > >> >> > > > >>> > because
>> >>> > > >> >> > > > >>> > they use different parts of core modules and
>> could
>> >>> be
>> >>> > > >> >> developed
>> >>> > > >> >> > > with
>> >>> > > >> >> > > > >>> > different velocity.
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>> > In reality, before creation of new repositories
>> we
>> >>> need
>> >>> > > >> wide
>> >>> > > >> >> > > > discussion
>> >>> > > >> >> > > > >>> > based on the document mentioned above.
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>> > P.S. Of course, we could experiment in the next
>> >>> release
>> >>> > > 2.8
>> >>> > > >> >> with
>> >>> > > >> >> > > one
>> >>> > > >> >> > > > >>> or two
>> >>> > > >> >> > > > >>> > integrations like twitter or ZeroMQ
>> >>> > > >> >> > > > >>> > Also, I'd like Denis Magda idea for the separate
>> >>> > > >> repositories
>> >>> > > >> >> in
>> >>> > > >> >> > > > Apache
>> >>> > > >> >> > > > >>> > Github to maintain them by anybody.
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra <
>> >>> > > >> >> > > saikat.maitra@gmail.com
>> >>> > > >> >> > > > >:
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>> > > Hi Denis, Emmanouil
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> > > Thank you for your email. I think creating a
>> >>> separate
>> >>> > > >> >> > integration
>> >>> > > >> >> > > > >>> project
>> >>> > > >> >> > > > >>> > > in github and also proposing it as an apache
>> >>> > incubator
>> >>> > > >> >> project
>> >>> > > >> >> > > will
>> >>> > > >> >> > > > >>> be a
>> >>> > > >> >> > > > >>> > > good move. The other separate integration
>> modules
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>>
>> >>> > > >> >> > > >
>> >>> > > >> >> > >
>> >>> > > >> >> >
>> >>> > > >> >>
>> >>> > > >>
>> >>> > >
>> >>> >
>> >>>
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> >>> > > >> >> > > > >>> > > can
>> >>> > > >> >> > > > >>> > > be moved to this new apache incubator project.
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> > > There are similar integration available for
>> >>> Flink and
>> >>> > > >> Spark
>> >>> > > >> >> in
>> >>> > > >> >> > > > Apache
>> >>> > > >> >> > > > >>> > Bahir
>> >>> > > >> >> > > > >>> > > https://bahir.apache.org/
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> > > Do you think we should reach out to Apache
>> Bahir
>> >>> > > >> community
>> >>> > > >> >> > with a
>> >>> > > >> >> > > > >>> > proposal
>> >>> > > >> >> > > > >>> > > or we should plan to create a separate Apache
>> >>> > Incubator
>> >>> > > >> >> > project?
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> > > Regards,
>> >>> > > >> >> > > > >>> > > Saikat
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil
>> >>> > Gkatziouras <
>> >>> > > >> >> > > > >>> > > gkatzioura@gmail.com>
>> >>> > > >> >> > > > >>> > > wrote:
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> > > > Hi all!
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > > > Based on the discussions most probably this
>> is
>> >>> > going
>> >>> > > to
>> >>> > > >> >> be on
>> >>> > > >> >> > > > >>> another
>> >>> > > >> >> > > > >>> > > > GitHub repo. Therefore I will prepare a
>> >>> standalone
>> >>> > > >> project
>> >>> > > >> >> > with
>> >>> > > >> >> > > > the
>> >>> > > >> >> > > > >>> > > Pub/Sub
>> >>> > > >> >> > > > >>> > > > module and once the repository is created a
>> >>> pull
>> >>> > > >> request
>> >>> > > >> >> > shall
>> >>> > > >> >> > > be
>> >>> > > >> >> > > > >>> > issued.
>> >>> > > >> >> > > > >>> > > > If there is anything else I can do in order
>> to
>> >>> > > >> facilitate
>> >>> > > >> >> > this
>> >>> > > >> >> > > > >>> please
>> >>> > > >> >> > > > >>> > let
>> >>> > > >> >> > > > >>> > > > me know.
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > > > If a ticket is created for this issue, I
>> shall
>> >>> be
>> >>> > > able
>> >>> > > >> to
>> >>> > > >> >> > > create
>> >>> > > >> >> > > > a
>> >>> > > >> >> > > > >>> pull
>> >>> > > >> >> > > > >>> > > > request and thus a build will take place on
>> >>> > ignite's
>> >>> > > >> CI.
>> >>> > > >> >> > > > >>> > > > If everyone is aligned with adding a Pub/Sub
>> >>> > > >> >> implementation
>> >>> > > >> >> > > may I
>> >>> > > >> >> > > > >>> > create
>> >>> > > >> >> > > > >>> > > a
>> >>> > > >> >> > > > >>> > > > ticket on JIRA?
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > > > My next question has to do on how Ignite
>> >>> configures
>> >>> > > the
>> >>> > > >> >> build
>> >>> > > >> >> > > > jobs
>> >>> > > >> >> > > > >>> in
>> >>> > > >> >> > > > >>> > > Team
>> >>> > > >> >> > > > >>> > > > City. The unit tests for my implementation
>> >>> need a
>> >>> > > local
>> >>> > > >> >> > Pub/Sub
>> >>> > > >> >> > > > >>> server
>> >>> > > >> >> > > > >>> > up
>> >>> > > >> >> > > > >>> > > > and running (they provide one for testing
>> >>> > purposes).
>> >>> > > >> Will
>> >>> > > >> >> > this
>> >>> > > >> >> > > > take
>> >>> > > >> >> > > > >>> > > effect
>> >>> > > >> >> > > > >>> > > > in a form of a build script inside my
>> >>> > implementation
>> >>> > > or
>> >>> > > >> >> is it
>> >>> > > >> >> > > > >>> something
>> >>> > > >> >> > > > >>> > > > that should be configured on Team City?
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > > > Kind regards,
>> >>> > > >> >> > > > >>> > > > Emmanouil
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > > > *Emmanouil Gkatziouras*
>> >>> > > >> >> > > > >>> > > > https://egkatzioura.com/ |
>> >>> > > >> >> > > > >>> > > >
>> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
>> >>> > > >> >> > > > >>> > > > https://github.com/gkatzioura
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis Magda <
>> >>> > > >> >> dmagda@apache.org
>> >>> > > >> >> > >
>> >>> > > >> >> > > > >>> wrote:
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > > > > Emmanouil, Saikat,
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > > After contemplating on this, let me
>> propose
>> >>> > another
>> >>> > > >> way
>> >>> > > >> >> > with
>> >>> > > >> >> > > > this
>> >>> > > >> >> > > > >>> > > > > particular integration that is aligned
>> with
>> >>> our
>> >>> > > >> >> > > modularization
>> >>> > > >> >> > > > >>> > > endeavor.
>> >>> > > >> >> > > > >>> > > > > The modularization [1] defines the Ignite
>> >>> core,
>> >>> > > >> Ignite
>> >>> > > >> >> > > modules,
>> >>> > > >> >> > > > >>> and
>> >>> > > >> >> > > > >>> > > > > independent integrations. The first two
>> are
>> >>> to be
>> >>> > > >> >> located
>> >>> > > >> >> > in
>> >>> > > >> >> > > > >>> Ignite
>> >>> > > >> >> > > > >>> > > > > repositories and to be tested/updated by
>> the
>> >>> > > >> community
>> >>> > > >> >> for
>> >>> > > >> >> > > > every
>> >>> > > >> >> > > > >>> > > release.
>> >>> > > >> >> > > > >>> > > > > The modules are a subset of existing
>> >>> integrations
>> >>> > > >> that
>> >>> > > >> >> are
>> >>> > > >> >> > > > >>> adopted
>> >>> > > >> >> > > > >>> > > widely
>> >>> > > >> >> > > > >>> > > > > and require community attention.
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > > As for the independent integrations [1],
>> >>> those
>> >>> > are
>> >>> > > >> >> supposed
>> >>> > > >> >> > > to
>> >>> > > >> >> > > > be
>> >>> > > >> >> > > > >>> > > located
>> >>> > > >> >> > > > >>> > > > > in a separate GitHub repo. I would advise
>> us
>> >>> to
>> >>> > > start
>> >>> > > >> >> > > > >>> implementing
>> >>> > > >> >> > > > >>> > this
>> >>> > > >> >> > > > >>> > > > > concept with the Pub/Sub integration. Even
>> >>> though
>> >>> > > it
>> >>> > > >> >> will
>> >>> > > >> >> > be
>> >>> > > >> >> > > > >>> located
>> >>> > > >> >> > > > >>> > > in a
>> >>> > > >> >> > > > >>> > > > > separate Github repo, it will be listed
>> among
>> >>> > > >> officially
>> >>> > > >> >> > > > >>> available
>> >>> > > >> >> > > > >>> > > > > integrations on the Ignite website
>> featuring
>> >>> > > >> Emmanouil
>> >>> > > >> >> as a
>> >>> > > >> >> > > > >>> > > contributor.
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > > What do you think?
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > > [1]
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>>
>> >>> > > >> >> > > >
>> >>> > > >> >> > >
>> >>> > > >> >> >
>> >>> > > >> >>
>> >>> > > >>
>> >>> > >
>> >>> >
>> >>>
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > > -
>> >>> > > >> >> > > > >>> > > > > Denis
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM Saikat
>> >>> Maitra <
>> >>> > > >> >> > > > >>> > saikat.maitra@gmail.com
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > > > > wrote:
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > > > Hi Emmanouil,
>> >>> > > >> >> > > > >>> > > > > >
>> >>> > > >> >> > > > >>> > > > > > The changes looks good to me. I wanted
>> to
>> >>> check
>> >>> > > if
>> >>> > > >> you
>> >>> > > >> >> > are
>> >>> > > >> >> > > > also
>> >>> > > >> >> > > > >>> > able
>> >>> > > >> >> > > > >>> > > to
>> >>> > > >> >> > > > >>> > > > > > validate that the once data is added to
>> >>> Ignite
>> >>> > > >> cluster
>> >>> > > >> >> > you
>> >>> > > >> >> > > > are
>> >>> > > >> >> > > > >>> also
>> >>> > > >> >> > > > >>> > > > able
>> >>> > > >> >> > > > >>> > > > > to
>> >>> > > >> >> > > > >>> > > > > > access it using another Ignite client or
>> >>> using
>> >>> > > rest
>> >>> > > >> >> > > > endpoints.
>> >>> > > >> >> > > > >>> > > > > >
>> >>> > > >> >> > > > >>> > > > > > We use Teamcity for CI process, the
>> >>> details are
>> >>> > > as
>> >>> > > >> >> > follows
>> >>> > > >> >> > > > >>> > > > > >
>> >>> > > >> >> > > > >>> > > > > > Trigger validation of those test suites
>> >>> that
>> >>> > have
>> >>> > > >> been
>> >>> > > >> >> > > > >>> affected by
>> >>> > > >> >> > > > >>> > > your
>> >>> > > >> >> > > > >>> > > > > > changes on TeamCity <
>> >>> > > http://ci.ignite.apache.org/>:
>> >>> > > >> <
>> >>> > > >> >> > > > >>> > > > > http://204.14.53.153/
>> >>> > > >> >> > > > >>> > > > > > >
>> >>> > > >> >> > > > >>> > > > > >
>> >>> > > >> >> > > > >>> > > > > >    - Locate a test suite you have to
>> check,
>> >>> > press
>> >>> > > >> >> button
>> >>> > > >> >> > > > named
>> >>> > > >> >> > > > >>> > "..."
>> >>> > > >> >> > > > >>> > > > that
>> >>> > > >> >> > > > >>> > > > > >    is located on the left of "Run"
>> button.
>> >>> "Run
>> >>> > > >> custom
>> >>> > > >> >> > > build"
>> >>> > > >> >> > > > >>> > dialog
>> >>> > > >> >> > > > >>> > > > will
>> >>> > > >> >> > > > >>> > > > > >    appear;
>> >>> > > >> >> > > > >>> > > > > >    - Go to "Changes" tab and choose
>> >>> > > >> >> > > > >>> > "pull/<pull-request-number>/head"
>> >>> > > >> >> > > > >>> > > > in
>> >>> > > >> >> > > > >>> > > > > >    "Build branch" dropdown list;
>> >>> > > >> >> > > > >>> > > > > >    - Press "Run build" button and
>> monitor
>> >>> tests
>> >>> > > >> >> results.
>> >>> > > >> >> > > > >>> > > > > >
>> >>> > > >> >> > > > >>> > > > > >
>> >>> > > >> >> > > > >>> > > > > >
>> >>> > > >> >> > > > >>> > > > > > @Roman rshtykh@yahoo.com - can you also
>> >>> please
>> >>> > > >> take a
>> >>> > > >> >> > > look?
>> >>> > > >> >> > > > >>> > > > > >
>> >>> > > >> >> > > > >>> > > > > > Regards,
>> >>> > > >> >> > > > >>> > > > > > Saikat
>> >>> > > >> >> > > > >>> > > > > >
>> >>> > > >> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20 PM
>> Emmanouil
>> >>> > > >> Gkatziouras
>> >>> > > >> >> <
>> >>> > > >> >> > > > >>> > > > > > gkatzioura@gmail.com>
>> >>> > > >> >> > > > >>> > > > > > wrote:
>> >>> > > >> >> > > > >>> > > > > >
>> >>> > > >> >> > > > >>> > > > > > > Hi @Saikat Maitra
>> >>> > > >> >> > > > >>> > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > I have my implementation on an Ignite
>> >>> fork on
>> >>> > > my
>> >>> > > >> >> > personal
>> >>> > > >> >> > > > >>> GitHub
>> >>> > > >> >> > > > >>> > > > > account
>> >>> > > >> >> > > > >>> > > > > > > [1]
>> >>> > > >> >> > > > >>> > > > > > > It is based on the Kafka
>> implementation
>> >>> which
>> >>> > > is
>> >>> > > >> of
>> >>> > > >> >> a
>> >>> > > >> >> > > > polling
>> >>> > > >> >> > > > >>> > > nature.
>> >>> > > >> >> > > > >>> > > > > > > GCP does not provide a service like
>> >>> Kafka.
>> >>> > > >> Instead
>> >>> > > >> >> it
>> >>> > > >> >> > > > >>> provides us
>> >>> > > >> >> > > > >>> > > > with
>> >>> > > >> >> > > > >>> > > > > > > Pub/Sub which is just like Kafka.
>> >>> > > >> >> > > > >>> > > > > > > Therefore using the Kafka streamer as
>> a
>> >>> > > guideline
>> >>> > > >> >> made
>> >>> > > >> >> > > more
>> >>> > > >> >> > > > >>> sense
>> >>> > > >> >> > > > >>> > > to
>> >>> > > >> >> > > > >>> > > > > me.
>> >>> > > >> >> > > > >>> > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > Apart from the implementation I have
>> also
>> >>> > > added a
>> >>> > > >> >> unit
>> >>> > > >> >> > > > test.
>> >>> > > >> >> > > > >>> > > > > > > It definitely needs some polishing
>> and a
>> >>> > double
>> >>> > > >> >> check
>> >>> > > >> >> > on
>> >>> > > >> >> > > my
>> >>> > > >> >> > > > >>> side
>> >>> > > >> >> > > > >>> > to
>> >>> > > >> >> > > > >>> > > > > make
>> >>> > > >> >> > > > >>> > > > > > > the review smooth.
>> >>> > > >> >> > > > >>> > > > > > > For the tests a local Pub/Sub [2]
>> server
>> >>> has
>> >>> > to
>> >>> > > >> be
>> >>> > > >> >> > setup
>> >>> > > >> >> > > > >>> however
>> >>> > > >> >> > > > >>> > I
>> >>> > > >> >> > > > >>> > > am
>> >>> > > >> >> > > > >>> > > > > not
>> >>> > > >> >> > > > >>> > > > > > > sure which is the best way to do so
>> with
>> >>> > > regards
>> >>> > > >> to
>> >>> > > >> >> > your
>> >>> > > >> >> > > > >>> CI/CD.
>> >>> > > >> >> > > > >>> > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > [1]
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> >
>> >>> https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
>> >>> > > >> >> > > > >>> > > > > > > [2]
>> >>> > > >> >> > > > >>>
>> >>> > https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
>> >>> > > >> >> > > > >>> > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > *Emmanouil Gkatziouras*
>> >>> > > >> >> > > > >>> > > > > > > https://egkatzioura.com/ |
>> >>> > > >> >> > > > >>> > > > > > >
>> >>> > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
>> >>> > > >> >> > > > >>> > > > > > > https://github.com/gkatzioura
>> >>> > > >> >> > > > >>> > > > > > >
>> >>> > > >> >> > > > >>> > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > On Sat, 12 Oct 2019 at 02:11, Saikat
>> >>> Maitra <
>> >>> > > >> >> > > > >>> > > saikat.maitra@gmail.com
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > > > > > wrote:
>> >>> > > >> >> > > > >>> > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > Hello Emmanouil,
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > Can you please share more info about
>> >>> the
>> >>> > > >> >> > > implementation?
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > We have similar implementation for
>> data
>> >>> > > >> streamers
>> >>> > > >> >> > with
>> >>> > > >> >> > > > >>> Kafka
>> >>> > > >> >> > > > >>> > and
>> >>> > > >> >> > > > >>> > > > > > RocketMQ
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > https://apacheignite-mix.readme.io/docs/kafka-streamer
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > >
>> >>> https://apacheignite-mix.readme.io/docs/rocketmq-streamer
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > Please review and share if you have
>> any
>> >>> > > >> questions.
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > Regards,
>> >>> > > >> >> > > > >>> > > > > > > > Saikat
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > On Fri, Oct 11, 2019 at 4:56 PM
>> >>> Emmanouil
>> >>> > > >> >> > Gkatziouras <
>> >>> > > >> >> > > > >>> > > > > > > > gkatzioura@gmail.com>
>> >>> > > >> >> > > > >>> > > > > > > > wrote:
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > Hi all,
>> >>> > > >> >> > > > >>> > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > By mentioning Pub/Sub I mean
>> Google
>> >>> > Cloud's
>> >>> > > >> >> Pub/Sub
>> >>> > > >> >> > > [1]
>> >>> > > >> >> > > > >>> > > > > > > > > Yes it is indeed cloud specific.
>> It
>> >>> is
>> >>> > the
>> >>> > > >> >> > messaging
>> >>> > > >> >> > > > >>> service
>> >>> > > >> >> > > > >>> > > > > provided
>> >>> > > >> >> > > > >>> > > > > > > by
>> >>> > > >> >> > > > >>> > > > > > > > > GCP.
>> >>> > > >> >> > > > >>> > > > > > > > > I have made the integration with
>> it,
>> >>> but
>> >>> > I
>> >>> > > >> >> haven't
>> >>> > > >> >> > > > >>> published
>> >>> > > >> >> > > > >>> > it
>> >>> > > >> >> > > > >>> > > > yet
>> >>> > > >> >> > > > >>> > > > > > (I
>> >>> > > >> >> > > > >>> > > > > > > am
>> >>> > > >> >> > > > >>> > > > > > > > > waiting for the discussion/ticket
>> and
>> >>> > > proceed
>> >>> > > >> >> with
>> >>> > > >> >> > > the
>> >>> > > >> >> > > > >>> > > > > contribution).
>> >>> > > >> >> > > > >>> > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > Kind regards,
>> >>> > > >> >> > > > >>> > > > > > > > > Emmanouil
>> >>> > > >> >> > > > >>> > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > [1]
>> >>> > > >> >> https://cloud.google.com/pubsub/docs/overview
>> >>> > > >> >> > > > >>> > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > *Emmanouil Gkatziouras*
>> >>> > > >> >> > > > >>> > > > > > > > > https://egkatzioura.com/ |
>> >>> > > >> >> > > > >>> > > > > > > > >
>> >>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
>> >>> > > >> >> > > > >>> > > > > > > > > https://github.com/gkatzioura
>> >>> > > >> >> > > > >>> > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > On Fri, 11 Oct 2019 at 21:45,
>> Denis
>> >>> > Magda <
>> >>> > > >> >> > > > >>> dmagda@apache.org
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> > > > > wrote:
>> >>> > > >> >> > > > >>> > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > Hello Emmanouil,
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > Are you referring to any
>> specific
>> >>> > > >> >> implementation
>> >>> > > >> >> > > > >>> tailored
>> >>> > > >> >> > > > >>> > > for a
>> >>> > > >> >> > > > >>> > > > > > > cloud?
>> >>> > > >> >> > > > >>> > > > > > > > If
>> >>> > > >> >> > > > >>> > > > > > > > > > I'm not mistaken you were
>> >>> mentioning
>> >>> > > >> AWS/GCE.
>> >>> > > >> >> Is
>> >>> > > >> >> > > > there
>> >>> > > >> >> > > > >>> any
>> >>> > > >> >> > > > >>> > > > Github
>> >>> > > >> >> > > > >>> > > > > > > > source
>> >>> > > >> >> > > > >>> > > > > > > > > > code?
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > Here is a list of existing
>> >>> integrations
>> >>> > > >> with
>> >>> > > >> >> > > > streaming
>> >>> > > >> >> > > > >>> and
>> >>> > > >> >> > > > >>> > > > > pub/sub
>> >>> > > >> >> > > > >>> > > > > > > > > > frameworks:
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> https://apacheignite-mix.readme.io/docs/overview
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > with the source code in Ignite
>> >>> master:
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> > > https://github.com/apache/ignite/tree/master/modules
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > Check out those resources to see
>> >>> what
>> >>> > are
>> >>> > > >> the
>> >>> > > >> >> > most
>> >>> > > >> >> > > > >>> suitable
>> >>> > > >> >> > > > >>> > > > > Ignite
>> >>> > > >> >> > > > >>> > > > > > > APIs
>> >>> > > >> >> > > > >>> > > > > > > > > for
>> >>> > > >> >> > > > >>> > > > > > > > > > such integrations.
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > @Saikat Maitra <
>> >>> > saikat.maitra@gmail.com>
>> >>> > > ,
>> >>> > > >> >> > @Roman
>> >>> > > >> >> > > <
>> >>> > > >> >> > > > >>> > > > > > rshtykh@yahoo.com
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > ,
>> >>> > > >> >> > > > >>> > > > > > > > > as
>> >>> > > >> >> > > > >>> > > > > > > > > > streaming technologies
>> >>> > > >> >> contributors/maintainers,
>> >>> > > >> >> > > > could
>> >>> > > >> >> > > > >>> you
>> >>> > > >> >> > > > >>> > > > please
>> >>> > > >> >> > > > >>> > > > > > > join
>> >>> > > >> >> > > > >>> > > > > > > > > this
>> >>> > > >> >> > > > >>> > > > > > > > > > thread?
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > -
>> >>> > > >> >> > > > >>> > > > > > > > > > Denis
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > On Thu, Oct 10, 2019 at 7:58 AM
>> >>> > Emmanouil
>> >>> > > >> >> > > > Gkatziouras <
>> >>> > > >> >> > > > >>> > > > > > > > > > gkatzioura@gmail.com>
>> >>> > > >> >> > > > >>> > > > > > > > > > wrote:
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > > Hello everyone,
>> >>> > > >> >> > > > >>> > > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > > I started using Ignite lately.
>> >>> Part
>> >>> > of
>> >>> > > my
>> >>> > > >> >> work
>> >>> > > >> >> > > > >>> involves
>> >>> > > >> >> > > > >>> > > > > Pub/Sub,
>> >>> > > >> >> > > > >>> > > > > > > > thus I
>> >>> > > >> >> > > > >>> > > > > > > > > > > created a pub/sub streamer.
>> >>> > > >> >> > > > >>> > > > > > > > > > > Pub/Sub is pretty close to
>> Kafka.
>> >>> > > >> >> > > > >>> > > > > > > > > > > Here's how the streamer works.
>> >>> Every
>> >>> > > node
>> >>> > > >> >> > creates
>> >>> > > >> >> > > > an
>> >>> > > >> >> > > > >>> > > instance
>> >>> > > >> >> > > > >>> > > > > of
>> >>> > > >> >> > > > >>> > > > > > a
>> >>> > > >> >> > > > >>> > > > > > > > > > Pub/Sub
>> >>> > > >> >> > > > >>> > > > > > > > > > > client.
>> >>> > > >> >> > > > >>> > > > > > > > > > > The clients subscribe to the
>> same
>> >>> > topic
>> >>> > > >> >> using
>> >>> > > >> >> > the
>> >>> > > >> >> > > > >>> same
>> >>> > > >> >> > > > >>> > > > > > subscription
>> >>> > > >> >> > > > >>> > > > > > > > > name.
>> >>> > > >> >> > > > >>> > > > > > > > > > > A message is being sent and
>> one
>> >>> of
>> >>> > the
>> >>> > > >> nodes
>> >>> > > >> >> > will
>> >>> > > >> >> > > > >>> receive
>> >>> > > >> >> > > > >>> > > the
>> >>> > > >> >> > > > >>> > > > > > > > message,
>> >>> > > >> >> > > > >>> > > > > > > > > > > process it and then forward
>> it to
>> >>> > > Ignite
>> >>> > > >> >> using
>> >>> > > >> >> > > the
>> >>> > > >> >> > > > >>> > > extractor
>> >>> > > >> >> > > > >>> > > > > > > > provided.
>> >>> > > >> >> > > > >>> > > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > > Any thoughts?
>> >>> > > >> >> > > > >>> > > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > > > *Emmanouil Gkatziouras*
>> >>> > > >> >> > > > >>> > > > > > > > > > > https://egkatzioura.com/ |
>> >>> > > >> >> > > > >>> > > > > > > > > > >
>> >>> > > >> >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> >>> > > >> >> > > > >>> > > > > > > > > > > https://github.com/gkatzioura
>> >>> > > >> >> > > > >>> > > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > > >
>> >>> > > >> >> > > > >>> > > > > > >
>> >>> > > >> >> > > > >>> > > > > >
>> >>> > > >> >> > > > >>> > > > >
>> >>> > > >> >> > > > >>> > > >
>> >>> > > >> >> > > > >>> > >
>> >>> > > >> >> > > > >>> >
>> >>> > > >> >> > > > >>>
>> >>> > > >> >> > > > >>
>> >>> > > >> >> > > >
>> >>> > > >> >> > >
>> >>> > > >> >> >
>> >>> > > >> >>
>> >>> > > >> >
>> >>> > > >>
>> >>> > > >
>> >>> > >
>> >>> >
>> >>>
>> >>
>>
>

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Emmanouil Gkatziouras <gk...@gmail.com>]

Hi Saikat!

I just rebased with the flink branch. Unfortunately I do not have read
access to the team city link you provided and thus evaluate the tests for
my pull request.
I guess it has to do with ignite extensions being new.
My username is gkatzioura and email the one I am using on this email

Thank you for your time.


*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura


On Sun, 24 Nov 2019 at 18:03, Saikat Maitra <sa...@gmail.com> wrote:

> Hi Emmanouil,
>
> The latest build on teamcity has passed on Flink pull request.
>
>
> https://ci.ignite.apache.org/viewLog.html?buildId=4788928&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead
>
> you should be able to take the changes I made in my PR and run build on
> your pull request.
>
> Please let me know if you have any questions.
>
> Regards,
> Saikat
>
> On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra <sa...@gmail.com>
> wrote:
>
> > Hi Emmanouil,
> >
> > I have fixed the Flink module testsuites and have setup new project in
> > teamcity.
> >
> >
> >
> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview
> >
> > I am looking into setting up the build.
> >
> > Regards,
> > Saikat
> >
> > On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra <sa...@gmail.com>
> > wrote:
> >
> >> Hi Emmanouil,
> >>
> >> Thank you for your email. Yes, the plan is once the Flink PR is merged
> it
> >> will provide the Licence, parent POMs etc and you can rebase from master
> >> branch and apply the changes on top of it.
> >>
> >> I am getting some issues with test failures in local with
> >> GridTestProperties as the test.properties is not present in this new
> >> project but should be available from dependencies.
> >>
> >> Once I address this issue, I will go ahead and merge the changes and
> then
> >> we can take it from there.
> >>
> >> Regards,
> >> Saikat
> >>
> >> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras <
> >> gkatzioura@gmail.com> wrote:
> >>
> >>> Hi all,
> >>>
> >>> I made my first pull request [1]. Since this project is brand new (no
> >>> parent poms, licensing), I reckoned it was best to use Saikat's branch
> on
> >>> Flink.
> >>> I suppose the Flink branch will be merged first. If not please give me
> >>> guidelines on how I should proceed next.
> >>>
> >>> Kind regards
> >>> Emmanouil
> >>>
> >>> [1] https://github.com/apache/ignite-extensions/pull/2
> >>>
> >>> *Emmanouil Gkatziouras*
> >>> https://egkatzioura.com/ |
> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> >>> https://github.com/gkatzioura
> >>>
> >>>
> >>> On Fri, 22 Nov 2019 at 20:55, Denis Magda <dm...@apache.org> wrote:
> >>>
> >>> > Awesome, ping us whenever you're ready!
> >>> >
> >>> > -
> >>> > Denis
> >>> >
> >>> >
> >>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <
> >>> > gkatzioura@gmail.com>
> >>> > wrote:
> >>> >
> >>> > > Hi all!
> >>> > >
> >>> > > I am sorry for being late on that. I was trying to refactor the
> test
> >>> in
> >>> > > order not to be in need of any external tools or spinning up a
> >>> server.
> >>> > > I did forked the new repo and indeed my changes there, so a pull
> >>> request
> >>> > is
> >>> > > a matter of time!
> >>> > >
> >>> > > Kind regards
> >>> > >
> >>> > > *Emmanouil Gkatziouras*
> >>> > > https://egkatzioura.com/ |
> >>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >>> > > https://github.com/gkatzioura
> >>> > >
> >>> > >
> >>> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda <dm...@apache.org>
> wrote:
> >>> > >
> >>> > > > Hi Emmanouil,
> >>> > > >
> >>> > > > Do you have any questions or need any support from the community?
> >>> > > >
> >>> > > > -
> >>> > > > Denis
> >>> > > >
> >>> > > >
> >>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <
> >>> saikat.maitra@gmail.com
> >>> > >
> >>> > > > wrote:
> >>> > > >
> >>> > > >> Hi Emmanouil,
> >>> > > >>
> >>> > > >> Can you please take a looks at dev utils, if this is something
> >>> you are
> >>> > > >> looking for
> >>> > > >> https://github.com/apache/ignite/tree/master/modules/dev-utils?
> >>> > > >>
> >>> > > >> IMO, if you can release Pub/Sub server in maven and then use it
> as
> >>> > > >> dependency, that would be great.
> >>> > > >>
> >>> > > >> Regards,
> >>> > > >> Saikat
> >>> > > >>
> >>> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <
> >>> > saikat.maitra@gmail.com>
> >>> > > >> wrote:
> >>> > > >>
> >>> > > >> > Hi Emmanouil,
> >>> > > >> >
> >>> > > >> > The master branch has been initialized. I have raised a PR to
> >>> > migrate
> >>> > > >> > Flink module to ignite-extensions.
> >>> > > >> >
> >>> > > >> > PR https://github.com/apache/ignite-extensions/pull/1
> >>> > > >> > Jira : https://issues.apache.org/jira/browse/IGNITE-12356
> >>> > > >> >
> >>> > > >> > Regards,
> >>> > > >> > Saikat
> >>> > > >> >
> >>> > > >> >
> >>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
> >>> > > >> > gkatzioura@gmail.com> wrote:
> >>> > > >> >
> >>> > > >> >> Hi all!
> >>> > > >> >>
> >>> > > >> >> This is an update. The integration is tested with mocks based
> >>> on
> >>> > the
> >>> > > >> >> original Ignite repo [1].
> >>> > > >> >> I am currently waiting for the first commits on the new
> >>> integration
> >>> > > >> >> project
> >>> > > >> >> to take place in order to create my pull request accordingly
> >>> to the
> >>> > > >> >> initial
> >>> > > >> >> commits.
> >>> > > >> >> In order to simulate the Pub/Sub server in a test I used a
> test
> >>> > > server
> >>> > > >> >> implementation [2]. Unfortunately they don't have this util
> >>> > uploaded
> >>> > > >> in a
> >>> > > >> >> maven repo, thus it needs to be hosted on a maven repo
> (Apache
> >>> > > License
> >>> > > >> >> 2.0). Is there a repo for utilities that Ignite uses for the
> >>> > > >> test/build?
> >>> > > >> >> I have also created a ticket [3]
> >>> > > >> >>
> >>> > > >> >> Kind regards
> >>> > > >> >>
> >>> > > >> >> [1]
> >>> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> >>> > > >> >> [2]
> >>> https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> >>> > > >> >> [3] https://issues.apache.org/jira/browse/IGNITE-12262
> >>> > > >> >>
> >>> > > >> >> *Emmanouil Gkatziouras*
> >>> > > >> >> https://egkatzioura.com/ |
> >>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> >>> > > >> >> https://github.com/gkatzioura
> >>> > > >> >>
> >>> > > >> >>
> >>> > > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <
> >>> > saikat.maitra@gmail.com
> >>> > > >
> >>> > > >> >> wrote:
> >>> > > >> >>
> >>> > > >> >> > Hello Ilya,
> >>> > > >> >> >
> >>> > > >> >> > Thank you for your email. Yes, before we remove any
> >>> integration
> >>> > we
> >>> > > >> will
> >>> > > >> >> > move them to another repository and release.
> >>> > > >> >> >
> >>> > > >> >> > We are thinking we will start with Pub/Sub Streamer
> >>> > implementation
> >>> > > >> and
> >>> > > >> >> > continue to move other integration as mentioned in the
> >>> following
> >>> > > doc.
> >>> > > >> >> >
> >>> > > >> >> >
> >>> > > >> >> >
> >>> > > >> >>
> >>> > > >>
> >>> > >
> >>> >
> >>>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >>> > > >> >> >
> >>> > > >> >> > Regards,
> >>> > > >> >> > Saikat
> >>> > > >> >> >
> >>> > > >> >> >
> >>> > > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
> >>> > > >> >> > ilya.kasnacheev@gmail.com>
> >>> > > >> >> > wrote:
> >>> > > >> >> >
> >>> > > >> >> > > Hello!
> >>> > > >> >> > >
> >>> > > >> >> > > My take on this: we want to offload some of integrations,
> >>> but
> >>> > > >> please
> >>> > > >> >> > commit
> >>> > > >> >> > > and release them to another project first, and only then
> we
> >>> > will
> >>> > > >> >> decide
> >>> > > >> >> > to
> >>> > > >> >> > > remove them from our repository.
> >>> > > >> >> > >
> >>> > > >> >> > > Please also make sure they are available with same
> artifact
> >>> > name
> >>> > > >> and
> >>> > > >> >> API.
> >>> > > >> >> > > If this has to be changed, let's wait for AI 3.0
> >>> > > >> >> > >
> >>> > > >> >> > > Regards,
> >>> > > >> >> > > --
> >>> > > >> >> > > Ilya Kasnacheev
> >>> > > >> >> > >
> >>> > > >> >> > >
> >>> > > >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <
> >>> > > >> saikat.maitra@gmail.com>:
> >>> > > >> >> > >
> >>> > > >> >> > > > Hello Emmanouil,
> >>> > > >> >> > > >
> >>> > > >> >> > > > As discussed earlier, I discussed with Apache Bahir
> >>> community
> >>> > > and
> >>> > > >> >> they
> >>> > > >> >> > > are
> >>> > > >> >> > > > interested to have Apache Ignite extensions as part of
> >>> Apache
> >>> > > >> Bahir
> >>> > > >> >> > > > project.
> >>> > > >> >> > > >
> >>> > > >> >> > > > Can you please share Pub/Sub streamer implementation
> >>> details
> >>> > > with
> >>> > > >> >> > Apache
> >>> > > >> >> > > > Bahir community and request for feedback.
> >>> > > >> >> > > >
> >>> > > >> >> > > > https://bahir.apache.org/
> >>> > > >> >> > > >
> >>> > > >> >> > > > I have also requested for contributor access in Jira
> for
> >>> > Apache
> >>> > > >> >> Bahir
> >>> > > >> >> > > > project so that I can create issues and assign to
> myself.
> >>> > > >> >> > > >
> >>> > > >> >> > > > I can help with code reviews as well.
> >>> > > >> >> > > >
> >>> > > >> >> > > > Here is the email thread for reference
> >>> > > >> >> > > >
> >>> > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> >>> > > >> >> > > >
> >>> > > >> >> > > > Regards,
> >>> > > >> >> > > > Saikat
> >>> > > >> >> > > >
> >>> > > >> >> > > >
> >>> > > >> >> > > >
> >>> > > >> >> > > >
> >>> > > >> >> > > >
> >>> > > >> >> > > >
> >>> > > >> >> > > >
> >>> > > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <
> >>> > > >> >> saikat.maitra@gmail.com
> >>> > > >> >> > >
> >>> > > >> >> > > > wrote:
> >>> > > >> >> > > >
> >>> > > >> >> > > > > Hello,
> >>> > > >> >> > > > >
> >>> > > >> >> > > > > As discussed I have created following discussion
> >>> threads on
> >>> > > our
> >>> > > >> >> > > migration
> >>> > > >> >> > > > > proposals for Apache Ignite extensions:
> >>> > > >> >> > > > >
> >>> > > >> >> > > > >
> >>> > > >> >> > > > >
> >>> > > >> >> > > >
> >>> > > >> >> > >
> >>> > > >> >> >
> >>> > > >> >>
> >>> > > >>
> >>> > >
> >>> >
> >>>
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> >>> > > >> >> > > > >
> >>> > > >> >> > > > >
> >>> > > >> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> >>> > > >> >> > > > >
> >>> > > >> >> > > > > I will share update as I hear more information.
> >>> > > >> >> > > > >
> >>> > > >> >> > > > > Regards,
> >>> > > >> >> > > > > Saikat
> >>> > > >> >> > > > >
> >>> > > >> >> > > > >
> >>> > > >> >> > > > >
> >>> > > >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra <
> >>> > > >> >> > saikat.maitra@gmail.com
> >>> > > >> >> > > >
> >>> > > >> >> > > > > wrote:
> >>> > > >> >> > > > >
> >>> > > >> >> > > > >> Hello Denis,
> >>> > > >> >> > > > >>
> >>> > > >> >> > > > >> Sure, sounds good. I will create a separate
> discussion
> >>> > > thread
> >>> > > >> to
> >>> > > >> >> get
> >>> > > >> >> > > > >> community feedback on whether we
> >>> > > >> >> > > > >> should join the Bahir or kick-off "Ignite
> Extensions"
> >>> > > project.
> >>> > > >> >> > > > >>
> >>> > > >> >> > > > >> Regards,
> >>> > > >> >> > > > >> Saikat
> >>> > > >> >> > > > >>
> >>> > > >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda <
> >>> > > >> dmagda@apache.org>
> >>> > > >> >> > > wrote:
> >>> > > >> >> > > > >>
> >>> > > >> >> > > > >>> Folks,
> >>> > > >> >> > > > >>>
> >>> > > >> >> > > > >>> The concept of Apache Bahir is precisely what we
> >>> need!
> >>> > > >> Saikat,
> >>> > > >> >> > thanks
> >>> > > >> >> > > > for
> >>> > > >> >> > > > >>> doing the research. Why I believe the * idea* fits
> us
> >>> > well:
> >>> > > >> >> > > > >>>
> >>> > > >> >> > > > >>>    - All integrations can be stored in separate
> >>> Github
> >>> > > >> >> repositories
> >>> > > >> >> > > and
> >>> > > >> >> > > > >>>    have their dev lifecycles. We've not obliged to
> >>> couple
> >>> > > all
> >>> > > >> >> the
> >>> > > >> >> > > > >>> integrations
> >>> > > >> >> > > > >>>    in a single repo. For instance, Spark can be
> >>> located
> >>> > in
> >>> > > a
> >>> > > >> >> > > dedicated
> >>> > > >> >> > > > >>> repo
> >>> > > >> >> > > > >>>    while streaming integrations might be in a
> single
> >>> one.
> >>> > > >> It's
> >>> > > >> >> up
> >>> > > >> >> > to
> >>> > > >> >> > > > us.
> >>> > > >> >> > > > >>>    - All the repositories and integrations belong
> to
> >>> ASF.
> >>> > > >> We're
> >>> > > >> >> not
> >>> > > >> >> > > > >>> dumping
> >>> > > >> >> > > > >>>    them on Github but rather keep supporting and
> >>> > developing
> >>> > > >> in
> >>> > > >> >> > > > >>> accordance with
> >>> > > >> >> > > > >>>    ASF vision and practices.
> >>> > > >> >> > > > >>>    - There is a way to reward contributors via
> >>> > > committership
> >>> > > >> and
> >>> > > >> >> > PMC
> >>> > > >> >> > > > >>>    membership. You won't get this if a project is
> >>> just
> >>> > one
> >>> > > of
> >>> > > >> >> the
> >>> > > >> >> > > > >>> millions of
> >>> > > >> >> > > > >>>    Github projects.
> >>> > > >> >> > > > >>>
> >>> > > >> >> > > > >>> Saikat, as Ignite PMC member, could you please
> >>> kick-off a
> >>> > > >> >> separate
> >>> > > >> >> > > > >>> dev-list
> >>> > > >> >> > > > >>> discussion to involve more community members and
> >>> > referring
> >>> > > to
> >>> > > >> >> this
> >>> > > >> >> > > > >>> thread?
> >>> > > >> >> > > > >>> I think the primary question the community needs to
> >>> > answer
> >>> > > >> >> whether
> >>> > > >> >> > we
> >>> > > >> >> > > > >>> should join the Bahir or kick-off "Ignite
> Extensions"
> >>> > > >> project?
> >>> > > >> >> > > > >>>
> >>> > > >> >> > > > >>> -
> >>> > > >> >> > > > >>> Denis
> >>> > > >> >> > > > >>>
> >>> > > >> >> > > > >>>
> >>> > > >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev <
> >>> > > >> >> > > > zaleslaw.sin@gmail.com>
> >>> > > >> >> > > > >>> wrote:
> >>> > > >> >> > > > >>>
> >>> > > >> >> > > > >>> > Maybe we could move all our Streaming
> Integrations
> >>> > there,
> >>> > > >> but
> >>> > > >> >> > what
> >>> > > >> >> > > is
> >>> > > >> >> > > > >>> about
> >>> > > >> >> > > > >>> > maintaining and committer permissions to the new
> >>> > > >> repositories?
> >>> > > >> >> > > > >>> > I see the list of the committers and PMC members
> >>> there
> >>> > > >> >> > > > >>> > https://bahir.apache.org/community-members/
> >>> > > >> >> > > > >>> >
> >>> > > >> >> > > > >>> > Could somebody from Ignite community be added to
> >>> this
> >>> > > list
> >>> > > >> as
> >>> > > >> >> a
> >>> > > >> >> > > > >>> > PMC/committer to maintain Ignite integrations?
> >>> > > >> >> > > > >>> > If yes, I'd happy to join this project and
> >>> collaborate
> >>> > > with
> >>> > > >> >> these
> >>> > > >> >> > > > guys
> >>> > > >> >> > > > >>> > together
> >>> > > >> >> > > > >>> > If no, and we should start from the zero level
> with
> >>> > > >> external
> >>> > > >> >> PRs
> >>> > > >> >> > -
> >>> > > >> >> > > > >>> hmmm,
> >>> > > >> >> > > > >>> > it's better to have our own external repository
> >>> with
> >>> > > >> >> ApacheBahirr
> >>> > > >> >> > > > >>> ideology.
> >>> > > >> >> > > > >>> >
> >>> > > >> >> > > > >>> > Also, I agree, that all connectors could be moved
> >>> there
> >>> > > (in
> >>> > > >> >> > > > >>> ApacheBahirr
> >>> > > >> >> > > > >>> > like repository), but not all modules from the
> page
> >>> > > >> >> > > > >>> >
> >>> > > >> >> > > > >>> >
> >>> > > >> >> > > > >>>
> >>> > > >> >> > > >
> >>> > > >> >> > >
> >>> > > >> >> >
> >>> > > >> >>
> >>> > > >>
> >>> > >
> >>> >
> >>>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >>> > > >> >> > > > >>> > because
> >>> > > >> >> > > > >>> > they use different parts of core modules and
> could
> >>> be
> >>> > > >> >> developed
> >>> > > >> >> > > with
> >>> > > >> >> > > > >>> > different velocity.
> >>> > > >> >> > > > >>> >
> >>> > > >> >> > > > >>> > In reality, before creation of new repositories
> we
> >>> need
> >>> > > >> wide
> >>> > > >> >> > > > discussion
> >>> > > >> >> > > > >>> > based on the document mentioned above.
> >>> > > >> >> > > > >>> >
> >>> > > >> >> > > > >>> > P.S. Of course, we could experiment in the next
> >>> release
> >>> > > 2.8
> >>> > > >> >> with
> >>> > > >> >> > > one
> >>> > > >> >> > > > >>> or two
> >>> > > >> >> > > > >>> > integrations like twitter or ZeroMQ
> >>> > > >> >> > > > >>> > Also, I'd like Denis Magda idea for the separate
> >>> > > >> repositories
> >>> > > >> >> in
> >>> > > >> >> > > > Apache
> >>> > > >> >> > > > >>> > Github to maintain them by anybody.
> >>> > > >> >> > > > >>> >
> >>> > > >> >> > > > >>> >
> >>> > > >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra <
> >>> > > >> >> > > saikat.maitra@gmail.com
> >>> > > >> >> > > > >:
> >>> > > >> >> > > > >>> >
> >>> > > >> >> > > > >>> > > Hi Denis, Emmanouil
> >>> > > >> >> > > > >>> > >
> >>> > > >> >> > > > >>> > > Thank you for your email. I think creating a
> >>> separate
> >>> > > >> >> > integration
> >>> > > >> >> > > > >>> project
> >>> > > >> >> > > > >>> > > in github and also proposing it as an apache
> >>> > incubator
> >>> > > >> >> project
> >>> > > >> >> > > will
> >>> > > >> >> > > > >>> be a
> >>> > > >> >> > > > >>> > > good move. The other separate integration
> modules
> >>> > > >> >> > > > >>> > >
> >>> > > >> >> > > > >>> > >
> >>> > > >> >> > > > >>> >
> >>> > > >> >> > > > >>>
> >>> > > >> >> > > >
> >>> > > >> >> > >
> >>> > > >> >> >
> >>> > > >> >>
> >>> > > >>
> >>> > >
> >>> >
> >>>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >>> > > >> >> > > > >>> > > can
> >>> > > >> >> > > > >>> > > be moved to this new apache incubator project.
> >>> > > >> >> > > > >>> > >
> >>> > > >> >> > > > >>> > > There are similar integration available for
> >>> Flink and
> >>> > > >> Spark
> >>> > > >> >> in
> >>> > > >> >> > > > Apache
> >>> > > >> >> > > > >>> > Bahir
> >>> > > >> >> > > > >>> > > https://bahir.apache.org/
> >>> > > >> >> > > > >>> > >
> >>> > > >> >> > > > >>> > > Do you think we should reach out to Apache
> Bahir
> >>> > > >> community
> >>> > > >> >> > with a
> >>> > > >> >> > > > >>> > proposal
> >>> > > >> >> > > > >>> > > or we should plan to create a separate Apache
> >>> > Incubator
> >>> > > >> >> > project?
> >>> > > >> >> > > > >>> > >
> >>> > > >> >> > > > >>> > > Regards,
> >>> > > >> >> > > > >>> > > Saikat
> >>> > > >> >> > > > >>> > >
> >>> > > >> >> > > > >>> > >
> >>> > > >> >> > > > >>> > >
> >>> > > >> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil
> >>> > Gkatziouras <
> >>> > > >> >> > > > >>> > > gkatzioura@gmail.com>
> >>> > > >> >> > > > >>> > > wrote:
> >>> > > >> >> > > > >>> > >
> >>> > > >> >> > > > >>> > > > Hi all!
> >>> > > >> >> > > > >>> > > >
> >>> > > >> >> > > > >>> > > > Based on the discussions most probably this
> is
> >>> > going
> >>> > > to
> >>> > > >> >> be on
> >>> > > >> >> > > > >>> another
> >>> > > >> >> > > > >>> > > > GitHub repo. Therefore I will prepare a
> >>> standalone
> >>> > > >> project
> >>> > > >> >> > with
> >>> > > >> >> > > > the
> >>> > > >> >> > > > >>> > > Pub/Sub
> >>> > > >> >> > > > >>> > > > module and once the repository is created a
> >>> pull
> >>> > > >> request
> >>> > > >> >> > shall
> >>> > > >> >> > > be
> >>> > > >> >> > > > >>> > issued.
> >>> > > >> >> > > > >>> > > > If there is anything else I can do in order
> to
> >>> > > >> facilitate
> >>> > > >> >> > this
> >>> > > >> >> > > > >>> please
> >>> > > >> >> > > > >>> > let
> >>> > > >> >> > > > >>> > > > me know.
> >>> > > >> >> > > > >>> > > >
> >>> > > >> >> > > > >>> > > > If a ticket is created for this issue, I
> shall
> >>> be
> >>> > > able
> >>> > > >> to
> >>> > > >> >> > > create
> >>> > > >> >> > > > a
> >>> > > >> >> > > > >>> pull
> >>> > > >> >> > > > >>> > > > request and thus a build will take place on
> >>> > ignite's
> >>> > > >> CI.
> >>> > > >> >> > > > >>> > > > If everyone is aligned with adding a Pub/Sub
> >>> > > >> >> implementation
> >>> > > >> >> > > may I
> >>> > > >> >> > > > >>> > create
> >>> > > >> >> > > > >>> > > a
> >>> > > >> >> > > > >>> > > > ticket on JIRA?
> >>> > > >> >> > > > >>> > > >
> >>> > > >> >> > > > >>> > > > My next question has to do on how Ignite
> >>> configures
> >>> > > the
> >>> > > >> >> build
> >>> > > >> >> > > > jobs
> >>> > > >> >> > > > >>> in
> >>> > > >> >> > > > >>> > > Team
> >>> > > >> >> > > > >>> > > > City. The unit tests for my implementation
> >>> need a
> >>> > > local
> >>> > > >> >> > Pub/Sub
> >>> > > >> >> > > > >>> server
> >>> > > >> >> > > > >>> > up
> >>> > > >> >> > > > >>> > > > and running (they provide one for testing
> >>> > purposes).
> >>> > > >> Will
> >>> > > >> >> > this
> >>> > > >> >> > > > take
> >>> > > >> >> > > > >>> > > effect
> >>> > > >> >> > > > >>> > > > in a form of a build script inside my
> >>> > implementation
> >>> > > or
> >>> > > >> >> is it
> >>> > > >> >> > > > >>> something
> >>> > > >> >> > > > >>> > > > that should be configured on Team City?
> >>> > > >> >> > > > >>> > > >
> >>> > > >> >> > > > >>> > > > Kind regards,
> >>> > > >> >> > > > >>> > > > Emmanouil
> >>> > > >> >> > > > >>> > > >
> >>> > > >> >> > > > >>> > > >
> >>> > > >> >> > > > >>> > > > *Emmanouil Gkatziouras*
> >>> > > >> >> > > > >>> > > > https://egkatzioura.com/ |
> >>> > > >> >> > > > >>> > > >
> >>> https://www.linkedin.com/in/gkatziourasemmanouil/
> >>> > > >> >> > > > >>> > > > https://github.com/gkatzioura
> >>> > > >> >> > > > >>> > > >
> >>> > > >> >> > > > >>> > > >
> >>> > > >> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis Magda <
> >>> > > >> >> dmagda@apache.org
> >>> > > >> >> > >
> >>> > > >> >> > > > >>> wrote:
> >>> > > >> >> > > > >>> > > >
> >>> > > >> >> > > > >>> > > > > Emmanouil, Saikat,
> >>> > > >> >> > > > >>> > > > >
> >>> > > >> >> > > > >>> > > > > After contemplating on this, let me propose
> >>> > another
> >>> > > >> way
> >>> > > >> >> > with
> >>> > > >> >> > > > this
> >>> > > >> >> > > > >>> > > > > particular integration that is aligned with
> >>> our
> >>> > > >> >> > > modularization
> >>> > > >> >> > > > >>> > > endeavor.
> >>> > > >> >> > > > >>> > > > > The modularization [1] defines the Ignite
> >>> core,
> >>> > > >> Ignite
> >>> > > >> >> > > modules,
> >>> > > >> >> > > > >>> and
> >>> > > >> >> > > > >>> > > > > independent integrations. The first two are
> >>> to be
> >>> > > >> >> located
> >>> > > >> >> > in
> >>> > > >> >> > > > >>> Ignite
> >>> > > >> >> > > > >>> > > > > repositories and to be tested/updated by
> the
> >>> > > >> community
> >>> > > >> >> for
> >>> > > >> >> > > > every
> >>> > > >> >> > > > >>> > > release.
> >>> > > >> >> > > > >>> > > > > The modules are a subset of existing
> >>> integrations
> >>> > > >> that
> >>> > > >> >> are
> >>> > > >> >> > > > >>> adopted
> >>> > > >> >> > > > >>> > > widely
> >>> > > >> >> > > > >>> > > > > and require community attention.
> >>> > > >> >> > > > >>> > > > >
> >>> > > >> >> > > > >>> > > > > As for the independent integrations [1],
> >>> those
> >>> > are
> >>> > > >> >> supposed
> >>> > > >> >> > > to
> >>> > > >> >> > > > be
> >>> > > >> >> > > > >>> > > located
> >>> > > >> >> > > > >>> > > > > in a separate GitHub repo. I would advise
> us
> >>> to
> >>> > > start
> >>> > > >> >> > > > >>> implementing
> >>> > > >> >> > > > >>> > this
> >>> > > >> >> > > > >>> > > > > concept with the Pub/Sub integration. Even
> >>> though
> >>> > > it
> >>> > > >> >> will
> >>> > > >> >> > be
> >>> > > >> >> > > > >>> located
> >>> > > >> >> > > > >>> > > in a
> >>> > > >> >> > > > >>> > > > > separate Github repo, it will be listed
> among
> >>> > > >> officially
> >>> > > >> >> > > > >>> available
> >>> > > >> >> > > > >>> > > > > integrations on the Ignite website
> featuring
> >>> > > >> Emmanouil
> >>> > > >> >> as a
> >>> > > >> >> > > > >>> > > contributor.
> >>> > > >> >> > > > >>> > > > >
> >>> > > >> >> > > > >>> > > > > What do you think?
> >>> > > >> >> > > > >>> > > > >
> >>> > > >> >> > > > >>> > > > >
> >>> > > >> >> > > > >>> > > > > [1]
> >>> > > >> >> > > > >>> > > > >
> >>> > > >> >> > > > >>> > > > >
> >>> > > >> >> > > > >>> > > >
> >>> > > >> >> > > > >>> > >
> >>> > > >> >> > > > >>> >
> >>> > > >> >> > > > >>>
> >>> > > >> >> > > >
> >>> > > >> >> > >
> >>> > > >> >> >
> >>> > > >> >>
> >>> > > >>
> >>> > >
> >>> >
> >>>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >>> > > >> >> > > > >>> > > > >
> >>> > > >> >> > > > >>> > > > >
> >>> > > >> >> > > > >>> > > > >
> >>> > > >> >> > > > >>> > > > > -
> >>> > > >> >> > > > >>> > > > > Denis
> >>> > > >> >> > > > >>> > > > >
> >>> > > >> >> > > > >>> > > > >
> >>> > > >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM Saikat
> >>> Maitra <
> >>> > > >> >> > > > >>> > saikat.maitra@gmail.com
> >>> > > >> >> > > > >>> > > >
> >>> > > >> >> > > > >>> > > > > wrote:
> >>> > > >> >> > > > >>> > > > >
> >>> > > >> >> > > > >>> > > > > > Hi Emmanouil,
> >>> > > >> >> > > > >>> > > > > >
> >>> > > >> >> > > > >>> > > > > > The changes looks good to me. I wanted to
> >>> check
> >>> > > if
> >>> > > >> you
> >>> > > >> >> > are
> >>> > > >> >> > > > also
> >>> > > >> >> > > > >>> > able
> >>> > > >> >> > > > >>> > > to
> >>> > > >> >> > > > >>> > > > > > validate that the once data is added to
> >>> Ignite
> >>> > > >> cluster
> >>> > > >> >> > you
> >>> > > >> >> > > > are
> >>> > > >> >> > > > >>> also
> >>> > > >> >> > > > >>> > > > able
> >>> > > >> >> > > > >>> > > > > to
> >>> > > >> >> > > > >>> > > > > > access it using another Ignite client or
> >>> using
> >>> > > rest
> >>> > > >> >> > > > endpoints.
> >>> > > >> >> > > > >>> > > > > >
> >>> > > >> >> > > > >>> > > > > > We use Teamcity for CI process, the
> >>> details are
> >>> > > as
> >>> > > >> >> > follows
> >>> > > >> >> > > > >>> > > > > >
> >>> > > >> >> > > > >>> > > > > > Trigger validation of those test suites
> >>> that
> >>> > have
> >>> > > >> been
> >>> > > >> >> > > > >>> affected by
> >>> > > >> >> > > > >>> > > your
> >>> > > >> >> > > > >>> > > > > > changes on TeamCity <
> >>> > > http://ci.ignite.apache.org/>:
> >>> > > >> <
> >>> > > >> >> > > > >>> > > > > http://204.14.53.153/
> >>> > > >> >> > > > >>> > > > > > >
> >>> > > >> >> > > > >>> > > > > >
> >>> > > >> >> > > > >>> > > > > >    - Locate a test suite you have to
> check,
> >>> > press
> >>> > > >> >> button
> >>> > > >> >> > > > named
> >>> > > >> >> > > > >>> > "..."
> >>> > > >> >> > > > >>> > > > that
> >>> > > >> >> > > > >>> > > > > >    is located on the left of "Run"
> button.
> >>> "Run
> >>> > > >> custom
> >>> > > >> >> > > build"
> >>> > > >> >> > > > >>> > dialog
> >>> > > >> >> > > > >>> > > > will
> >>> > > >> >> > > > >>> > > > > >    appear;
> >>> > > >> >> > > > >>> > > > > >    - Go to "Changes" tab and choose
> >>> > > >> >> > > > >>> > "pull/<pull-request-number>/head"
> >>> > > >> >> > > > >>> > > > in
> >>> > > >> >> > > > >>> > > > > >    "Build branch" dropdown list;
> >>> > > >> >> > > > >>> > > > > >    - Press "Run build" button and monitor
> >>> tests
> >>> > > >> >> results.
> >>> > > >> >> > > > >>> > > > > >
> >>> > > >> >> > > > >>> > > > > >
> >>> > > >> >> > > > >>> > > > > >
> >>> > > >> >> > > > >>> > > > > > @Roman rshtykh@yahoo.com - can you also
> >>> please
> >>> > > >> take a
> >>> > > >> >> > > look?
> >>> > > >> >> > > > >>> > > > > >
> >>> > > >> >> > > > >>> > > > > > Regards,
> >>> > > >> >> > > > >>> > > > > > Saikat
> >>> > > >> >> > > > >>> > > > > >
> >>> > > >> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20 PM Emmanouil
> >>> > > >> Gkatziouras
> >>> > > >> >> <
> >>> > > >> >> > > > >>> > > > > > gkatzioura@gmail.com>
> >>> > > >> >> > > > >>> > > > > > wrote:
> >>> > > >> >> > > > >>> > > > > >
> >>> > > >> >> > > > >>> > > > > > > Hi @Saikat Maitra
> >>> > > >> >> > > > >>> > > > > > >
> >>> > > >> >> > > > >>> > > > > > > I have my implementation on an Ignite
> >>> fork on
> >>> > > my
> >>> > > >> >> > personal
> >>> > > >> >> > > > >>> GitHub
> >>> > > >> >> > > > >>> > > > > account
> >>> > > >> >> > > > >>> > > > > > > [1]
> >>> > > >> >> > > > >>> > > > > > > It is based on the Kafka implementation
> >>> which
> >>> > > is
> >>> > > >> of
> >>> > > >> >> a
> >>> > > >> >> > > > polling
> >>> > > >> >> > > > >>> > > nature.
> >>> > > >> >> > > > >>> > > > > > > GCP does not provide a service like
> >>> Kafka.
> >>> > > >> Instead
> >>> > > >> >> it
> >>> > > >> >> > > > >>> provides us
> >>> > > >> >> > > > >>> > > > with
> >>> > > >> >> > > > >>> > > > > > > Pub/Sub which is just like Kafka.
> >>> > > >> >> > > > >>> > > > > > > Therefore using the Kafka streamer as a
> >>> > > guideline
> >>> > > >> >> made
> >>> > > >> >> > > more
> >>> > > >> >> > > > >>> sense
> >>> > > >> >> > > > >>> > > to
> >>> > > >> >> > > > >>> > > > > me.
> >>> > > >> >> > > > >>> > > > > > >
> >>> > > >> >> > > > >>> > > > > > > Apart from the implementation I have
> also
> >>> > > added a
> >>> > > >> >> unit
> >>> > > >> >> > > > test.
> >>> > > >> >> > > > >>> > > > > > > It definitely needs some polishing and
> a
> >>> > double
> >>> > > >> >> check
> >>> > > >> >> > on
> >>> > > >> >> > > my
> >>> > > >> >> > > > >>> side
> >>> > > >> >> > > > >>> > to
> >>> > > >> >> > > > >>> > > > > make
> >>> > > >> >> > > > >>> > > > > > > the review smooth.
> >>> > > >> >> > > > >>> > > > > > > For the tests a local Pub/Sub [2]
> server
> >>> has
> >>> > to
> >>> > > >> be
> >>> > > >> >> > setup
> >>> > > >> >> > > > >>> however
> >>> > > >> >> > > > >>> > I
> >>> > > >> >> > > > >>> > > am
> >>> > > >> >> > > > >>> > > > > not
> >>> > > >> >> > > > >>> > > > > > > sure which is the best way to do so
> with
> >>> > > regards
> >>> > > >> to
> >>> > > >> >> > your
> >>> > > >> >> > > > >>> CI/CD.
> >>> > > >> >> > > > >>> > > > > > >
> >>> > > >> >> > > > >>> > > > > > > [1]
> >>> > > >> >> > > > >>> > >
> >>> > > >> >> >
> >>> https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> >>> > > >> >> > > > >>> > > > > > > [2]
> >>> > > >> >> > > > >>>
> >>> > https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
> >>> > > >> >> > > > >>> > > > > > >
> >>> > > >> >> > > > >>> > > > > > > *Emmanouil Gkatziouras*
> >>> > > >> >> > > > >>> > > > > > > https://egkatzioura.com/ |
> >>> > > >> >> > > > >>> > > > > > >
> >>> > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> >>> > > >> >> > > > >>> > > > > > > https://github.com/gkatzioura
> >>> > > >> >> > > > >>> > > > > > >
> >>> > > >> >> > > > >>> > > > > > >
> >>> > > >> >> > > > >>> > > > > > > On Sat, 12 Oct 2019 at 02:11, Saikat
> >>> Maitra <
> >>> > > >> >> > > > >>> > > saikat.maitra@gmail.com
> >>> > > >> >> > > > >>> > > > >
> >>> > > >> >> > > > >>> > > > > > > wrote:
> >>> > > >> >> > > > >>> > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > Hello Emmanouil,
> >>> > > >> >> > > > >>> > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > Can you please share more info about
> >>> the
> >>> > > >> >> > > implementation?
> >>> > > >> >> > > > >>> > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > We have similar implementation for
> data
> >>> > > >> streamers
> >>> > > >> >> > with
> >>> > > >> >> > > > >>> Kafka
> >>> > > >> >> > > > >>> > and
> >>> > > >> >> > > > >>> > > > > > RocketMQ
> >>> > > >> >> > > > >>> > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > >
> >>> > > >> >> > https://apacheignite-mix.readme.io/docs/kafka-streamer
> >>> > > >> >> > > > >>> > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > >
> >>> > > >> >> > > >
> >>> https://apacheignite-mix.readme.io/docs/rocketmq-streamer
> >>> > > >> >> > > > >>> > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > Please review and share if you have
> any
> >>> > > >> questions.
> >>> > > >> >> > > > >>> > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > Regards,
> >>> > > >> >> > > > >>> > > > > > > > Saikat
> >>> > > >> >> > > > >>> > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > On Fri, Oct 11, 2019 at 4:56 PM
> >>> Emmanouil
> >>> > > >> >> > Gkatziouras <
> >>> > > >> >> > > > >>> > > > > > > > gkatzioura@gmail.com>
> >>> > > >> >> > > > >>> > > > > > > > wrote:
> >>> > > >> >> > > > >>> > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > Hi all,
> >>> > > >> >> > > > >>> > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > By mentioning Pub/Sub I mean Google
> >>> > Cloud's
> >>> > > >> >> Pub/Sub
> >>> > > >> >> > > [1]
> >>> > > >> >> > > > >>> > > > > > > > > Yes it is indeed cloud specific. It
> >>> is
> >>> > the
> >>> > > >> >> > messaging
> >>> > > >> >> > > > >>> service
> >>> > > >> >> > > > >>> > > > > provided
> >>> > > >> >> > > > >>> > > > > > > by
> >>> > > >> >> > > > >>> > > > > > > > > GCP.
> >>> > > >> >> > > > >>> > > > > > > > > I have made the integration with
> it,
> >>> but
> >>> > I
> >>> > > >> >> haven't
> >>> > > >> >> > > > >>> published
> >>> > > >> >> > > > >>> > it
> >>> > > >> >> > > > >>> > > > yet
> >>> > > >> >> > > > >>> > > > > > (I
> >>> > > >> >> > > > >>> > > > > > > am
> >>> > > >> >> > > > >>> > > > > > > > > waiting for the discussion/ticket
> and
> >>> > > proceed
> >>> > > >> >> with
> >>> > > >> >> > > the
> >>> > > >> >> > > > >>> > > > > contribution).
> >>> > > >> >> > > > >>> > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > Kind regards,
> >>> > > >> >> > > > >>> > > > > > > > > Emmanouil
> >>> > > >> >> > > > >>> > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > [1]
> >>> > > >> >> https://cloud.google.com/pubsub/docs/overview
> >>> > > >> >> > > > >>> > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > *Emmanouil Gkatziouras*
> >>> > > >> >> > > > >>> > > > > > > > > https://egkatzioura.com/ |
> >>> > > >> >> > > > >>> > > > > > > > >
> >>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> >>> > > >> >> > > > >>> > > > > > > > > https://github.com/gkatzioura
> >>> > > >> >> > > > >>> > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > On Fri, 11 Oct 2019 at 21:45, Denis
> >>> > Magda <
> >>> > > >> >> > > > >>> dmagda@apache.org
> >>> > > >> >> > > > >>> > >
> >>> > > >> >> > > > >>> > > > > wrote:
> >>> > > >> >> > > > >>> > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > > Hello Emmanouil,
> >>> > > >> >> > > > >>> > > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > > Are you referring to any specific
> >>> > > >> >> implementation
> >>> > > >> >> > > > >>> tailored
> >>> > > >> >> > > > >>> > > for a
> >>> > > >> >> > > > >>> > > > > > > cloud?
> >>> > > >> >> > > > >>> > > > > > > > If
> >>> > > >> >> > > > >>> > > > > > > > > > I'm not mistaken you were
> >>> mentioning
> >>> > > >> AWS/GCE.
> >>> > > >> >> Is
> >>> > > >> >> > > > there
> >>> > > >> >> > > > >>> any
> >>> > > >> >> > > > >>> > > > Github
> >>> > > >> >> > > > >>> > > > > > > > source
> >>> > > >> >> > > > >>> > > > > > > > > > code?
> >>> > > >> >> > > > >>> > > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > > Here is a list of existing
> >>> integrations
> >>> > > >> with
> >>> > > >> >> > > > streaming
> >>> > > >> >> > > > >>> and
> >>> > > >> >> > > > >>> > > > > pub/sub
> >>> > > >> >> > > > >>> > > > > > > > > > frameworks:
> >>> > > >> >> > > > >>> > > > > > > > > >
> >>> > > >> >> https://apacheignite-mix.readme.io/docs/overview
> >>> > > >> >> > > > >>> > > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > > with the source code in Ignite
> >>> master:
> >>> > > >> >> > > > >>> > > > > > > > > >
> >>> > > >> >> > > https://github.com/apache/ignite/tree/master/modules
> >>> > > >> >> > > > >>> > > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > > Check out those resources to see
> >>> what
> >>> > are
> >>> > > >> the
> >>> > > >> >> > most
> >>> > > >> >> > > > >>> suitable
> >>> > > >> >> > > > >>> > > > > Ignite
> >>> > > >> >> > > > >>> > > > > > > APIs
> >>> > > >> >> > > > >>> > > > > > > > > for
> >>> > > >> >> > > > >>> > > > > > > > > > such integrations.
> >>> > > >> >> > > > >>> > > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > > @Saikat Maitra <
> >>> > saikat.maitra@gmail.com>
> >>> > > ,
> >>> > > >> >> > @Roman
> >>> > > >> >> > > <
> >>> > > >> >> > > > >>> > > > > > rshtykh@yahoo.com
> >>> > > >> >> > > > >>> > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > ,
> >>> > > >> >> > > > >>> > > > > > > > > as
> >>> > > >> >> > > > >>> > > > > > > > > > streaming technologies
> >>> > > >> >> contributors/maintainers,
> >>> > > >> >> > > > could
> >>> > > >> >> > > > >>> you
> >>> > > >> >> > > > >>> > > > please
> >>> > > >> >> > > > >>> > > > > > > join
> >>> > > >> >> > > > >>> > > > > > > > > this
> >>> > > >> >> > > > >>> > > > > > > > > > thread?
> >>> > > >> >> > > > >>> > > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > > -
> >>> > > >> >> > > > >>> > > > > > > > > > Denis
> >>> > > >> >> > > > >>> > > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > > On Thu, Oct 10, 2019 at 7:58 AM
> >>> > Emmanouil
> >>> > > >> >> > > > Gkatziouras <
> >>> > > >> >> > > > >>> > > > > > > > > > gkatzioura@gmail.com>
> >>> > > >> >> > > > >>> > > > > > > > > > wrote:
> >>> > > >> >> > > > >>> > > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > > > Hello everyone,
> >>> > > >> >> > > > >>> > > > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > > > I started using Ignite lately.
> >>> Part
> >>> > of
> >>> > > my
> >>> > > >> >> work
> >>> > > >> >> > > > >>> involves
> >>> > > >> >> > > > >>> > > > > Pub/Sub,
> >>> > > >> >> > > > >>> > > > > > > > thus I
> >>> > > >> >> > > > >>> > > > > > > > > > > created a pub/sub streamer.
> >>> > > >> >> > > > >>> > > > > > > > > > > Pub/Sub is pretty close to
> Kafka.
> >>> > > >> >> > > > >>> > > > > > > > > > > Here's how the streamer works.
> >>> Every
> >>> > > node
> >>> > > >> >> > creates
> >>> > > >> >> > > > an
> >>> > > >> >> > > > >>> > > instance
> >>> > > >> >> > > > >>> > > > > of
> >>> > > >> >> > > > >>> > > > > > a
> >>> > > >> >> > > > >>> > > > > > > > > > Pub/Sub
> >>> > > >> >> > > > >>> > > > > > > > > > > client.
> >>> > > >> >> > > > >>> > > > > > > > > > > The clients subscribe to the
> same
> >>> > topic
> >>> > > >> >> using
> >>> > > >> >> > the
> >>> > > >> >> > > > >>> same
> >>> > > >> >> > > > >>> > > > > > subscription
> >>> > > >> >> > > > >>> > > > > > > > > name.
> >>> > > >> >> > > > >>> > > > > > > > > > > A message is being sent and one
> >>> of
> >>> > the
> >>> > > >> nodes
> >>> > > >> >> > will
> >>> > > >> >> > > > >>> receive
> >>> > > >> >> > > > >>> > > the
> >>> > > >> >> > > > >>> > > > > > > > message,
> >>> > > >> >> > > > >>> > > > > > > > > > > process it and then forward it
> to
> >>> > > Ignite
> >>> > > >> >> using
> >>> > > >> >> > > the
> >>> > > >> >> > > > >>> > > extractor
> >>> > > >> >> > > > >>> > > > > > > > provided.
> >>> > > >> >> > > > >>> > > > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > > > Any thoughts?
> >>> > > >> >> > > > >>> > > > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > > > *Emmanouil Gkatziouras*
> >>> > > >> >> > > > >>> > > > > > > > > > > https://egkatzioura.com/ |
> >>> > > >> >> > > > >>> > > > > > > > > > >
> >>> > > >> >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >>> > > >> >> > > > >>> > > > > > > > > > > https://github.com/gkatzioura
> >>> > > >> >> > > > >>> > > > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > > >
> >>> > > >> >> > > > >>> > > > > > > >
> >>> > > >> >> > > > >>> > > > > > >
> >>> > > >> >> > > > >>> > > > > >
> >>> > > >> >> > > > >>> > > > >
> >>> > > >> >> > > > >>> > > >
> >>> > > >> >> > > > >>> > >
> >>> > > >> >> > > > >>> >
> >>> > > >> >> > > > >>>
> >>> > > >> >> > > > >>
> >>> > > >> >> > > >
> >>> > > >> >> > >
> >>> > > >> >> >
> >>> > > >> >>
> >>> > > >> >
> >>> > > >>
> >>> > > >
> >>> > >
> >>> >
> >>>
> >>
>

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra <sa...@gmail.com>]

Hi Emmanouil,

The latest build on teamcity has passed on Flink pull request.

https://ci.ignite.apache.org/viewLog.html?buildId=4788928&buildTypeId=IgniteExtensions_Build&tab=buildResultsDiv&branch_IgniteExtensions=pull%2F1%2Fhead

you should be able to take the changes I made in my PR and run build on
your pull request.

Please let me know if you have any questions.

Regards,
Saikat

On Sun, Nov 24, 2019 at 10:44 AM Saikat Maitra <sa...@gmail.com>
wrote:

> Hi Emmanouil,
>
> I have fixed the Flink module testsuites and have setup new project in
> teamcity.
>
>
> https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview
>
> I am looking into setting up the build.
>
> Regards,
> Saikat
>
> On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra <sa...@gmail.com>
> wrote:
>
>> Hi Emmanouil,
>>
>> Thank you for your email. Yes, the plan is once the Flink PR is merged it
>> will provide the Licence, parent POMs etc and you can rebase from master
>> branch and apply the changes on top of it.
>>
>> I am getting some issues with test failures in local with
>> GridTestProperties as the test.properties is not present in this new
>> project but should be available from dependencies.
>>
>> Once I address this issue, I will go ahead and merge the changes and then
>> we can take it from there.
>>
>> Regards,
>> Saikat
>>
>> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras <
>> gkatzioura@gmail.com> wrote:
>>
>>> Hi all,
>>>
>>> I made my first pull request [1]. Since this project is brand new (no
>>> parent poms, licensing), I reckoned it was best to use Saikat's branch on
>>> Flink.
>>> I suppose the Flink branch will be merged first. If not please give me
>>> guidelines on how I should proceed next.
>>>
>>> Kind regards
>>> Emmanouil
>>>
>>> [1] https://github.com/apache/ignite-extensions/pull/2
>>>
>>> *Emmanouil Gkatziouras*
>>> https://egkatzioura.com/ |
>>> https://www.linkedin.com/in/gkatziourasemmanouil/
>>> https://github.com/gkatzioura
>>>
>>>
>>> On Fri, 22 Nov 2019 at 20:55, Denis Magda <dm...@apache.org> wrote:
>>>
>>> > Awesome, ping us whenever you're ready!
>>> >
>>> > -
>>> > Denis
>>> >
>>> >
>>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <
>>> > gkatzioura@gmail.com>
>>> > wrote:
>>> >
>>> > > Hi all!
>>> > >
>>> > > I am sorry for being late on that. I was trying to refactor the test
>>> in
>>> > > order not to be in need of any external tools or spinning up a
>>> server.
>>> > > I did forked the new repo and indeed my changes there, so a pull
>>> request
>>> > is
>>> > > a matter of time!
>>> > >
>>> > > Kind regards
>>> > >
>>> > > *Emmanouil Gkatziouras*
>>> > > https://egkatzioura.com/ |
>>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
>>> > > https://github.com/gkatzioura
>>> > >
>>> > >
>>> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda <dm...@apache.org> wrote:
>>> > >
>>> > > > Hi Emmanouil,
>>> > > >
>>> > > > Do you have any questions or need any support from the community?
>>> > > >
>>> > > > -
>>> > > > Denis
>>> > > >
>>> > > >
>>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <
>>> saikat.maitra@gmail.com
>>> > >
>>> > > > wrote:
>>> > > >
>>> > > >> Hi Emmanouil,
>>> > > >>
>>> > > >> Can you please take a looks at dev utils, if this is something
>>> you are
>>> > > >> looking for
>>> > > >> https://github.com/apache/ignite/tree/master/modules/dev-utils?
>>> > > >>
>>> > > >> IMO, if you can release Pub/Sub server in maven and then use it as
>>> > > >> dependency, that would be great.
>>> > > >>
>>> > > >> Regards,
>>> > > >> Saikat
>>> > > >>
>>> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <
>>> > saikat.maitra@gmail.com>
>>> > > >> wrote:
>>> > > >>
>>> > > >> > Hi Emmanouil,
>>> > > >> >
>>> > > >> > The master branch has been initialized. I have raised a PR to
>>> > migrate
>>> > > >> > Flink module to ignite-extensions.
>>> > > >> >
>>> > > >> > PR https://github.com/apache/ignite-extensions/pull/1
>>> > > >> > Jira : https://issues.apache.org/jira/browse/IGNITE-12356
>>> > > >> >
>>> > > >> > Regards,
>>> > > >> > Saikat
>>> > > >> >
>>> > > >> >
>>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
>>> > > >> > gkatzioura@gmail.com> wrote:
>>> > > >> >
>>> > > >> >> Hi all!
>>> > > >> >>
>>> > > >> >> This is an update. The integration is tested with mocks based
>>> on
>>> > the
>>> > > >> >> original Ignite repo [1].
>>> > > >> >> I am currently waiting for the first commits on the new
>>> integration
>>> > > >> >> project
>>> > > >> >> to take place in order to create my pull request accordingly
>>> to the
>>> > > >> >> initial
>>> > > >> >> commits.
>>> > > >> >> In order to simulate the Pub/Sub server in a test I used a test
>>> > > server
>>> > > >> >> implementation [2]. Unfortunately they don't have this util
>>> > uploaded
>>> > > >> in a
>>> > > >> >> maven repo, thus it needs to be hosted on a maven repo (Apache
>>> > > License
>>> > > >> >> 2.0). Is there a repo for utilities that Ignite uses for the
>>> > > >> test/build?
>>> > > >> >> I have also created a ticket [3]
>>> > > >> >>
>>> > > >> >> Kind regards
>>> > > >> >>
>>> > > >> >> [1]
>>> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
>>> > > >> >> [2]
>>> https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
>>> > > >> >> [3] https://issues.apache.org/jira/browse/IGNITE-12262
>>> > > >> >>
>>> > > >> >> *Emmanouil Gkatziouras*
>>> > > >> >> https://egkatzioura.com/ |
>>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
>>> > > >> >> https://github.com/gkatzioura
>>> > > >> >>
>>> > > >> >>
>>> > > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <
>>> > saikat.maitra@gmail.com
>>> > > >
>>> > > >> >> wrote:
>>> > > >> >>
>>> > > >> >> > Hello Ilya,
>>> > > >> >> >
>>> > > >> >> > Thank you for your email. Yes, before we remove any
>>> integration
>>> > we
>>> > > >> will
>>> > > >> >> > move them to another repository and release.
>>> > > >> >> >
>>> > > >> >> > We are thinking we will start with Pub/Sub Streamer
>>> > implementation
>>> > > >> and
>>> > > >> >> > continue to move other integration as mentioned in the
>>> following
>>> > > doc.
>>> > > >> >> >
>>> > > >> >> >
>>> > > >> >> >
>>> > > >> >>
>>> > > >>
>>> > >
>>> >
>>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>>> > > >> >> >
>>> > > >> >> > Regards,
>>> > > >> >> > Saikat
>>> > > >> >> >
>>> > > >> >> >
>>> > > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
>>> > > >> >> > ilya.kasnacheev@gmail.com>
>>> > > >> >> > wrote:
>>> > > >> >> >
>>> > > >> >> > > Hello!
>>> > > >> >> > >
>>> > > >> >> > > My take on this: we want to offload some of integrations,
>>> but
>>> > > >> please
>>> > > >> >> > commit
>>> > > >> >> > > and release them to another project first, and only then we
>>> > will
>>> > > >> >> decide
>>> > > >> >> > to
>>> > > >> >> > > remove them from our repository.
>>> > > >> >> > >
>>> > > >> >> > > Please also make sure they are available with same artifact
>>> > name
>>> > > >> and
>>> > > >> >> API.
>>> > > >> >> > > If this has to be changed, let's wait for AI 3.0
>>> > > >> >> > >
>>> > > >> >> > > Regards,
>>> > > >> >> > > --
>>> > > >> >> > > Ilya Kasnacheev
>>> > > >> >> > >
>>> > > >> >> > >
>>> > > >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <
>>> > > >> saikat.maitra@gmail.com>:
>>> > > >> >> > >
>>> > > >> >> > > > Hello Emmanouil,
>>> > > >> >> > > >
>>> > > >> >> > > > As discussed earlier, I discussed with Apache Bahir
>>> community
>>> > > and
>>> > > >> >> they
>>> > > >> >> > > are
>>> > > >> >> > > > interested to have Apache Ignite extensions as part of
>>> Apache
>>> > > >> Bahir
>>> > > >> >> > > > project.
>>> > > >> >> > > >
>>> > > >> >> > > > Can you please share Pub/Sub streamer implementation
>>> details
>>> > > with
>>> > > >> >> > Apache
>>> > > >> >> > > > Bahir community and request for feedback.
>>> > > >> >> > > >
>>> > > >> >> > > > https://bahir.apache.org/
>>> > > >> >> > > >
>>> > > >> >> > > > I have also requested for contributor access in Jira for
>>> > Apache
>>> > > >> >> Bahir
>>> > > >> >> > > > project so that I can create issues and assign to myself.
>>> > > >> >> > > >
>>> > > >> >> > > > I can help with code reviews as well.
>>> > > >> >> > > >
>>> > > >> >> > > > Here is the email thread for reference
>>> > > >> >> > > >
>>> > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
>>> > > >> >> > > >
>>> > > >> >> > > > Regards,
>>> > > >> >> > > > Saikat
>>> > > >> >> > > >
>>> > > >> >> > > >
>>> > > >> >> > > >
>>> > > >> >> > > >
>>> > > >> >> > > >
>>> > > >> >> > > >
>>> > > >> >> > > >
>>> > > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <
>>> > > >> >> saikat.maitra@gmail.com
>>> > > >> >> > >
>>> > > >> >> > > > wrote:
>>> > > >> >> > > >
>>> > > >> >> > > > > Hello,
>>> > > >> >> > > > >
>>> > > >> >> > > > > As discussed I have created following discussion
>>> threads on
>>> > > our
>>> > > >> >> > > migration
>>> > > >> >> > > > > proposals for Apache Ignite extensions:
>>> > > >> >> > > > >
>>> > > >> >> > > > >
>>> > > >> >> > > > >
>>> > > >> >> > > >
>>> > > >> >> > >
>>> > > >> >> >
>>> > > >> >>
>>> > > >>
>>> > >
>>> >
>>> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
>>> > > >> >> > > > >
>>> > > >> >> > > > >
>>> > > >> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
>>> > > >> >> > > > >
>>> > > >> >> > > > > I will share update as I hear more information.
>>> > > >> >> > > > >
>>> > > >> >> > > > > Regards,
>>> > > >> >> > > > > Saikat
>>> > > >> >> > > > >
>>> > > >> >> > > > >
>>> > > >> >> > > > >
>>> > > >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra <
>>> > > >> >> > saikat.maitra@gmail.com
>>> > > >> >> > > >
>>> > > >> >> > > > > wrote:
>>> > > >> >> > > > >
>>> > > >> >> > > > >> Hello Denis,
>>> > > >> >> > > > >>
>>> > > >> >> > > > >> Sure, sounds good. I will create a separate discussion
>>> > > thread
>>> > > >> to
>>> > > >> >> get
>>> > > >> >> > > > >> community feedback on whether we
>>> > > >> >> > > > >> should join the Bahir or kick-off "Ignite Extensions"
>>> > > project.
>>> > > >> >> > > > >>
>>> > > >> >> > > > >> Regards,
>>> > > >> >> > > > >> Saikat
>>> > > >> >> > > > >>
>>> > > >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda <
>>> > > >> dmagda@apache.org>
>>> > > >> >> > > wrote:
>>> > > >> >> > > > >>
>>> > > >> >> > > > >>> Folks,
>>> > > >> >> > > > >>>
>>> > > >> >> > > > >>> The concept of Apache Bahir is precisely what we
>>> need!
>>> > > >> Saikat,
>>> > > >> >> > thanks
>>> > > >> >> > > > for
>>> > > >> >> > > > >>> doing the research. Why I believe the * idea* fits us
>>> > well:
>>> > > >> >> > > > >>>
>>> > > >> >> > > > >>>    - All integrations can be stored in separate
>>> Github
>>> > > >> >> repositories
>>> > > >> >> > > and
>>> > > >> >> > > > >>>    have their dev lifecycles. We've not obliged to
>>> couple
>>> > > all
>>> > > >> >> the
>>> > > >> >> > > > >>> integrations
>>> > > >> >> > > > >>>    in a single repo. For instance, Spark can be
>>> located
>>> > in
>>> > > a
>>> > > >> >> > > dedicated
>>> > > >> >> > > > >>> repo
>>> > > >> >> > > > >>>    while streaming integrations might be in a single
>>> one.
>>> > > >> It's
>>> > > >> >> up
>>> > > >> >> > to
>>> > > >> >> > > > us.
>>> > > >> >> > > > >>>    - All the repositories and integrations belong to
>>> ASF.
>>> > > >> We're
>>> > > >> >> not
>>> > > >> >> > > > >>> dumping
>>> > > >> >> > > > >>>    them on Github but rather keep supporting and
>>> > developing
>>> > > >> in
>>> > > >> >> > > > >>> accordance with
>>> > > >> >> > > > >>>    ASF vision and practices.
>>> > > >> >> > > > >>>    - There is a way to reward contributors via
>>> > > committership
>>> > > >> and
>>> > > >> >> > PMC
>>> > > >> >> > > > >>>    membership. You won't get this if a project is
>>> just
>>> > one
>>> > > of
>>> > > >> >> the
>>> > > >> >> > > > >>> millions of
>>> > > >> >> > > > >>>    Github projects.
>>> > > >> >> > > > >>>
>>> > > >> >> > > > >>> Saikat, as Ignite PMC member, could you please
>>> kick-off a
>>> > > >> >> separate
>>> > > >> >> > > > >>> dev-list
>>> > > >> >> > > > >>> discussion to involve more community members and
>>> > referring
>>> > > to
>>> > > >> >> this
>>> > > >> >> > > > >>> thread?
>>> > > >> >> > > > >>> I think the primary question the community needs to
>>> > answer
>>> > > >> >> whether
>>> > > >> >> > we
>>> > > >> >> > > > >>> should join the Bahir or kick-off "Ignite Extensions"
>>> > > >> project?
>>> > > >> >> > > > >>>
>>> > > >> >> > > > >>> -
>>> > > >> >> > > > >>> Denis
>>> > > >> >> > > > >>>
>>> > > >> >> > > > >>>
>>> > > >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev <
>>> > > >> >> > > > zaleslaw.sin@gmail.com>
>>> > > >> >> > > > >>> wrote:
>>> > > >> >> > > > >>>
>>> > > >> >> > > > >>> > Maybe we could move all our Streaming Integrations
>>> > there,
>>> > > >> but
>>> > > >> >> > what
>>> > > >> >> > > is
>>> > > >> >> > > > >>> about
>>> > > >> >> > > > >>> > maintaining and committer permissions to the new
>>> > > >> repositories?
>>> > > >> >> > > > >>> > I see the list of the committers and PMC members
>>> there
>>> > > >> >> > > > >>> > https://bahir.apache.org/community-members/
>>> > > >> >> > > > >>> >
>>> > > >> >> > > > >>> > Could somebody from Ignite community be added to
>>> this
>>> > > list
>>> > > >> as
>>> > > >> >> a
>>> > > >> >> > > > >>> > PMC/committer to maintain Ignite integrations?
>>> > > >> >> > > > >>> > If yes, I'd happy to join this project and
>>> collaborate
>>> > > with
>>> > > >> >> these
>>> > > >> >> > > > guys
>>> > > >> >> > > > >>> > together
>>> > > >> >> > > > >>> > If no, and we should start from the zero level with
>>> > > >> external
>>> > > >> >> PRs
>>> > > >> >> > -
>>> > > >> >> > > > >>> hmmm,
>>> > > >> >> > > > >>> > it's better to have our own external repository
>>> with
>>> > > >> >> ApacheBahirr
>>> > > >> >> > > > >>> ideology.
>>> > > >> >> > > > >>> >
>>> > > >> >> > > > >>> > Also, I agree, that all connectors could be moved
>>> there
>>> > > (in
>>> > > >> >> > > > >>> ApacheBahirr
>>> > > >> >> > > > >>> > like repository), but not all modules from the page
>>> > > >> >> > > > >>> >
>>> > > >> >> > > > >>> >
>>> > > >> >> > > > >>>
>>> > > >> >> > > >
>>> > > >> >> > >
>>> > > >> >> >
>>> > > >> >>
>>> > > >>
>>> > >
>>> >
>>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>>> > > >> >> > > > >>> > because
>>> > > >> >> > > > >>> > they use different parts of core modules and could
>>> be
>>> > > >> >> developed
>>> > > >> >> > > with
>>> > > >> >> > > > >>> > different velocity.
>>> > > >> >> > > > >>> >
>>> > > >> >> > > > >>> > In reality, before creation of new repositories we
>>> need
>>> > > >> wide
>>> > > >> >> > > > discussion
>>> > > >> >> > > > >>> > based on the document mentioned above.
>>> > > >> >> > > > >>> >
>>> > > >> >> > > > >>> > P.S. Of course, we could experiment in the next
>>> release
>>> > > 2.8
>>> > > >> >> with
>>> > > >> >> > > one
>>> > > >> >> > > > >>> or two
>>> > > >> >> > > > >>> > integrations like twitter or ZeroMQ
>>> > > >> >> > > > >>> > Also, I'd like Denis Magda idea for the separate
>>> > > >> repositories
>>> > > >> >> in
>>> > > >> >> > > > Apache
>>> > > >> >> > > > >>> > Github to maintain them by anybody.
>>> > > >> >> > > > >>> >
>>> > > >> >> > > > >>> >
>>> > > >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra <
>>> > > >> >> > > saikat.maitra@gmail.com
>>> > > >> >> > > > >:
>>> > > >> >> > > > >>> >
>>> > > >> >> > > > >>> > > Hi Denis, Emmanouil
>>> > > >> >> > > > >>> > >
>>> > > >> >> > > > >>> > > Thank you for your email. I think creating a
>>> separate
>>> > > >> >> > integration
>>> > > >> >> > > > >>> project
>>> > > >> >> > > > >>> > > in github and also proposing it as an apache
>>> > incubator
>>> > > >> >> project
>>> > > >> >> > > will
>>> > > >> >> > > > >>> be a
>>> > > >> >> > > > >>> > > good move. The other separate integration modules
>>> > > >> >> > > > >>> > >
>>> > > >> >> > > > >>> > >
>>> > > >> >> > > > >>> >
>>> > > >> >> > > > >>>
>>> > > >> >> > > >
>>> > > >> >> > >
>>> > > >> >> >
>>> > > >> >>
>>> > > >>
>>> > >
>>> >
>>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>>> > > >> >> > > > >>> > > can
>>> > > >> >> > > > >>> > > be moved to this new apache incubator project.
>>> > > >> >> > > > >>> > >
>>> > > >> >> > > > >>> > > There are similar integration available for
>>> Flink and
>>> > > >> Spark
>>> > > >> >> in
>>> > > >> >> > > > Apache
>>> > > >> >> > > > >>> > Bahir
>>> > > >> >> > > > >>> > > https://bahir.apache.org/
>>> > > >> >> > > > >>> > >
>>> > > >> >> > > > >>> > > Do you think we should reach out to Apache Bahir
>>> > > >> community
>>> > > >> >> > with a
>>> > > >> >> > > > >>> > proposal
>>> > > >> >> > > > >>> > > or we should plan to create a separate Apache
>>> > Incubator
>>> > > >> >> > project?
>>> > > >> >> > > > >>> > >
>>> > > >> >> > > > >>> > > Regards,
>>> > > >> >> > > > >>> > > Saikat
>>> > > >> >> > > > >>> > >
>>> > > >> >> > > > >>> > >
>>> > > >> >> > > > >>> > >
>>> > > >> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil
>>> > Gkatziouras <
>>> > > >> >> > > > >>> > > gkatzioura@gmail.com>
>>> > > >> >> > > > >>> > > wrote:
>>> > > >> >> > > > >>> > >
>>> > > >> >> > > > >>> > > > Hi all!
>>> > > >> >> > > > >>> > > >
>>> > > >> >> > > > >>> > > > Based on the discussions most probably this is
>>> > going
>>> > > to
>>> > > >> >> be on
>>> > > >> >> > > > >>> another
>>> > > >> >> > > > >>> > > > GitHub repo. Therefore I will prepare a
>>> standalone
>>> > > >> project
>>> > > >> >> > with
>>> > > >> >> > > > the
>>> > > >> >> > > > >>> > > Pub/Sub
>>> > > >> >> > > > >>> > > > module and once the repository is created a
>>> pull
>>> > > >> request
>>> > > >> >> > shall
>>> > > >> >> > > be
>>> > > >> >> > > > >>> > issued.
>>> > > >> >> > > > >>> > > > If there is anything else I can do in order to
>>> > > >> facilitate
>>> > > >> >> > this
>>> > > >> >> > > > >>> please
>>> > > >> >> > > > >>> > let
>>> > > >> >> > > > >>> > > > me know.
>>> > > >> >> > > > >>> > > >
>>> > > >> >> > > > >>> > > > If a ticket is created for this issue, I shall
>>> be
>>> > > able
>>> > > >> to
>>> > > >> >> > > create
>>> > > >> >> > > > a
>>> > > >> >> > > > >>> pull
>>> > > >> >> > > > >>> > > > request and thus a build will take place on
>>> > ignite's
>>> > > >> CI.
>>> > > >> >> > > > >>> > > > If everyone is aligned with adding a Pub/Sub
>>> > > >> >> implementation
>>> > > >> >> > > may I
>>> > > >> >> > > > >>> > create
>>> > > >> >> > > > >>> > > a
>>> > > >> >> > > > >>> > > > ticket on JIRA?
>>> > > >> >> > > > >>> > > >
>>> > > >> >> > > > >>> > > > My next question has to do on how Ignite
>>> configures
>>> > > the
>>> > > >> >> build
>>> > > >> >> > > > jobs
>>> > > >> >> > > > >>> in
>>> > > >> >> > > > >>> > > Team
>>> > > >> >> > > > >>> > > > City. The unit tests for my implementation
>>> need a
>>> > > local
>>> > > >> >> > Pub/Sub
>>> > > >> >> > > > >>> server
>>> > > >> >> > > > >>> > up
>>> > > >> >> > > > >>> > > > and running (they provide one for testing
>>> > purposes).
>>> > > >> Will
>>> > > >> >> > this
>>> > > >> >> > > > take
>>> > > >> >> > > > >>> > > effect
>>> > > >> >> > > > >>> > > > in a form of a build script inside my
>>> > implementation
>>> > > or
>>> > > >> >> is it
>>> > > >> >> > > > >>> something
>>> > > >> >> > > > >>> > > > that should be configured on Team City?
>>> > > >> >> > > > >>> > > >
>>> > > >> >> > > > >>> > > > Kind regards,
>>> > > >> >> > > > >>> > > > Emmanouil
>>> > > >> >> > > > >>> > > >
>>> > > >> >> > > > >>> > > >
>>> > > >> >> > > > >>> > > > *Emmanouil Gkatziouras*
>>> > > >> >> > > > >>> > > > https://egkatzioura.com/ |
>>> > > >> >> > > > >>> > > >
>>> https://www.linkedin.com/in/gkatziourasemmanouil/
>>> > > >> >> > > > >>> > > > https://github.com/gkatzioura
>>> > > >> >> > > > >>> > > >
>>> > > >> >> > > > >>> > > >
>>> > > >> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis Magda <
>>> > > >> >> dmagda@apache.org
>>> > > >> >> > >
>>> > > >> >> > > > >>> wrote:
>>> > > >> >> > > > >>> > > >
>>> > > >> >> > > > >>> > > > > Emmanouil, Saikat,
>>> > > >> >> > > > >>> > > > >
>>> > > >> >> > > > >>> > > > > After contemplating on this, let me propose
>>> > another
>>> > > >> way
>>> > > >> >> > with
>>> > > >> >> > > > this
>>> > > >> >> > > > >>> > > > > particular integration that is aligned with
>>> our
>>> > > >> >> > > modularization
>>> > > >> >> > > > >>> > > endeavor.
>>> > > >> >> > > > >>> > > > > The modularization [1] defines the Ignite
>>> core,
>>> > > >> Ignite
>>> > > >> >> > > modules,
>>> > > >> >> > > > >>> and
>>> > > >> >> > > > >>> > > > > independent integrations. The first two are
>>> to be
>>> > > >> >> located
>>> > > >> >> > in
>>> > > >> >> > > > >>> Ignite
>>> > > >> >> > > > >>> > > > > repositories and to be tested/updated by the
>>> > > >> community
>>> > > >> >> for
>>> > > >> >> > > > every
>>> > > >> >> > > > >>> > > release.
>>> > > >> >> > > > >>> > > > > The modules are a subset of existing
>>> integrations
>>> > > >> that
>>> > > >> >> are
>>> > > >> >> > > > >>> adopted
>>> > > >> >> > > > >>> > > widely
>>> > > >> >> > > > >>> > > > > and require community attention.
>>> > > >> >> > > > >>> > > > >
>>> > > >> >> > > > >>> > > > > As for the independent integrations [1],
>>> those
>>> > are
>>> > > >> >> supposed
>>> > > >> >> > > to
>>> > > >> >> > > > be
>>> > > >> >> > > > >>> > > located
>>> > > >> >> > > > >>> > > > > in a separate GitHub repo. I would advise us
>>> to
>>> > > start
>>> > > >> >> > > > >>> implementing
>>> > > >> >> > > > >>> > this
>>> > > >> >> > > > >>> > > > > concept with the Pub/Sub integration. Even
>>> though
>>> > > it
>>> > > >> >> will
>>> > > >> >> > be
>>> > > >> >> > > > >>> located
>>> > > >> >> > > > >>> > > in a
>>> > > >> >> > > > >>> > > > > separate Github repo, it will be listed among
>>> > > >> officially
>>> > > >> >> > > > >>> available
>>> > > >> >> > > > >>> > > > > integrations on the Ignite website featuring
>>> > > >> Emmanouil
>>> > > >> >> as a
>>> > > >> >> > > > >>> > > contributor.
>>> > > >> >> > > > >>> > > > >
>>> > > >> >> > > > >>> > > > > What do you think?
>>> > > >> >> > > > >>> > > > >
>>> > > >> >> > > > >>> > > > >
>>> > > >> >> > > > >>> > > > > [1]
>>> > > >> >> > > > >>> > > > >
>>> > > >> >> > > > >>> > > > >
>>> > > >> >> > > > >>> > > >
>>> > > >> >> > > > >>> > >
>>> > > >> >> > > > >>> >
>>> > > >> >> > > > >>>
>>> > > >> >> > > >
>>> > > >> >> > >
>>> > > >> >> >
>>> > > >> >>
>>> > > >>
>>> > >
>>> >
>>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>>> > > >> >> > > > >>> > > > >
>>> > > >> >> > > > >>> > > > >
>>> > > >> >> > > > >>> > > > >
>>> > > >> >> > > > >>> > > > > -
>>> > > >> >> > > > >>> > > > > Denis
>>> > > >> >> > > > >>> > > > >
>>> > > >> >> > > > >>> > > > >
>>> > > >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM Saikat
>>> Maitra <
>>> > > >> >> > > > >>> > saikat.maitra@gmail.com
>>> > > >> >> > > > >>> > > >
>>> > > >> >> > > > >>> > > > > wrote:
>>> > > >> >> > > > >>> > > > >
>>> > > >> >> > > > >>> > > > > > Hi Emmanouil,
>>> > > >> >> > > > >>> > > > > >
>>> > > >> >> > > > >>> > > > > > The changes looks good to me. I wanted to
>>> check
>>> > > if
>>> > > >> you
>>> > > >> >> > are
>>> > > >> >> > > > also
>>> > > >> >> > > > >>> > able
>>> > > >> >> > > > >>> > > to
>>> > > >> >> > > > >>> > > > > > validate that the once data is added to
>>> Ignite
>>> > > >> cluster
>>> > > >> >> > you
>>> > > >> >> > > > are
>>> > > >> >> > > > >>> also
>>> > > >> >> > > > >>> > > > able
>>> > > >> >> > > > >>> > > > > to
>>> > > >> >> > > > >>> > > > > > access it using another Ignite client or
>>> using
>>> > > rest
>>> > > >> >> > > > endpoints.
>>> > > >> >> > > > >>> > > > > >
>>> > > >> >> > > > >>> > > > > > We use Teamcity for CI process, the
>>> details are
>>> > > as
>>> > > >> >> > follows
>>> > > >> >> > > > >>> > > > > >
>>> > > >> >> > > > >>> > > > > > Trigger validation of those test suites
>>> that
>>> > have
>>> > > >> been
>>> > > >> >> > > > >>> affected by
>>> > > >> >> > > > >>> > > your
>>> > > >> >> > > > >>> > > > > > changes on TeamCity <
>>> > > http://ci.ignite.apache.org/>:
>>> > > >> <
>>> > > >> >> > > > >>> > > > > http://204.14.53.153/
>>> > > >> >> > > > >>> > > > > > >
>>> > > >> >> > > > >>> > > > > >
>>> > > >> >> > > > >>> > > > > >    - Locate a test suite you have to check,
>>> > press
>>> > > >> >> button
>>> > > >> >> > > > named
>>> > > >> >> > > > >>> > "..."
>>> > > >> >> > > > >>> > > > that
>>> > > >> >> > > > >>> > > > > >    is located on the left of "Run" button.
>>> "Run
>>> > > >> custom
>>> > > >> >> > > build"
>>> > > >> >> > > > >>> > dialog
>>> > > >> >> > > > >>> > > > will
>>> > > >> >> > > > >>> > > > > >    appear;
>>> > > >> >> > > > >>> > > > > >    - Go to "Changes" tab and choose
>>> > > >> >> > > > >>> > "pull/<pull-request-number>/head"
>>> > > >> >> > > > >>> > > > in
>>> > > >> >> > > > >>> > > > > >    "Build branch" dropdown list;
>>> > > >> >> > > > >>> > > > > >    - Press "Run build" button and monitor
>>> tests
>>> > > >> >> results.
>>> > > >> >> > > > >>> > > > > >
>>> > > >> >> > > > >>> > > > > >
>>> > > >> >> > > > >>> > > > > >
>>> > > >> >> > > > >>> > > > > > @Roman rshtykh@yahoo.com - can you also
>>> please
>>> > > >> take a
>>> > > >> >> > > look?
>>> > > >> >> > > > >>> > > > > >
>>> > > >> >> > > > >>> > > > > > Regards,
>>> > > >> >> > > > >>> > > > > > Saikat
>>> > > >> >> > > > >>> > > > > >
>>> > > >> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20 PM Emmanouil
>>> > > >> Gkatziouras
>>> > > >> >> <
>>> > > >> >> > > > >>> > > > > > gkatzioura@gmail.com>
>>> > > >> >> > > > >>> > > > > > wrote:
>>> > > >> >> > > > >>> > > > > >
>>> > > >> >> > > > >>> > > > > > > Hi @Saikat Maitra
>>> > > >> >> > > > >>> > > > > > >
>>> > > >> >> > > > >>> > > > > > > I have my implementation on an Ignite
>>> fork on
>>> > > my
>>> > > >> >> > personal
>>> > > >> >> > > > >>> GitHub
>>> > > >> >> > > > >>> > > > > account
>>> > > >> >> > > > >>> > > > > > > [1]
>>> > > >> >> > > > >>> > > > > > > It is based on the Kafka implementation
>>> which
>>> > > is
>>> > > >> of
>>> > > >> >> a
>>> > > >> >> > > > polling
>>> > > >> >> > > > >>> > > nature.
>>> > > >> >> > > > >>> > > > > > > GCP does not provide a service like
>>> Kafka.
>>> > > >> Instead
>>> > > >> >> it
>>> > > >> >> > > > >>> provides us
>>> > > >> >> > > > >>> > > > with
>>> > > >> >> > > > >>> > > > > > > Pub/Sub which is just like Kafka.
>>> > > >> >> > > > >>> > > > > > > Therefore using the Kafka streamer as a
>>> > > guideline
>>> > > >> >> made
>>> > > >> >> > > more
>>> > > >> >> > > > >>> sense
>>> > > >> >> > > > >>> > > to
>>> > > >> >> > > > >>> > > > > me.
>>> > > >> >> > > > >>> > > > > > >
>>> > > >> >> > > > >>> > > > > > > Apart from the implementation I have also
>>> > > added a
>>> > > >> >> unit
>>> > > >> >> > > > test.
>>> > > >> >> > > > >>> > > > > > > It definitely needs some polishing and a
>>> > double
>>> > > >> >> check
>>> > > >> >> > on
>>> > > >> >> > > my
>>> > > >> >> > > > >>> side
>>> > > >> >> > > > >>> > to
>>> > > >> >> > > > >>> > > > > make
>>> > > >> >> > > > >>> > > > > > > the review smooth.
>>> > > >> >> > > > >>> > > > > > > For the tests a local Pub/Sub [2] server
>>> has
>>> > to
>>> > > >> be
>>> > > >> >> > setup
>>> > > >> >> > > > >>> however
>>> > > >> >> > > > >>> > I
>>> > > >> >> > > > >>> > > am
>>> > > >> >> > > > >>> > > > > not
>>> > > >> >> > > > >>> > > > > > > sure which is the best way to do so with
>>> > > regards
>>> > > >> to
>>> > > >> >> > your
>>> > > >> >> > > > >>> CI/CD.
>>> > > >> >> > > > >>> > > > > > >
>>> > > >> >> > > > >>> > > > > > > [1]
>>> > > >> >> > > > >>> > >
>>> > > >> >> >
>>> https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
>>> > > >> >> > > > >>> > > > > > > [2]
>>> > > >> >> > > > >>>
>>> > https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
>>> > > >> >> > > > >>> > > > > > >
>>> > > >> >> > > > >>> > > > > > > *Emmanouil Gkatziouras*
>>> > > >> >> > > > >>> > > > > > > https://egkatzioura.com/ |
>>> > > >> >> > > > >>> > > > > > >
>>> > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
>>> > > >> >> > > > >>> > > > > > > https://github.com/gkatzioura
>>> > > >> >> > > > >>> > > > > > >
>>> > > >> >> > > > >>> > > > > > >
>>> > > >> >> > > > >>> > > > > > > On Sat, 12 Oct 2019 at 02:11, Saikat
>>> Maitra <
>>> > > >> >> > > > >>> > > saikat.maitra@gmail.com
>>> > > >> >> > > > >>> > > > >
>>> > > >> >> > > > >>> > > > > > > wrote:
>>> > > >> >> > > > >>> > > > > > >
>>> > > >> >> > > > >>> > > > > > > > Hello Emmanouil,
>>> > > >> >> > > > >>> > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > Can you please share more info about
>>> the
>>> > > >> >> > > implementation?
>>> > > >> >> > > > >>> > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > We have similar implementation for data
>>> > > >> streamers
>>> > > >> >> > with
>>> > > >> >> > > > >>> Kafka
>>> > > >> >> > > > >>> > and
>>> > > >> >> > > > >>> > > > > > RocketMQ
>>> > > >> >> > > > >>> > > > > > > >
>>> > > >> >> > > > >>> > > > > > > >
>>> > > >> >> > https://apacheignite-mix.readme.io/docs/kafka-streamer
>>> > > >> >> > > > >>> > > > > > > >
>>> > > >> >> > > > >>> > > > > > > >
>>> > > >> >> > > >
>>> https://apacheignite-mix.readme.io/docs/rocketmq-streamer
>>> > > >> >> > > > >>> > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > Please review and share if you have any
>>> > > >> questions.
>>> > > >> >> > > > >>> > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > Regards,
>>> > > >> >> > > > >>> > > > > > > > Saikat
>>> > > >> >> > > > >>> > > > > > > >
>>> > > >> >> > > > >>> > > > > > > >
>>> > > >> >> > > > >>> > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > On Fri, Oct 11, 2019 at 4:56 PM
>>> Emmanouil
>>> > > >> >> > Gkatziouras <
>>> > > >> >> > > > >>> > > > > > > > gkatzioura@gmail.com>
>>> > > >> >> > > > >>> > > > > > > > wrote:
>>> > > >> >> > > > >>> > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > Hi all,
>>> > > >> >> > > > >>> > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > By mentioning Pub/Sub I mean Google
>>> > Cloud's
>>> > > >> >> Pub/Sub
>>> > > >> >> > > [1]
>>> > > >> >> > > > >>> > > > > > > > > Yes it is indeed cloud specific. It
>>> is
>>> > the
>>> > > >> >> > messaging
>>> > > >> >> > > > >>> service
>>> > > >> >> > > > >>> > > > > provided
>>> > > >> >> > > > >>> > > > > > > by
>>> > > >> >> > > > >>> > > > > > > > > GCP.
>>> > > >> >> > > > >>> > > > > > > > > I have made the integration with it,
>>> but
>>> > I
>>> > > >> >> haven't
>>> > > >> >> > > > >>> published
>>> > > >> >> > > > >>> > it
>>> > > >> >> > > > >>> > > > yet
>>> > > >> >> > > > >>> > > > > > (I
>>> > > >> >> > > > >>> > > > > > > am
>>> > > >> >> > > > >>> > > > > > > > > waiting for the discussion/ticket and
>>> > > proceed
>>> > > >> >> with
>>> > > >> >> > > the
>>> > > >> >> > > > >>> > > > > contribution).
>>> > > >> >> > > > >>> > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > Kind regards,
>>> > > >> >> > > > >>> > > > > > > > > Emmanouil
>>> > > >> >> > > > >>> > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > [1]
>>> > > >> >> https://cloud.google.com/pubsub/docs/overview
>>> > > >> >> > > > >>> > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > *Emmanouil Gkatziouras*
>>> > > >> >> > > > >>> > > > > > > > > https://egkatzioura.com/ |
>>> > > >> >> > > > >>> > > > > > > > >
>>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
>>> > > >> >> > > > >>> > > > > > > > > https://github.com/gkatzioura
>>> > > >> >> > > > >>> > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > On Fri, 11 Oct 2019 at 21:45, Denis
>>> > Magda <
>>> > > >> >> > > > >>> dmagda@apache.org
>>> > > >> >> > > > >>> > >
>>> > > >> >> > > > >>> > > > > wrote:
>>> > > >> >> > > > >>> > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > > Hello Emmanouil,
>>> > > >> >> > > > >>> > > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > > Are you referring to any specific
>>> > > >> >> implementation
>>> > > >> >> > > > >>> tailored
>>> > > >> >> > > > >>> > > for a
>>> > > >> >> > > > >>> > > > > > > cloud?
>>> > > >> >> > > > >>> > > > > > > > If
>>> > > >> >> > > > >>> > > > > > > > > > I'm not mistaken you were
>>> mentioning
>>> > > >> AWS/GCE.
>>> > > >> >> Is
>>> > > >> >> > > > there
>>> > > >> >> > > > >>> any
>>> > > >> >> > > > >>> > > > Github
>>> > > >> >> > > > >>> > > > > > > > source
>>> > > >> >> > > > >>> > > > > > > > > > code?
>>> > > >> >> > > > >>> > > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > > Here is a list of existing
>>> integrations
>>> > > >> with
>>> > > >> >> > > > streaming
>>> > > >> >> > > > >>> and
>>> > > >> >> > > > >>> > > > > pub/sub
>>> > > >> >> > > > >>> > > > > > > > > > frameworks:
>>> > > >> >> > > > >>> > > > > > > > > >
>>> > > >> >> https://apacheignite-mix.readme.io/docs/overview
>>> > > >> >> > > > >>> > > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > > with the source code in Ignite
>>> master:
>>> > > >> >> > > > >>> > > > > > > > > >
>>> > > >> >> > > https://github.com/apache/ignite/tree/master/modules
>>> > > >> >> > > > >>> > > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > > Check out those resources to see
>>> what
>>> > are
>>> > > >> the
>>> > > >> >> > most
>>> > > >> >> > > > >>> suitable
>>> > > >> >> > > > >>> > > > > Ignite
>>> > > >> >> > > > >>> > > > > > > APIs
>>> > > >> >> > > > >>> > > > > > > > > for
>>> > > >> >> > > > >>> > > > > > > > > > such integrations.
>>> > > >> >> > > > >>> > > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > > @Saikat Maitra <
>>> > saikat.maitra@gmail.com>
>>> > > ,
>>> > > >> >> > @Roman
>>> > > >> >> > > <
>>> > > >> >> > > > >>> > > > > > rshtykh@yahoo.com
>>> > > >> >> > > > >>> > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > ,
>>> > > >> >> > > > >>> > > > > > > > > as
>>> > > >> >> > > > >>> > > > > > > > > > streaming technologies
>>> > > >> >> contributors/maintainers,
>>> > > >> >> > > > could
>>> > > >> >> > > > >>> you
>>> > > >> >> > > > >>> > > > please
>>> > > >> >> > > > >>> > > > > > > join
>>> > > >> >> > > > >>> > > > > > > > > this
>>> > > >> >> > > > >>> > > > > > > > > > thread?
>>> > > >> >> > > > >>> > > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > > -
>>> > > >> >> > > > >>> > > > > > > > > > Denis
>>> > > >> >> > > > >>> > > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > > On Thu, Oct 10, 2019 at 7:58 AM
>>> > Emmanouil
>>> > > >> >> > > > Gkatziouras <
>>> > > >> >> > > > >>> > > > > > > > > > gkatzioura@gmail.com>
>>> > > >> >> > > > >>> > > > > > > > > > wrote:
>>> > > >> >> > > > >>> > > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > > > Hello everyone,
>>> > > >> >> > > > >>> > > > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > > > I started using Ignite lately.
>>> Part
>>> > of
>>> > > my
>>> > > >> >> work
>>> > > >> >> > > > >>> involves
>>> > > >> >> > > > >>> > > > > Pub/Sub,
>>> > > >> >> > > > >>> > > > > > > > thus I
>>> > > >> >> > > > >>> > > > > > > > > > > created a pub/sub streamer.
>>> > > >> >> > > > >>> > > > > > > > > > > Pub/Sub is pretty close to Kafka.
>>> > > >> >> > > > >>> > > > > > > > > > > Here's how the streamer works.
>>> Every
>>> > > node
>>> > > >> >> > creates
>>> > > >> >> > > > an
>>> > > >> >> > > > >>> > > instance
>>> > > >> >> > > > >>> > > > > of
>>> > > >> >> > > > >>> > > > > > a
>>> > > >> >> > > > >>> > > > > > > > > > Pub/Sub
>>> > > >> >> > > > >>> > > > > > > > > > > client.
>>> > > >> >> > > > >>> > > > > > > > > > > The clients subscribe to the same
>>> > topic
>>> > > >> >> using
>>> > > >> >> > the
>>> > > >> >> > > > >>> same
>>> > > >> >> > > > >>> > > > > > subscription
>>> > > >> >> > > > >>> > > > > > > > > name.
>>> > > >> >> > > > >>> > > > > > > > > > > A message is being sent and one
>>> of
>>> > the
>>> > > >> nodes
>>> > > >> >> > will
>>> > > >> >> > > > >>> receive
>>> > > >> >> > > > >>> > > the
>>> > > >> >> > > > >>> > > > > > > > message,
>>> > > >> >> > > > >>> > > > > > > > > > > process it and then forward it to
>>> > > Ignite
>>> > > >> >> using
>>> > > >> >> > > the
>>> > > >> >> > > > >>> > > extractor
>>> > > >> >> > > > >>> > > > > > > > provided.
>>> > > >> >> > > > >>> > > > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > > > Any thoughts?
>>> > > >> >> > > > >>> > > > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > > > *Emmanouil Gkatziouras*
>>> > > >> >> > > > >>> > > > > > > > > > > https://egkatzioura.com/ |
>>> > > >> >> > > > >>> > > > > > > > > > >
>>> > > >> >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
>>> > > >> >> > > > >>> > > > > > > > > > > https://github.com/gkatzioura
>>> > > >> >> > > > >>> > > > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > > >
>>> > > >> >> > > > >>> > > > > > > >
>>> > > >> >> > > > >>> > > > > > >
>>> > > >> >> > > > >>> > > > > >
>>> > > >> >> > > > >>> > > > >
>>> > > >> >> > > > >>> > > >
>>> > > >> >> > > > >>> > >
>>> > > >> >> > > > >>> >
>>> > > >> >> > > > >>>
>>> > > >> >> > > > >>
>>> > > >> >> > > >
>>> > > >> >> > >
>>> > > >> >> >
>>> > > >> >>
>>> > > >> >
>>> > > >>
>>> > > >
>>> > >
>>> >
>>>
>>

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra <sa...@gmail.com>]

Hi Emmanouil,

I have fixed the Flink module testsuites and have setup new project in
teamcity.

https://ci.ignite.apache.org/project.html?projectId=IgniteExtensions&tab=projectOverview

I am looking into setting up the build.

Regards,
Saikat

On Sat, Nov 23, 2019 at 9:52 AM Saikat Maitra <sa...@gmail.com>
wrote:

> Hi Emmanouil,
>
> Thank you for your email. Yes, the plan is once the Flink PR is merged it
> will provide the Licence, parent POMs etc and you can rebase from master
> branch and apply the changes on top of it.
>
> I am getting some issues with test failures in local with
> GridTestProperties as the test.properties is not present in this new
> project but should be available from dependencies.
>
> Once I address this issue, I will go ahead and merge the changes and then
> we can take it from there.
>
> Regards,
> Saikat
>
> On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras <
> gkatzioura@gmail.com> wrote:
>
>> Hi all,
>>
>> I made my first pull request [1]. Since this project is brand new (no
>> parent poms, licensing), I reckoned it was best to use Saikat's branch on
>> Flink.
>> I suppose the Flink branch will be merged first. If not please give me
>> guidelines on how I should proceed next.
>>
>> Kind regards
>> Emmanouil
>>
>> [1] https://github.com/apache/ignite-extensions/pull/2
>>
>> *Emmanouil Gkatziouras*
>> https://egkatzioura.com/ |
>> https://www.linkedin.com/in/gkatziourasemmanouil/
>> https://github.com/gkatzioura
>>
>>
>> On Fri, 22 Nov 2019 at 20:55, Denis Magda <dm...@apache.org> wrote:
>>
>> > Awesome, ping us whenever you're ready!
>> >
>> > -
>> > Denis
>> >
>> >
>> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <
>> > gkatzioura@gmail.com>
>> > wrote:
>> >
>> > > Hi all!
>> > >
>> > > I am sorry for being late on that. I was trying to refactor the test
>> in
>> > > order not to be in need of any external tools or spinning up a server.
>> > > I did forked the new repo and indeed my changes there, so a pull
>> request
>> > is
>> > > a matter of time!
>> > >
>> > > Kind regards
>> > >
>> > > *Emmanouil Gkatziouras*
>> > > https://egkatzioura.com/ |
>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > https://github.com/gkatzioura
>> > >
>> > >
>> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda <dm...@apache.org> wrote:
>> > >
>> > > > Hi Emmanouil,
>> > > >
>> > > > Do you have any questions or need any support from the community?
>> > > >
>> > > > -
>> > > > Denis
>> > > >
>> > > >
>> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <
>> saikat.maitra@gmail.com
>> > >
>> > > > wrote:
>> > > >
>> > > >> Hi Emmanouil,
>> > > >>
>> > > >> Can you please take a looks at dev utils, if this is something you
>> are
>> > > >> looking for
>> > > >> https://github.com/apache/ignite/tree/master/modules/dev-utils?
>> > > >>
>> > > >> IMO, if you can release Pub/Sub server in maven and then use it as
>> > > >> dependency, that would be great.
>> > > >>
>> > > >> Regards,
>> > > >> Saikat
>> > > >>
>> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <
>> > saikat.maitra@gmail.com>
>> > > >> wrote:
>> > > >>
>> > > >> > Hi Emmanouil,
>> > > >> >
>> > > >> > The master branch has been initialized. I have raised a PR to
>> > migrate
>> > > >> > Flink module to ignite-extensions.
>> > > >> >
>> > > >> > PR https://github.com/apache/ignite-extensions/pull/1
>> > > >> > Jira : https://issues.apache.org/jira/browse/IGNITE-12356
>> > > >> >
>> > > >> > Regards,
>> > > >> > Saikat
>> > > >> >
>> > > >> >
>> > > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
>> > > >> > gkatzioura@gmail.com> wrote:
>> > > >> >
>> > > >> >> Hi all!
>> > > >> >>
>> > > >> >> This is an update. The integration is tested with mocks based on
>> > the
>> > > >> >> original Ignite repo [1].
>> > > >> >> I am currently waiting for the first commits on the new
>> integration
>> > > >> >> project
>> > > >> >> to take place in order to create my pull request accordingly to
>> the
>> > > >> >> initial
>> > > >> >> commits.
>> > > >> >> In order to simulate the Pub/Sub server in a test I used a test
>> > > server
>> > > >> >> implementation [2]. Unfortunately they don't have this util
>> > uploaded
>> > > >> in a
>> > > >> >> maven repo, thus it needs to be hosted on a maven repo (Apache
>> > > License
>> > > >> >> 2.0). Is there a repo for utilities that Ignite uses for the
>> > > >> test/build?
>> > > >> >> I have also created a ticket [3]
>> > > >> >>
>> > > >> >> Kind regards
>> > > >> >>
>> > > >> >> [1]
>> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
>> > > >> >> [2]
>> https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
>> > > >> >> [3] https://issues.apache.org/jira/browse/IGNITE-12262
>> > > >> >>
>> > > >> >> *Emmanouil Gkatziouras*
>> > > >> >> https://egkatzioura.com/ |
>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > >> >> https://github.com/gkatzioura
>> > > >> >>
>> > > >> >>
>> > > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <
>> > saikat.maitra@gmail.com
>> > > >
>> > > >> >> wrote:
>> > > >> >>
>> > > >> >> > Hello Ilya,
>> > > >> >> >
>> > > >> >> > Thank you for your email. Yes, before we remove any
>> integration
>> > we
>> > > >> will
>> > > >> >> > move them to another repository and release.
>> > > >> >> >
>> > > >> >> > We are thinking we will start with Pub/Sub Streamer
>> > implementation
>> > > >> and
>> > > >> >> > continue to move other integration as mentioned in the
>> following
>> > > doc.
>> > > >> >> >
>> > > >> >> >
>> > > >> >> >
>> > > >> >>
>> > > >>
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> > > >> >> >
>> > > >> >> > Regards,
>> > > >> >> > Saikat
>> > > >> >> >
>> > > >> >> >
>> > > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
>> > > >> >> > ilya.kasnacheev@gmail.com>
>> > > >> >> > wrote:
>> > > >> >> >
>> > > >> >> > > Hello!
>> > > >> >> > >
>> > > >> >> > > My take on this: we want to offload some of integrations,
>> but
>> > > >> please
>> > > >> >> > commit
>> > > >> >> > > and release them to another project first, and only then we
>> > will
>> > > >> >> decide
>> > > >> >> > to
>> > > >> >> > > remove them from our repository.
>> > > >> >> > >
>> > > >> >> > > Please also make sure they are available with same artifact
>> > name
>> > > >> and
>> > > >> >> API.
>> > > >> >> > > If this has to be changed, let's wait for AI 3.0
>> > > >> >> > >
>> > > >> >> > > Regards,
>> > > >> >> > > --
>> > > >> >> > > Ilya Kasnacheev
>> > > >> >> > >
>> > > >> >> > >
>> > > >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <
>> > > >> saikat.maitra@gmail.com>:
>> > > >> >> > >
>> > > >> >> > > > Hello Emmanouil,
>> > > >> >> > > >
>> > > >> >> > > > As discussed earlier, I discussed with Apache Bahir
>> community
>> > > and
>> > > >> >> they
>> > > >> >> > > are
>> > > >> >> > > > interested to have Apache Ignite extensions as part of
>> Apache
>> > > >> Bahir
>> > > >> >> > > > project.
>> > > >> >> > > >
>> > > >> >> > > > Can you please share Pub/Sub streamer implementation
>> details
>> > > with
>> > > >> >> > Apache
>> > > >> >> > > > Bahir community and request for feedback.
>> > > >> >> > > >
>> > > >> >> > > > https://bahir.apache.org/
>> > > >> >> > > >
>> > > >> >> > > > I have also requested for contributor access in Jira for
>> > Apache
>> > > >> >> Bahir
>> > > >> >> > > > project so that I can create issues and assign to myself.
>> > > >> >> > > >
>> > > >> >> > > > I can help with code reviews as well.
>> > > >> >> > > >
>> > > >> >> > > > Here is the email thread for reference
>> > > >> >> > > >
>> > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
>> > > >> >> > > >
>> > > >> >> > > > Regards,
>> > > >> >> > > > Saikat
>> > > >> >> > > >
>> > > >> >> > > >
>> > > >> >> > > >
>> > > >> >> > > >
>> > > >> >> > > >
>> > > >> >> > > >
>> > > >> >> > > >
>> > > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <
>> > > >> >> saikat.maitra@gmail.com
>> > > >> >> > >
>> > > >> >> > > > wrote:
>> > > >> >> > > >
>> > > >> >> > > > > Hello,
>> > > >> >> > > > >
>> > > >> >> > > > > As discussed I have created following discussion
>> threads on
>> > > our
>> > > >> >> > > migration
>> > > >> >> > > > > proposals for Apache Ignite extensions:
>> > > >> >> > > > >
>> > > >> >> > > > >
>> > > >> >> > > > >
>> > > >> >> > > >
>> > > >> >> > >
>> > > >> >> >
>> > > >> >>
>> > > >>
>> > >
>> >
>> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
>> > > >> >> > > > >
>> > > >> >> > > > >
>> > > >> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
>> > > >> >> > > > >
>> > > >> >> > > > > I will share update as I hear more information.
>> > > >> >> > > > >
>> > > >> >> > > > > Regards,
>> > > >> >> > > > > Saikat
>> > > >> >> > > > >
>> > > >> >> > > > >
>> > > >> >> > > > >
>> > > >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra <
>> > > >> >> > saikat.maitra@gmail.com
>> > > >> >> > > >
>> > > >> >> > > > > wrote:
>> > > >> >> > > > >
>> > > >> >> > > > >> Hello Denis,
>> > > >> >> > > > >>
>> > > >> >> > > > >> Sure, sounds good. I will create a separate discussion
>> > > thread
>> > > >> to
>> > > >> >> get
>> > > >> >> > > > >> community feedback on whether we
>> > > >> >> > > > >> should join the Bahir or kick-off "Ignite Extensions"
>> > > project.
>> > > >> >> > > > >>
>> > > >> >> > > > >> Regards,
>> > > >> >> > > > >> Saikat
>> > > >> >> > > > >>
>> > > >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda <
>> > > >> dmagda@apache.org>
>> > > >> >> > > wrote:
>> > > >> >> > > > >>
>> > > >> >> > > > >>> Folks,
>> > > >> >> > > > >>>
>> > > >> >> > > > >>> The concept of Apache Bahir is precisely what we need!
>> > > >> Saikat,
>> > > >> >> > thanks
>> > > >> >> > > > for
>> > > >> >> > > > >>> doing the research. Why I believe the * idea* fits us
>> > well:
>> > > >> >> > > > >>>
>> > > >> >> > > > >>>    - All integrations can be stored in separate Github
>> > > >> >> repositories
>> > > >> >> > > and
>> > > >> >> > > > >>>    have their dev lifecycles. We've not obliged to
>> couple
>> > > all
>> > > >> >> the
>> > > >> >> > > > >>> integrations
>> > > >> >> > > > >>>    in a single repo. For instance, Spark can be
>> located
>> > in
>> > > a
>> > > >> >> > > dedicated
>> > > >> >> > > > >>> repo
>> > > >> >> > > > >>>    while streaming integrations might be in a single
>> one.
>> > > >> It's
>> > > >> >> up
>> > > >> >> > to
>> > > >> >> > > > us.
>> > > >> >> > > > >>>    - All the repositories and integrations belong to
>> ASF.
>> > > >> We're
>> > > >> >> not
>> > > >> >> > > > >>> dumping
>> > > >> >> > > > >>>    them on Github but rather keep supporting and
>> > developing
>> > > >> in
>> > > >> >> > > > >>> accordance with
>> > > >> >> > > > >>>    ASF vision and practices.
>> > > >> >> > > > >>>    - There is a way to reward contributors via
>> > > committership
>> > > >> and
>> > > >> >> > PMC
>> > > >> >> > > > >>>    membership. You won't get this if a project is just
>> > one
>> > > of
>> > > >> >> the
>> > > >> >> > > > >>> millions of
>> > > >> >> > > > >>>    Github projects.
>> > > >> >> > > > >>>
>> > > >> >> > > > >>> Saikat, as Ignite PMC member, could you please
>> kick-off a
>> > > >> >> separate
>> > > >> >> > > > >>> dev-list
>> > > >> >> > > > >>> discussion to involve more community members and
>> > referring
>> > > to
>> > > >> >> this
>> > > >> >> > > > >>> thread?
>> > > >> >> > > > >>> I think the primary question the community needs to
>> > answer
>> > > >> >> whether
>> > > >> >> > we
>> > > >> >> > > > >>> should join the Bahir or kick-off "Ignite Extensions"
>> > > >> project?
>> > > >> >> > > > >>>
>> > > >> >> > > > >>> -
>> > > >> >> > > > >>> Denis
>> > > >> >> > > > >>>
>> > > >> >> > > > >>>
>> > > >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev <
>> > > >> >> > > > zaleslaw.sin@gmail.com>
>> > > >> >> > > > >>> wrote:
>> > > >> >> > > > >>>
>> > > >> >> > > > >>> > Maybe we could move all our Streaming Integrations
>> > there,
>> > > >> but
>> > > >> >> > what
>> > > >> >> > > is
>> > > >> >> > > > >>> about
>> > > >> >> > > > >>> > maintaining and committer permissions to the new
>> > > >> repositories?
>> > > >> >> > > > >>> > I see the list of the committers and PMC members
>> there
>> > > >> >> > > > >>> > https://bahir.apache.org/community-members/
>> > > >> >> > > > >>> >
>> > > >> >> > > > >>> > Could somebody from Ignite community be added to
>> this
>> > > list
>> > > >> as
>> > > >> >> a
>> > > >> >> > > > >>> > PMC/committer to maintain Ignite integrations?
>> > > >> >> > > > >>> > If yes, I'd happy to join this project and
>> collaborate
>> > > with
>> > > >> >> these
>> > > >> >> > > > guys
>> > > >> >> > > > >>> > together
>> > > >> >> > > > >>> > If no, and we should start from the zero level with
>> > > >> external
>> > > >> >> PRs
>> > > >> >> > -
>> > > >> >> > > > >>> hmmm,
>> > > >> >> > > > >>> > it's better to have our own external repository with
>> > > >> >> ApacheBahirr
>> > > >> >> > > > >>> ideology.
>> > > >> >> > > > >>> >
>> > > >> >> > > > >>> > Also, I agree, that all connectors could be moved
>> there
>> > > (in
>> > > >> >> > > > >>> ApacheBahirr
>> > > >> >> > > > >>> > like repository), but not all modules from the page
>> > > >> >> > > > >>> >
>> > > >> >> > > > >>> >
>> > > >> >> > > > >>>
>> > > >> >> > > >
>> > > >> >> > >
>> > > >> >> >
>> > > >> >>
>> > > >>
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> > > >> >> > > > >>> > because
>> > > >> >> > > > >>> > they use different parts of core modules and could
>> be
>> > > >> >> developed
>> > > >> >> > > with
>> > > >> >> > > > >>> > different velocity.
>> > > >> >> > > > >>> >
>> > > >> >> > > > >>> > In reality, before creation of new repositories we
>> need
>> > > >> wide
>> > > >> >> > > > discussion
>> > > >> >> > > > >>> > based on the document mentioned above.
>> > > >> >> > > > >>> >
>> > > >> >> > > > >>> > P.S. Of course, we could experiment in the next
>> release
>> > > 2.8
>> > > >> >> with
>> > > >> >> > > one
>> > > >> >> > > > >>> or two
>> > > >> >> > > > >>> > integrations like twitter or ZeroMQ
>> > > >> >> > > > >>> > Also, I'd like Denis Magda idea for the separate
>> > > >> repositories
>> > > >> >> in
>> > > >> >> > > > Apache
>> > > >> >> > > > >>> > Github to maintain them by anybody.
>> > > >> >> > > > >>> >
>> > > >> >> > > > >>> >
>> > > >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra <
>> > > >> >> > > saikat.maitra@gmail.com
>> > > >> >> > > > >:
>> > > >> >> > > > >>> >
>> > > >> >> > > > >>> > > Hi Denis, Emmanouil
>> > > >> >> > > > >>> > >
>> > > >> >> > > > >>> > > Thank you for your email. I think creating a
>> separate
>> > > >> >> > integration
>> > > >> >> > > > >>> project
>> > > >> >> > > > >>> > > in github and also proposing it as an apache
>> > incubator
>> > > >> >> project
>> > > >> >> > > will
>> > > >> >> > > > >>> be a
>> > > >> >> > > > >>> > > good move. The other separate integration modules
>> > > >> >> > > > >>> > >
>> > > >> >> > > > >>> > >
>> > > >> >> > > > >>> >
>> > > >> >> > > > >>>
>> > > >> >> > > >
>> > > >> >> > >
>> > > >> >> >
>> > > >> >>
>> > > >>
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> > > >> >> > > > >>> > > can
>> > > >> >> > > > >>> > > be moved to this new apache incubator project.
>> > > >> >> > > > >>> > >
>> > > >> >> > > > >>> > > There are similar integration available for Flink
>> and
>> > > >> Spark
>> > > >> >> in
>> > > >> >> > > > Apache
>> > > >> >> > > > >>> > Bahir
>> > > >> >> > > > >>> > > https://bahir.apache.org/
>> > > >> >> > > > >>> > >
>> > > >> >> > > > >>> > > Do you think we should reach out to Apache Bahir
>> > > >> community
>> > > >> >> > with a
>> > > >> >> > > > >>> > proposal
>> > > >> >> > > > >>> > > or we should plan to create a separate Apache
>> > Incubator
>> > > >> >> > project?
>> > > >> >> > > > >>> > >
>> > > >> >> > > > >>> > > Regards,
>> > > >> >> > > > >>> > > Saikat
>> > > >> >> > > > >>> > >
>> > > >> >> > > > >>> > >
>> > > >> >> > > > >>> > >
>> > > >> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil
>> > Gkatziouras <
>> > > >> >> > > > >>> > > gkatzioura@gmail.com>
>> > > >> >> > > > >>> > > wrote:
>> > > >> >> > > > >>> > >
>> > > >> >> > > > >>> > > > Hi all!
>> > > >> >> > > > >>> > > >
>> > > >> >> > > > >>> > > > Based on the discussions most probably this is
>> > going
>> > > to
>> > > >> >> be on
>> > > >> >> > > > >>> another
>> > > >> >> > > > >>> > > > GitHub repo. Therefore I will prepare a
>> standalone
>> > > >> project
>> > > >> >> > with
>> > > >> >> > > > the
>> > > >> >> > > > >>> > > Pub/Sub
>> > > >> >> > > > >>> > > > module and once the repository is created a pull
>> > > >> request
>> > > >> >> > shall
>> > > >> >> > > be
>> > > >> >> > > > >>> > issued.
>> > > >> >> > > > >>> > > > If there is anything else I can do in order to
>> > > >> facilitate
>> > > >> >> > this
>> > > >> >> > > > >>> please
>> > > >> >> > > > >>> > let
>> > > >> >> > > > >>> > > > me know.
>> > > >> >> > > > >>> > > >
>> > > >> >> > > > >>> > > > If a ticket is created for this issue, I shall
>> be
>> > > able
>> > > >> to
>> > > >> >> > > create
>> > > >> >> > > > a
>> > > >> >> > > > >>> pull
>> > > >> >> > > > >>> > > > request and thus a build will take place on
>> > ignite's
>> > > >> CI.
>> > > >> >> > > > >>> > > > If everyone is aligned with adding a Pub/Sub
>> > > >> >> implementation
>> > > >> >> > > may I
>> > > >> >> > > > >>> > create
>> > > >> >> > > > >>> > > a
>> > > >> >> > > > >>> > > > ticket on JIRA?
>> > > >> >> > > > >>> > > >
>> > > >> >> > > > >>> > > > My next question has to do on how Ignite
>> configures
>> > > the
>> > > >> >> build
>> > > >> >> > > > jobs
>> > > >> >> > > > >>> in
>> > > >> >> > > > >>> > > Team
>> > > >> >> > > > >>> > > > City. The unit tests for my implementation need
>> a
>> > > local
>> > > >> >> > Pub/Sub
>> > > >> >> > > > >>> server
>> > > >> >> > > > >>> > up
>> > > >> >> > > > >>> > > > and running (they provide one for testing
>> > purposes).
>> > > >> Will
>> > > >> >> > this
>> > > >> >> > > > take
>> > > >> >> > > > >>> > > effect
>> > > >> >> > > > >>> > > > in a form of a build script inside my
>> > implementation
>> > > or
>> > > >> >> is it
>> > > >> >> > > > >>> something
>> > > >> >> > > > >>> > > > that should be configured on Team City?
>> > > >> >> > > > >>> > > >
>> > > >> >> > > > >>> > > > Kind regards,
>> > > >> >> > > > >>> > > > Emmanouil
>> > > >> >> > > > >>> > > >
>> > > >> >> > > > >>> > > >
>> > > >> >> > > > >>> > > > *Emmanouil Gkatziouras*
>> > > >> >> > > > >>> > > > https://egkatzioura.com/ |
>> > > >> >> > > > >>> > > >
>> https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > >> >> > > > >>> > > > https://github.com/gkatzioura
>> > > >> >> > > > >>> > > >
>> > > >> >> > > > >>> > > >
>> > > >> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis Magda <
>> > > >> >> dmagda@apache.org
>> > > >> >> > >
>> > > >> >> > > > >>> wrote:
>> > > >> >> > > > >>> > > >
>> > > >> >> > > > >>> > > > > Emmanouil, Saikat,
>> > > >> >> > > > >>> > > > >
>> > > >> >> > > > >>> > > > > After contemplating on this, let me propose
>> > another
>> > > >> way
>> > > >> >> > with
>> > > >> >> > > > this
>> > > >> >> > > > >>> > > > > particular integration that is aligned with
>> our
>> > > >> >> > > modularization
>> > > >> >> > > > >>> > > endeavor.
>> > > >> >> > > > >>> > > > > The modularization [1] defines the Ignite
>> core,
>> > > >> Ignite
>> > > >> >> > > modules,
>> > > >> >> > > > >>> and
>> > > >> >> > > > >>> > > > > independent integrations. The first two are
>> to be
>> > > >> >> located
>> > > >> >> > in
>> > > >> >> > > > >>> Ignite
>> > > >> >> > > > >>> > > > > repositories and to be tested/updated by the
>> > > >> community
>> > > >> >> for
>> > > >> >> > > > every
>> > > >> >> > > > >>> > > release.
>> > > >> >> > > > >>> > > > > The modules are a subset of existing
>> integrations
>> > > >> that
>> > > >> >> are
>> > > >> >> > > > >>> adopted
>> > > >> >> > > > >>> > > widely
>> > > >> >> > > > >>> > > > > and require community attention.
>> > > >> >> > > > >>> > > > >
>> > > >> >> > > > >>> > > > > As for the independent integrations [1], those
>> > are
>> > > >> >> supposed
>> > > >> >> > > to
>> > > >> >> > > > be
>> > > >> >> > > > >>> > > located
>> > > >> >> > > > >>> > > > > in a separate GitHub repo. I would advise us
>> to
>> > > start
>> > > >> >> > > > >>> implementing
>> > > >> >> > > > >>> > this
>> > > >> >> > > > >>> > > > > concept with the Pub/Sub integration. Even
>> though
>> > > it
>> > > >> >> will
>> > > >> >> > be
>> > > >> >> > > > >>> located
>> > > >> >> > > > >>> > > in a
>> > > >> >> > > > >>> > > > > separate Github repo, it will be listed among
>> > > >> officially
>> > > >> >> > > > >>> available
>> > > >> >> > > > >>> > > > > integrations on the Ignite website featuring
>> > > >> Emmanouil
>> > > >> >> as a
>> > > >> >> > > > >>> > > contributor.
>> > > >> >> > > > >>> > > > >
>> > > >> >> > > > >>> > > > > What do you think?
>> > > >> >> > > > >>> > > > >
>> > > >> >> > > > >>> > > > >
>> > > >> >> > > > >>> > > > > [1]
>> > > >> >> > > > >>> > > > >
>> > > >> >> > > > >>> > > > >
>> > > >> >> > > > >>> > > >
>> > > >> >> > > > >>> > >
>> > > >> >> > > > >>> >
>> > > >> >> > > > >>>
>> > > >> >> > > >
>> > > >> >> > >
>> > > >> >> >
>> > > >> >>
>> > > >>
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> > > >> >> > > > >>> > > > >
>> > > >> >> > > > >>> > > > >
>> > > >> >> > > > >>> > > > >
>> > > >> >> > > > >>> > > > > -
>> > > >> >> > > > >>> > > > > Denis
>> > > >> >> > > > >>> > > > >
>> > > >> >> > > > >>> > > > >
>> > > >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM Saikat Maitra
>> <
>> > > >> >> > > > >>> > saikat.maitra@gmail.com
>> > > >> >> > > > >>> > > >
>> > > >> >> > > > >>> > > > > wrote:
>> > > >> >> > > > >>> > > > >
>> > > >> >> > > > >>> > > > > > Hi Emmanouil,
>> > > >> >> > > > >>> > > > > >
>> > > >> >> > > > >>> > > > > > The changes looks good to me. I wanted to
>> check
>> > > if
>> > > >> you
>> > > >> >> > are
>> > > >> >> > > > also
>> > > >> >> > > > >>> > able
>> > > >> >> > > > >>> > > to
>> > > >> >> > > > >>> > > > > > validate that the once data is added to
>> Ignite
>> > > >> cluster
>> > > >> >> > you
>> > > >> >> > > > are
>> > > >> >> > > > >>> also
>> > > >> >> > > > >>> > > > able
>> > > >> >> > > > >>> > > > > to
>> > > >> >> > > > >>> > > > > > access it using another Ignite client or
>> using
>> > > rest
>> > > >> >> > > > endpoints.
>> > > >> >> > > > >>> > > > > >
>> > > >> >> > > > >>> > > > > > We use Teamcity for CI process, the details
>> are
>> > > as
>> > > >> >> > follows
>> > > >> >> > > > >>> > > > > >
>> > > >> >> > > > >>> > > > > > Trigger validation of those test suites that
>> > have
>> > > >> been
>> > > >> >> > > > >>> affected by
>> > > >> >> > > > >>> > > your
>> > > >> >> > > > >>> > > > > > changes on TeamCity <
>> > > http://ci.ignite.apache.org/>:
>> > > >> <
>> > > >> >> > > > >>> > > > > http://204.14.53.153/
>> > > >> >> > > > >>> > > > > > >
>> > > >> >> > > > >>> > > > > >
>> > > >> >> > > > >>> > > > > >    - Locate a test suite you have to check,
>> > press
>> > > >> >> button
>> > > >> >> > > > named
>> > > >> >> > > > >>> > "..."
>> > > >> >> > > > >>> > > > that
>> > > >> >> > > > >>> > > > > >    is located on the left of "Run" button.
>> "Run
>> > > >> custom
>> > > >> >> > > build"
>> > > >> >> > > > >>> > dialog
>> > > >> >> > > > >>> > > > will
>> > > >> >> > > > >>> > > > > >    appear;
>> > > >> >> > > > >>> > > > > >    - Go to "Changes" tab and choose
>> > > >> >> > > > >>> > "pull/<pull-request-number>/head"
>> > > >> >> > > > >>> > > > in
>> > > >> >> > > > >>> > > > > >    "Build branch" dropdown list;
>> > > >> >> > > > >>> > > > > >    - Press "Run build" button and monitor
>> tests
>> > > >> >> results.
>> > > >> >> > > > >>> > > > > >
>> > > >> >> > > > >>> > > > > >
>> > > >> >> > > > >>> > > > > >
>> > > >> >> > > > >>> > > > > > @Roman rshtykh@yahoo.com - can you also
>> please
>> > > >> take a
>> > > >> >> > > look?
>> > > >> >> > > > >>> > > > > >
>> > > >> >> > > > >>> > > > > > Regards,
>> > > >> >> > > > >>> > > > > > Saikat
>> > > >> >> > > > >>> > > > > >
>> > > >> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20 PM Emmanouil
>> > > >> Gkatziouras
>> > > >> >> <
>> > > >> >> > > > >>> > > > > > gkatzioura@gmail.com>
>> > > >> >> > > > >>> > > > > > wrote:
>> > > >> >> > > > >>> > > > > >
>> > > >> >> > > > >>> > > > > > > Hi @Saikat Maitra
>> > > >> >> > > > >>> > > > > > >
>> > > >> >> > > > >>> > > > > > > I have my implementation on an Ignite
>> fork on
>> > > my
>> > > >> >> > personal
>> > > >> >> > > > >>> GitHub
>> > > >> >> > > > >>> > > > > account
>> > > >> >> > > > >>> > > > > > > [1]
>> > > >> >> > > > >>> > > > > > > It is based on the Kafka implementation
>> which
>> > > is
>> > > >> of
>> > > >> >> a
>> > > >> >> > > > polling
>> > > >> >> > > > >>> > > nature.
>> > > >> >> > > > >>> > > > > > > GCP does not provide a service like Kafka.
>> > > >> Instead
>> > > >> >> it
>> > > >> >> > > > >>> provides us
>> > > >> >> > > > >>> > > > with
>> > > >> >> > > > >>> > > > > > > Pub/Sub which is just like Kafka.
>> > > >> >> > > > >>> > > > > > > Therefore using the Kafka streamer as a
>> > > guideline
>> > > >> >> made
>> > > >> >> > > more
>> > > >> >> > > > >>> sense
>> > > >> >> > > > >>> > > to
>> > > >> >> > > > >>> > > > > me.
>> > > >> >> > > > >>> > > > > > >
>> > > >> >> > > > >>> > > > > > > Apart from the implementation I have also
>> > > added a
>> > > >> >> unit
>> > > >> >> > > > test.
>> > > >> >> > > > >>> > > > > > > It definitely needs some polishing and a
>> > double
>> > > >> >> check
>> > > >> >> > on
>> > > >> >> > > my
>> > > >> >> > > > >>> side
>> > > >> >> > > > >>> > to
>> > > >> >> > > > >>> > > > > make
>> > > >> >> > > > >>> > > > > > > the review smooth.
>> > > >> >> > > > >>> > > > > > > For the tests a local Pub/Sub [2] server
>> has
>> > to
>> > > >> be
>> > > >> >> > setup
>> > > >> >> > > > >>> however
>> > > >> >> > > > >>> > I
>> > > >> >> > > > >>> > > am
>> > > >> >> > > > >>> > > > > not
>> > > >> >> > > > >>> > > > > > > sure which is the best way to do so with
>> > > regards
>> > > >> to
>> > > >> >> > your
>> > > >> >> > > > >>> CI/CD.
>> > > >> >> > > > >>> > > > > > >
>> > > >> >> > > > >>> > > > > > > [1]
>> > > >> >> > > > >>> > >
>> > > >> >> >
>> https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
>> > > >> >> > > > >>> > > > > > > [2]
>> > > >> >> > > > >>>
>> > https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
>> > > >> >> > > > >>> > > > > > >
>> > > >> >> > > > >>> > > > > > > *Emmanouil Gkatziouras*
>> > > >> >> > > > >>> > > > > > > https://egkatzioura.com/ |
>> > > >> >> > > > >>> > > > > > >
>> > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > >> >> > > > >>> > > > > > > https://github.com/gkatzioura
>> > > >> >> > > > >>> > > > > > >
>> > > >> >> > > > >>> > > > > > >
>> > > >> >> > > > >>> > > > > > > On Sat, 12 Oct 2019 at 02:11, Saikat
>> Maitra <
>> > > >> >> > > > >>> > > saikat.maitra@gmail.com
>> > > >> >> > > > >>> > > > >
>> > > >> >> > > > >>> > > > > > > wrote:
>> > > >> >> > > > >>> > > > > > >
>> > > >> >> > > > >>> > > > > > > > Hello Emmanouil,
>> > > >> >> > > > >>> > > > > > > >
>> > > >> >> > > > >>> > > > > > > > Can you please share more info about the
>> > > >> >> > > implementation?
>> > > >> >> > > > >>> > > > > > > >
>> > > >> >> > > > >>> > > > > > > > We have similar implementation for data
>> > > >> streamers
>> > > >> >> > with
>> > > >> >> > > > >>> Kafka
>> > > >> >> > > > >>> > and
>> > > >> >> > > > >>> > > > > > RocketMQ
>> > > >> >> > > > >>> > > > > > > >
>> > > >> >> > > > >>> > > > > > > >
>> > > >> >> > https://apacheignite-mix.readme.io/docs/kafka-streamer
>> > > >> >> > > > >>> > > > > > > >
>> > > >> >> > > > >>> > > > > > > >
>> > > >> >> > > > https://apacheignite-mix.readme.io/docs/rocketmq-streamer
>> > > >> >> > > > >>> > > > > > > >
>> > > >> >> > > > >>> > > > > > > > Please review and share if you have any
>> > > >> questions.
>> > > >> >> > > > >>> > > > > > > >
>> > > >> >> > > > >>> > > > > > > > Regards,
>> > > >> >> > > > >>> > > > > > > > Saikat
>> > > >> >> > > > >>> > > > > > > >
>> > > >> >> > > > >>> > > > > > > >
>> > > >> >> > > > >>> > > > > > > >
>> > > >> >> > > > >>> > > > > > > > On Fri, Oct 11, 2019 at 4:56 PM
>> Emmanouil
>> > > >> >> > Gkatziouras <
>> > > >> >> > > > >>> > > > > > > > gkatzioura@gmail.com>
>> > > >> >> > > > >>> > > > > > > > wrote:
>> > > >> >> > > > >>> > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > Hi all,
>> > > >> >> > > > >>> > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > By mentioning Pub/Sub I mean Google
>> > Cloud's
>> > > >> >> Pub/Sub
>> > > >> >> > > [1]
>> > > >> >> > > > >>> > > > > > > > > Yes it is indeed cloud specific. It is
>> > the
>> > > >> >> > messaging
>> > > >> >> > > > >>> service
>> > > >> >> > > > >>> > > > > provided
>> > > >> >> > > > >>> > > > > > > by
>> > > >> >> > > > >>> > > > > > > > > GCP.
>> > > >> >> > > > >>> > > > > > > > > I have made the integration with it,
>> but
>> > I
>> > > >> >> haven't
>> > > >> >> > > > >>> published
>> > > >> >> > > > >>> > it
>> > > >> >> > > > >>> > > > yet
>> > > >> >> > > > >>> > > > > > (I
>> > > >> >> > > > >>> > > > > > > am
>> > > >> >> > > > >>> > > > > > > > > waiting for the discussion/ticket and
>> > > proceed
>> > > >> >> with
>> > > >> >> > > the
>> > > >> >> > > > >>> > > > > contribution).
>> > > >> >> > > > >>> > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > Kind regards,
>> > > >> >> > > > >>> > > > > > > > > Emmanouil
>> > > >> >> > > > >>> > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > [1]
>> > > >> >> https://cloud.google.com/pubsub/docs/overview
>> > > >> >> > > > >>> > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > *Emmanouil Gkatziouras*
>> > > >> >> > > > >>> > > > > > > > > https://egkatzioura.com/ |
>> > > >> >> > > > >>> > > > > > > > >
>> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > >> >> > > > >>> > > > > > > > > https://github.com/gkatzioura
>> > > >> >> > > > >>> > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > On Fri, 11 Oct 2019 at 21:45, Denis
>> > Magda <
>> > > >> >> > > > >>> dmagda@apache.org
>> > > >> >> > > > >>> > >
>> > > >> >> > > > >>> > > > > wrote:
>> > > >> >> > > > >>> > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > > Hello Emmanouil,
>> > > >> >> > > > >>> > > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > > Are you referring to any specific
>> > > >> >> implementation
>> > > >> >> > > > >>> tailored
>> > > >> >> > > > >>> > > for a
>> > > >> >> > > > >>> > > > > > > cloud?
>> > > >> >> > > > >>> > > > > > > > If
>> > > >> >> > > > >>> > > > > > > > > > I'm not mistaken you were mentioning
>> > > >> AWS/GCE.
>> > > >> >> Is
>> > > >> >> > > > there
>> > > >> >> > > > >>> any
>> > > >> >> > > > >>> > > > Github
>> > > >> >> > > > >>> > > > > > > > source
>> > > >> >> > > > >>> > > > > > > > > > code?
>> > > >> >> > > > >>> > > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > > Here is a list of existing
>> integrations
>> > > >> with
>> > > >> >> > > > streaming
>> > > >> >> > > > >>> and
>> > > >> >> > > > >>> > > > > pub/sub
>> > > >> >> > > > >>> > > > > > > > > > frameworks:
>> > > >> >> > > > >>> > > > > > > > > >
>> > > >> >> https://apacheignite-mix.readme.io/docs/overview
>> > > >> >> > > > >>> > > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > > with the source code in Ignite
>> master:
>> > > >> >> > > > >>> > > > > > > > > >
>> > > >> >> > > https://github.com/apache/ignite/tree/master/modules
>> > > >> >> > > > >>> > > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > > Check out those resources to see
>> what
>> > are
>> > > >> the
>> > > >> >> > most
>> > > >> >> > > > >>> suitable
>> > > >> >> > > > >>> > > > > Ignite
>> > > >> >> > > > >>> > > > > > > APIs
>> > > >> >> > > > >>> > > > > > > > > for
>> > > >> >> > > > >>> > > > > > > > > > such integrations.
>> > > >> >> > > > >>> > > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > > @Saikat Maitra <
>> > saikat.maitra@gmail.com>
>> > > ,
>> > > >> >> > @Roman
>> > > >> >> > > <
>> > > >> >> > > > >>> > > > > > rshtykh@yahoo.com
>> > > >> >> > > > >>> > > > > > > >
>> > > >> >> > > > >>> > > > > > > > ,
>> > > >> >> > > > >>> > > > > > > > > as
>> > > >> >> > > > >>> > > > > > > > > > streaming technologies
>> > > >> >> contributors/maintainers,
>> > > >> >> > > > could
>> > > >> >> > > > >>> you
>> > > >> >> > > > >>> > > > please
>> > > >> >> > > > >>> > > > > > > join
>> > > >> >> > > > >>> > > > > > > > > this
>> > > >> >> > > > >>> > > > > > > > > > thread?
>> > > >> >> > > > >>> > > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > > -
>> > > >> >> > > > >>> > > > > > > > > > Denis
>> > > >> >> > > > >>> > > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > > On Thu, Oct 10, 2019 at 7:58 AM
>> > Emmanouil
>> > > >> >> > > > Gkatziouras <
>> > > >> >> > > > >>> > > > > > > > > > gkatzioura@gmail.com>
>> > > >> >> > > > >>> > > > > > > > > > wrote:
>> > > >> >> > > > >>> > > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > > > Hello everyone,
>> > > >> >> > > > >>> > > > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > > > I started using Ignite lately.
>> Part
>> > of
>> > > my
>> > > >> >> work
>> > > >> >> > > > >>> involves
>> > > >> >> > > > >>> > > > > Pub/Sub,
>> > > >> >> > > > >>> > > > > > > > thus I
>> > > >> >> > > > >>> > > > > > > > > > > created a pub/sub streamer.
>> > > >> >> > > > >>> > > > > > > > > > > Pub/Sub is pretty close to Kafka.
>> > > >> >> > > > >>> > > > > > > > > > > Here's how the streamer works.
>> Every
>> > > node
>> > > >> >> > creates
>> > > >> >> > > > an
>> > > >> >> > > > >>> > > instance
>> > > >> >> > > > >>> > > > > of
>> > > >> >> > > > >>> > > > > > a
>> > > >> >> > > > >>> > > > > > > > > > Pub/Sub
>> > > >> >> > > > >>> > > > > > > > > > > client.
>> > > >> >> > > > >>> > > > > > > > > > > The clients subscribe to the same
>> > topic
>> > > >> >> using
>> > > >> >> > the
>> > > >> >> > > > >>> same
>> > > >> >> > > > >>> > > > > > subscription
>> > > >> >> > > > >>> > > > > > > > > name.
>> > > >> >> > > > >>> > > > > > > > > > > A message is being sent and one of
>> > the
>> > > >> nodes
>> > > >> >> > will
>> > > >> >> > > > >>> receive
>> > > >> >> > > > >>> > > the
>> > > >> >> > > > >>> > > > > > > > message,
>> > > >> >> > > > >>> > > > > > > > > > > process it and then forward it to
>> > > Ignite
>> > > >> >> using
>> > > >> >> > > the
>> > > >> >> > > > >>> > > extractor
>> > > >> >> > > > >>> > > > > > > > provided.
>> > > >> >> > > > >>> > > > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > > > Any thoughts?
>> > > >> >> > > > >>> > > > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > > > *Emmanouil Gkatziouras*
>> > > >> >> > > > >>> > > > > > > > > > > https://egkatzioura.com/ |
>> > > >> >> > > > >>> > > > > > > > > > >
>> > > >> >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > >> >> > > > >>> > > > > > > > > > > https://github.com/gkatzioura
>> > > >> >> > > > >>> > > > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > > >
>> > > >> >> > > > >>> > > > > > > > >
>> > > >> >> > > > >>> > > > > > > >
>> > > >> >> > > > >>> > > > > > >
>> > > >> >> > > > >>> > > > > >
>> > > >> >> > > > >>> > > > >
>> > > >> >> > > > >>> > > >
>> > > >> >> > > > >>> > >
>> > > >> >> > > > >>> >
>> > > >> >> > > > >>>
>> > > >> >> > > > >>
>> > > >> >> > > >
>> > > >> >> > >
>> > > >> >> >
>> > > >> >>
>> > > >> >
>> > > >>
>> > > >
>> > >
>> >
>>
>

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra <sa...@gmail.com>]

Hi Emmanouil,

Thank you for your email. Yes, the plan is once the Flink PR is merged it
will provide the Licence, parent POMs etc and you can rebase from master
branch and apply the changes on top of it.

I am getting some issues with test failures in local with
GridTestProperties as the test.properties is not present in this new
project but should be available from dependencies.

Once I address this issue, I will go ahead and merge the changes and then
we can take it from there.

Regards,
Saikat

On Fri, Nov 22, 2019 at 5:23 PM Emmanouil Gkatziouras <gk...@gmail.com>
wrote:

> Hi all,
>
> I made my first pull request [1]. Since this project is brand new (no
> parent poms, licensing), I reckoned it was best to use Saikat's branch on
> Flink.
> I suppose the Flink branch will be merged first. If not please give me
> guidelines on how I should proceed next.
>
> Kind regards
> Emmanouil
>
> [1] https://github.com/apache/ignite-extensions/pull/2
>
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>
>
> On Fri, 22 Nov 2019 at 20:55, Denis Magda <dm...@apache.org> wrote:
>
> > Awesome, ping us whenever you're ready!
> >
> > -
> > Denis
> >
> >
> > On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <
> > gkatzioura@gmail.com>
> > wrote:
> >
> > > Hi all!
> > >
> > > I am sorry for being late on that. I was trying to refactor the test in
> > > order not to be in need of any external tools or spinning up a server.
> > > I did forked the new repo and indeed my changes there, so a pull
> request
> > is
> > > a matter of time!
> > >
> > > Kind regards
> > >
> > > *Emmanouil Gkatziouras*
> > > https://egkatzioura.com/ |
> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > https://github.com/gkatzioura
> > >
> > >
> > > On Fri, 22 Nov 2019 at 20:45, Denis Magda <dm...@apache.org> wrote:
> > >
> > > > Hi Emmanouil,
> > > >
> > > > Do you have any questions or need any support from the community?
> > > >
> > > > -
> > > > Denis
> > > >
> > > >
> > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <
> saikat.maitra@gmail.com
> > >
> > > > wrote:
> > > >
> > > >> Hi Emmanouil,
> > > >>
> > > >> Can you please take a looks at dev utils, if this is something you
> are
> > > >> looking for
> > > >> https://github.com/apache/ignite/tree/master/modules/dev-utils?
> > > >>
> > > >> IMO, if you can release Pub/Sub server in maven and then use it as
> > > >> dependency, that would be great.
> > > >>
> > > >> Regards,
> > > >> Saikat
> > > >>
> > > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <
> > saikat.maitra@gmail.com>
> > > >> wrote:
> > > >>
> > > >> > Hi Emmanouil,
> > > >> >
> > > >> > The master branch has been initialized. I have raised a PR to
> > migrate
> > > >> > Flink module to ignite-extensions.
> > > >> >
> > > >> > PR https://github.com/apache/ignite-extensions/pull/1
> > > >> > Jira : https://issues.apache.org/jira/browse/IGNITE-12356
> > > >> >
> > > >> > Regards,
> > > >> > Saikat
> > > >> >
> > > >> >
> > > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
> > > >> > gkatzioura@gmail.com> wrote:
> > > >> >
> > > >> >> Hi all!
> > > >> >>
> > > >> >> This is an update. The integration is tested with mocks based on
> > the
> > > >> >> original Ignite repo [1].
> > > >> >> I am currently waiting for the first commits on the new
> integration
> > > >> >> project
> > > >> >> to take place in order to create my pull request accordingly to
> the
> > > >> >> initial
> > > >> >> commits.
> > > >> >> In order to simulate the Pub/Sub server in a test I used a test
> > > server
> > > >> >> implementation [2]. Unfortunately they don't have this util
> > uploaded
> > > >> in a
> > > >> >> maven repo, thus it needs to be hosted on a maven repo (Apache
> > > License
> > > >> >> 2.0). Is there a repo for utilities that Ignite uses for the
> > > >> test/build?
> > > >> >> I have also created a ticket [3]
> > > >> >>
> > > >> >> Kind regards
> > > >> >>
> > > >> >> [1]
> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > > >> >> [2] https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> > > >> >> [3] https://issues.apache.org/jira/browse/IGNITE-12262
> > > >> >>
> > > >> >> *Emmanouil Gkatziouras*
> > > >> >> https://egkatzioura.com/ |
> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> >> https://github.com/gkatzioura
> > > >> >>
> > > >> >>
> > > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <
> > saikat.maitra@gmail.com
> > > >
> > > >> >> wrote:
> > > >> >>
> > > >> >> > Hello Ilya,
> > > >> >> >
> > > >> >> > Thank you for your email. Yes, before we remove any integration
> > we
> > > >> will
> > > >> >> > move them to another repository and release.
> > > >> >> >
> > > >> >> > We are thinking we will start with Pub/Sub Streamer
> > implementation
> > > >> and
> > > >> >> > continue to move other integration as mentioned in the
> following
> > > doc.
> > > >> >> >
> > > >> >> >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > >> >> >
> > > >> >> > Regards,
> > > >> >> > Saikat
> > > >> >> >
> > > >> >> >
> > > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
> > > >> >> > ilya.kasnacheev@gmail.com>
> > > >> >> > wrote:
> > > >> >> >
> > > >> >> > > Hello!
> > > >> >> > >
> > > >> >> > > My take on this: we want to offload some of integrations, but
> > > >> please
> > > >> >> > commit
> > > >> >> > > and release them to another project first, and only then we
> > will
> > > >> >> decide
> > > >> >> > to
> > > >> >> > > remove them from our repository.
> > > >> >> > >
> > > >> >> > > Please also make sure they are available with same artifact
> > name
> > > >> and
> > > >> >> API.
> > > >> >> > > If this has to be changed, let's wait for AI 3.0
> > > >> >> > >
> > > >> >> > > Regards,
> > > >> >> > > --
> > > >> >> > > Ilya Kasnacheev
> > > >> >> > >
> > > >> >> > >
> > > >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <
> > > >> saikat.maitra@gmail.com>:
> > > >> >> > >
> > > >> >> > > > Hello Emmanouil,
> > > >> >> > > >
> > > >> >> > > > As discussed earlier, I discussed with Apache Bahir
> community
> > > and
> > > >> >> they
> > > >> >> > > are
> > > >> >> > > > interested to have Apache Ignite extensions as part of
> Apache
> > > >> Bahir
> > > >> >> > > > project.
> > > >> >> > > >
> > > >> >> > > > Can you please share Pub/Sub streamer implementation
> details
> > > with
> > > >> >> > Apache
> > > >> >> > > > Bahir community and request for feedback.
> > > >> >> > > >
> > > >> >> > > > https://bahir.apache.org/
> > > >> >> > > >
> > > >> >> > > > I have also requested for contributor access in Jira for
> > Apache
> > > >> >> Bahir
> > > >> >> > > > project so that I can create issues and assign to myself.
> > > >> >> > > >
> > > >> >> > > > I can help with code reviews as well.
> > > >> >> > > >
> > > >> >> > > > Here is the email thread for reference
> > > >> >> > > >
> > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > > >> >> > > >
> > > >> >> > > > Regards,
> > > >> >> > > > Saikat
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <
> > > >> >> saikat.maitra@gmail.com
> > > >> >> > >
> > > >> >> > > > wrote:
> > > >> >> > > >
> > > >> >> > > > > Hello,
> > > >> >> > > > >
> > > >> >> > > > > As discussed I have created following discussion threads
> on
> > > our
> > > >> >> > > migration
> > > >> >> > > > > proposals for Apache Ignite extensions:
> > > >> >> > > > >
> > > >> >> > > > >
> > > >> >> > > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> > > >> >> > > > >
> > > >> >> > > > >
> > > >> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > > >> >> > > > >
> > > >> >> > > > > I will share update as I hear more information.
> > > >> >> > > > >
> > > >> >> > > > > Regards,
> > > >> >> > > > > Saikat
> > > >> >> > > > >
> > > >> >> > > > >
> > > >> >> > > > >
> > > >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra <
> > > >> >> > saikat.maitra@gmail.com
> > > >> >> > > >
> > > >> >> > > > > wrote:
> > > >> >> > > > >
> > > >> >> > > > >> Hello Denis,
> > > >> >> > > > >>
> > > >> >> > > > >> Sure, sounds good. I will create a separate discussion
> > > thread
> > > >> to
> > > >> >> get
> > > >> >> > > > >> community feedback on whether we
> > > >> >> > > > >> should join the Bahir or kick-off "Ignite Extensions"
> > > project.
> > > >> >> > > > >>
> > > >> >> > > > >> Regards,
> > > >> >> > > > >> Saikat
> > > >> >> > > > >>
> > > >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda <
> > > >> dmagda@apache.org>
> > > >> >> > > wrote:
> > > >> >> > > > >>
> > > >> >> > > > >>> Folks,
> > > >> >> > > > >>>
> > > >> >> > > > >>> The concept of Apache Bahir is precisely what we need!
> > > >> Saikat,
> > > >> >> > thanks
> > > >> >> > > > for
> > > >> >> > > > >>> doing the research. Why I believe the * idea* fits us
> > well:
> > > >> >> > > > >>>
> > > >> >> > > > >>>    - All integrations can be stored in separate Github
> > > >> >> repositories
> > > >> >> > > and
> > > >> >> > > > >>>    have their dev lifecycles. We've not obliged to
> couple
> > > all
> > > >> >> the
> > > >> >> > > > >>> integrations
> > > >> >> > > > >>>    in a single repo. For instance, Spark can be located
> > in
> > > a
> > > >> >> > > dedicated
> > > >> >> > > > >>> repo
> > > >> >> > > > >>>    while streaming integrations might be in a single
> one.
> > > >> It's
> > > >> >> up
> > > >> >> > to
> > > >> >> > > > us.
> > > >> >> > > > >>>    - All the repositories and integrations belong to
> ASF.
> > > >> We're
> > > >> >> not
> > > >> >> > > > >>> dumping
> > > >> >> > > > >>>    them on Github but rather keep supporting and
> > developing
> > > >> in
> > > >> >> > > > >>> accordance with
> > > >> >> > > > >>>    ASF vision and practices.
> > > >> >> > > > >>>    - There is a way to reward contributors via
> > > committership
> > > >> and
> > > >> >> > PMC
> > > >> >> > > > >>>    membership. You won't get this if a project is just
> > one
> > > of
> > > >> >> the
> > > >> >> > > > >>> millions of
> > > >> >> > > > >>>    Github projects.
> > > >> >> > > > >>>
> > > >> >> > > > >>> Saikat, as Ignite PMC member, could you please
> kick-off a
> > > >> >> separate
> > > >> >> > > > >>> dev-list
> > > >> >> > > > >>> discussion to involve more community members and
> > referring
> > > to
> > > >> >> this
> > > >> >> > > > >>> thread?
> > > >> >> > > > >>> I think the primary question the community needs to
> > answer
> > > >> >> whether
> > > >> >> > we
> > > >> >> > > > >>> should join the Bahir or kick-off "Ignite Extensions"
> > > >> project?
> > > >> >> > > > >>>
> > > >> >> > > > >>> -
> > > >> >> > > > >>> Denis
> > > >> >> > > > >>>
> > > >> >> > > > >>>
> > > >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev <
> > > >> >> > > > zaleslaw.sin@gmail.com>
> > > >> >> > > > >>> wrote:
> > > >> >> > > > >>>
> > > >> >> > > > >>> > Maybe we could move all our Streaming Integrations
> > there,
> > > >> but
> > > >> >> > what
> > > >> >> > > is
> > > >> >> > > > >>> about
> > > >> >> > > > >>> > maintaining and committer permissions to the new
> > > >> repositories?
> > > >> >> > > > >>> > I see the list of the committers and PMC members
> there
> > > >> >> > > > >>> > https://bahir.apache.org/community-members/
> > > >> >> > > > >>> >
> > > >> >> > > > >>> > Could somebody from Ignite community be added to this
> > > list
> > > >> as
> > > >> >> a
> > > >> >> > > > >>> > PMC/committer to maintain Ignite integrations?
> > > >> >> > > > >>> > If yes, I'd happy to join this project and
> collaborate
> > > with
> > > >> >> these
> > > >> >> > > > guys
> > > >> >> > > > >>> > together
> > > >> >> > > > >>> > If no, and we should start from the zero level with
> > > >> external
> > > >> >> PRs
> > > >> >> > -
> > > >> >> > > > >>> hmmm,
> > > >> >> > > > >>> > it's better to have our own external repository with
> > > >> >> ApacheBahirr
> > > >> >> > > > >>> ideology.
> > > >> >> > > > >>> >
> > > >> >> > > > >>> > Also, I agree, that all connectors could be moved
> there
> > > (in
> > > >> >> > > > >>> ApacheBahirr
> > > >> >> > > > >>> > like repository), but not all modules from the page
> > > >> >> > > > >>> >
> > > >> >> > > > >>> >
> > > >> >> > > > >>>
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > >> >> > > > >>> > because
> > > >> >> > > > >>> > they use different parts of core modules and could be
> > > >> >> developed
> > > >> >> > > with
> > > >> >> > > > >>> > different velocity.
> > > >> >> > > > >>> >
> > > >> >> > > > >>> > In reality, before creation of new repositories we
> need
> > > >> wide
> > > >> >> > > > discussion
> > > >> >> > > > >>> > based on the document mentioned above.
> > > >> >> > > > >>> >
> > > >> >> > > > >>> > P.S. Of course, we could experiment in the next
> release
> > > 2.8
> > > >> >> with
> > > >> >> > > one
> > > >> >> > > > >>> or two
> > > >> >> > > > >>> > integrations like twitter or ZeroMQ
> > > >> >> > > > >>> > Also, I'd like Denis Magda idea for the separate
> > > >> repositories
> > > >> >> in
> > > >> >> > > > Apache
> > > >> >> > > > >>> > Github to maintain them by anybody.
> > > >> >> > > > >>> >
> > > >> >> > > > >>> >
> > > >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra <
> > > >> >> > > saikat.maitra@gmail.com
> > > >> >> > > > >:
> > > >> >> > > > >>> >
> > > >> >> > > > >>> > > Hi Denis, Emmanouil
> > > >> >> > > > >>> > >
> > > >> >> > > > >>> > > Thank you for your email. I think creating a
> separate
> > > >> >> > integration
> > > >> >> > > > >>> project
> > > >> >> > > > >>> > > in github and also proposing it as an apache
> > incubator
> > > >> >> project
> > > >> >> > > will
> > > >> >> > > > >>> be a
> > > >> >> > > > >>> > > good move. The other separate integration modules
> > > >> >> > > > >>> > >
> > > >> >> > > > >>> > >
> > > >> >> > > > >>> >
> > > >> >> > > > >>>
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > >> >> > > > >>> > > can
> > > >> >> > > > >>> > > be moved to this new apache incubator project.
> > > >> >> > > > >>> > >
> > > >> >> > > > >>> > > There are similar integration available for Flink
> and
> > > >> Spark
> > > >> >> in
> > > >> >> > > > Apache
> > > >> >> > > > >>> > Bahir
> > > >> >> > > > >>> > > https://bahir.apache.org/
> > > >> >> > > > >>> > >
> > > >> >> > > > >>> > > Do you think we should reach out to Apache Bahir
> > > >> community
> > > >> >> > with a
> > > >> >> > > > >>> > proposal
> > > >> >> > > > >>> > > or we should plan to create a separate Apache
> > Incubator
> > > >> >> > project?
> > > >> >> > > > >>> > >
> > > >> >> > > > >>> > > Regards,
> > > >> >> > > > >>> > > Saikat
> > > >> >> > > > >>> > >
> > > >> >> > > > >>> > >
> > > >> >> > > > >>> > >
> > > >> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil
> > Gkatziouras <
> > > >> >> > > > >>> > > gkatzioura@gmail.com>
> > > >> >> > > > >>> > > wrote:
> > > >> >> > > > >>> > >
> > > >> >> > > > >>> > > > Hi all!
> > > >> >> > > > >>> > > >
> > > >> >> > > > >>> > > > Based on the discussions most probably this is
> > going
> > > to
> > > >> >> be on
> > > >> >> > > > >>> another
> > > >> >> > > > >>> > > > GitHub repo. Therefore I will prepare a
> standalone
> > > >> project
> > > >> >> > with
> > > >> >> > > > the
> > > >> >> > > > >>> > > Pub/Sub
> > > >> >> > > > >>> > > > module and once the repository is created a pull
> > > >> request
> > > >> >> > shall
> > > >> >> > > be
> > > >> >> > > > >>> > issued.
> > > >> >> > > > >>> > > > If there is anything else I can do in order to
> > > >> facilitate
> > > >> >> > this
> > > >> >> > > > >>> please
> > > >> >> > > > >>> > let
> > > >> >> > > > >>> > > > me know.
> > > >> >> > > > >>> > > >
> > > >> >> > > > >>> > > > If a ticket is created for this issue, I shall be
> > > able
> > > >> to
> > > >> >> > > create
> > > >> >> > > > a
> > > >> >> > > > >>> pull
> > > >> >> > > > >>> > > > request and thus a build will take place on
> > ignite's
> > > >> CI.
> > > >> >> > > > >>> > > > If everyone is aligned with adding a Pub/Sub
> > > >> >> implementation
> > > >> >> > > may I
> > > >> >> > > > >>> > create
> > > >> >> > > > >>> > > a
> > > >> >> > > > >>> > > > ticket on JIRA?
> > > >> >> > > > >>> > > >
> > > >> >> > > > >>> > > > My next question has to do on how Ignite
> configures
> > > the
> > > >> >> build
> > > >> >> > > > jobs
> > > >> >> > > > >>> in
> > > >> >> > > > >>> > > Team
> > > >> >> > > > >>> > > > City. The unit tests for my implementation need a
> > > local
> > > >> >> > Pub/Sub
> > > >> >> > > > >>> server
> > > >> >> > > > >>> > up
> > > >> >> > > > >>> > > > and running (they provide one for testing
> > purposes).
> > > >> Will
> > > >> >> > this
> > > >> >> > > > take
> > > >> >> > > > >>> > > effect
> > > >> >> > > > >>> > > > in a form of a build script inside my
> > implementation
> > > or
> > > >> >> is it
> > > >> >> > > > >>> something
> > > >> >> > > > >>> > > > that should be configured on Team City?
> > > >> >> > > > >>> > > >
> > > >> >> > > > >>> > > > Kind regards,
> > > >> >> > > > >>> > > > Emmanouil
> > > >> >> > > > >>> > > >
> > > >> >> > > > >>> > > >
> > > >> >> > > > >>> > > > *Emmanouil Gkatziouras*
> > > >> >> > > > >>> > > > https://egkatzioura.com/ |
> > > >> >> > > > >>> > > >
> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> >> > > > >>> > > > https://github.com/gkatzioura
> > > >> >> > > > >>> > > >
> > > >> >> > > > >>> > > >
> > > >> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis Magda <
> > > >> >> dmagda@apache.org
> > > >> >> > >
> > > >> >> > > > >>> wrote:
> > > >> >> > > > >>> > > >
> > > >> >> > > > >>> > > > > Emmanouil, Saikat,
> > > >> >> > > > >>> > > > >
> > > >> >> > > > >>> > > > > After contemplating on this, let me propose
> > another
> > > >> way
> > > >> >> > with
> > > >> >> > > > this
> > > >> >> > > > >>> > > > > particular integration that is aligned with our
> > > >> >> > > modularization
> > > >> >> > > > >>> > > endeavor.
> > > >> >> > > > >>> > > > > The modularization [1] defines the Ignite core,
> > > >> Ignite
> > > >> >> > > modules,
> > > >> >> > > > >>> and
> > > >> >> > > > >>> > > > > independent integrations. The first two are to
> be
> > > >> >> located
> > > >> >> > in
> > > >> >> > > > >>> Ignite
> > > >> >> > > > >>> > > > > repositories and to be tested/updated by the
> > > >> community
> > > >> >> for
> > > >> >> > > > every
> > > >> >> > > > >>> > > release.
> > > >> >> > > > >>> > > > > The modules are a subset of existing
> integrations
> > > >> that
> > > >> >> are
> > > >> >> > > > >>> adopted
> > > >> >> > > > >>> > > widely
> > > >> >> > > > >>> > > > > and require community attention.
> > > >> >> > > > >>> > > > >
> > > >> >> > > > >>> > > > > As for the independent integrations [1], those
> > are
> > > >> >> supposed
> > > >> >> > > to
> > > >> >> > > > be
> > > >> >> > > > >>> > > located
> > > >> >> > > > >>> > > > > in a separate GitHub repo. I would advise us to
> > > start
> > > >> >> > > > >>> implementing
> > > >> >> > > > >>> > this
> > > >> >> > > > >>> > > > > concept with the Pub/Sub integration. Even
> though
> > > it
> > > >> >> will
> > > >> >> > be
> > > >> >> > > > >>> located
> > > >> >> > > > >>> > > in a
> > > >> >> > > > >>> > > > > separate Github repo, it will be listed among
> > > >> officially
> > > >> >> > > > >>> available
> > > >> >> > > > >>> > > > > integrations on the Ignite website featuring
> > > >> Emmanouil
> > > >> >> as a
> > > >> >> > > > >>> > > contributor.
> > > >> >> > > > >>> > > > >
> > > >> >> > > > >>> > > > > What do you think?
> > > >> >> > > > >>> > > > >
> > > >> >> > > > >>> > > > >
> > > >> >> > > > >>> > > > > [1]
> > > >> >> > > > >>> > > > >
> > > >> >> > > > >>> > > > >
> > > >> >> > > > >>> > > >
> > > >> >> > > > >>> > >
> > > >> >> > > > >>> >
> > > >> >> > > > >>>
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > >> >> > > > >>> > > > >
> > > >> >> > > > >>> > > > >
> > > >> >> > > > >>> > > > >
> > > >> >> > > > >>> > > > > -
> > > >> >> > > > >>> > > > > Denis
> > > >> >> > > > >>> > > > >
> > > >> >> > > > >>> > > > >
> > > >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM Saikat Maitra <
> > > >> >> > > > >>> > saikat.maitra@gmail.com
> > > >> >> > > > >>> > > >
> > > >> >> > > > >>> > > > > wrote:
> > > >> >> > > > >>> > > > >
> > > >> >> > > > >>> > > > > > Hi Emmanouil,
> > > >> >> > > > >>> > > > > >
> > > >> >> > > > >>> > > > > > The changes looks good to me. I wanted to
> check
> > > if
> > > >> you
> > > >> >> > are
> > > >> >> > > > also
> > > >> >> > > > >>> > able
> > > >> >> > > > >>> > > to
> > > >> >> > > > >>> > > > > > validate that the once data is added to
> Ignite
> > > >> cluster
> > > >> >> > you
> > > >> >> > > > are
> > > >> >> > > > >>> also
> > > >> >> > > > >>> > > > able
> > > >> >> > > > >>> > > > > to
> > > >> >> > > > >>> > > > > > access it using another Ignite client or
> using
> > > rest
> > > >> >> > > > endpoints.
> > > >> >> > > > >>> > > > > >
> > > >> >> > > > >>> > > > > > We use Teamcity for CI process, the details
> are
> > > as
> > > >> >> > follows
> > > >> >> > > > >>> > > > > >
> > > >> >> > > > >>> > > > > > Trigger validation of those test suites that
> > have
> > > >> been
> > > >> >> > > > >>> affected by
> > > >> >> > > > >>> > > your
> > > >> >> > > > >>> > > > > > changes on TeamCity <
> > > http://ci.ignite.apache.org/>:
> > > >> <
> > > >> >> > > > >>> > > > > http://204.14.53.153/
> > > >> >> > > > >>> > > > > > >
> > > >> >> > > > >>> > > > > >
> > > >> >> > > > >>> > > > > >    - Locate a test suite you have to check,
> > press
> > > >> >> button
> > > >> >> > > > named
> > > >> >> > > > >>> > "..."
> > > >> >> > > > >>> > > > that
> > > >> >> > > > >>> > > > > >    is located on the left of "Run" button.
> "Run
> > > >> custom
> > > >> >> > > build"
> > > >> >> > > > >>> > dialog
> > > >> >> > > > >>> > > > will
> > > >> >> > > > >>> > > > > >    appear;
> > > >> >> > > > >>> > > > > >    - Go to "Changes" tab and choose
> > > >> >> > > > >>> > "pull/<pull-request-number>/head"
> > > >> >> > > > >>> > > > in
> > > >> >> > > > >>> > > > > >    "Build branch" dropdown list;
> > > >> >> > > > >>> > > > > >    - Press "Run build" button and monitor
> tests
> > > >> >> results.
> > > >> >> > > > >>> > > > > >
> > > >> >> > > > >>> > > > > >
> > > >> >> > > > >>> > > > > >
> > > >> >> > > > >>> > > > > > @Roman rshtykh@yahoo.com - can you also
> please
> > > >> take a
> > > >> >> > > look?
> > > >> >> > > > >>> > > > > >
> > > >> >> > > > >>> > > > > > Regards,
> > > >> >> > > > >>> > > > > > Saikat
> > > >> >> > > > >>> > > > > >
> > > >> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20 PM Emmanouil
> > > >> Gkatziouras
> > > >> >> <
> > > >> >> > > > >>> > > > > > gkatzioura@gmail.com>
> > > >> >> > > > >>> > > > > > wrote:
> > > >> >> > > > >>> > > > > >
> > > >> >> > > > >>> > > > > > > Hi @Saikat Maitra
> > > >> >> > > > >>> > > > > > >
> > > >> >> > > > >>> > > > > > > I have my implementation on an Ignite fork
> on
> > > my
> > > >> >> > personal
> > > >> >> > > > >>> GitHub
> > > >> >> > > > >>> > > > > account
> > > >> >> > > > >>> > > > > > > [1]
> > > >> >> > > > >>> > > > > > > It is based on the Kafka implementation
> which
> > > is
> > > >> of
> > > >> >> a
> > > >> >> > > > polling
> > > >> >> > > > >>> > > nature.
> > > >> >> > > > >>> > > > > > > GCP does not provide a service like Kafka.
> > > >> Instead
> > > >> >> it
> > > >> >> > > > >>> provides us
> > > >> >> > > > >>> > > > with
> > > >> >> > > > >>> > > > > > > Pub/Sub which is just like Kafka.
> > > >> >> > > > >>> > > > > > > Therefore using the Kafka streamer as a
> > > guideline
> > > >> >> made
> > > >> >> > > more
> > > >> >> > > > >>> sense
> > > >> >> > > > >>> > > to
> > > >> >> > > > >>> > > > > me.
> > > >> >> > > > >>> > > > > > >
> > > >> >> > > > >>> > > > > > > Apart from the implementation I have also
> > > added a
> > > >> >> unit
> > > >> >> > > > test.
> > > >> >> > > > >>> > > > > > > It definitely needs some polishing and a
> > double
> > > >> >> check
> > > >> >> > on
> > > >> >> > > my
> > > >> >> > > > >>> side
> > > >> >> > > > >>> > to
> > > >> >> > > > >>> > > > > make
> > > >> >> > > > >>> > > > > > > the review smooth.
> > > >> >> > > > >>> > > > > > > For the tests a local Pub/Sub [2] server
> has
> > to
> > > >> be
> > > >> >> > setup
> > > >> >> > > > >>> however
> > > >> >> > > > >>> > I
> > > >> >> > > > >>> > > am
> > > >> >> > > > >>> > > > > not
> > > >> >> > > > >>> > > > > > > sure which is the best way to do so with
> > > regards
> > > >> to
> > > >> >> > your
> > > >> >> > > > >>> CI/CD.
> > > >> >> > > > >>> > > > > > >
> > > >> >> > > > >>> > > > > > > [1]
> > > >> >> > > > >>> > >
> > > >> >> >
> https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > > >> >> > > > >>> > > > > > > [2]
> > > >> >> > > > >>>
> > https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
> > > >> >> > > > >>> > > > > > >
> > > >> >> > > > >>> > > > > > > *Emmanouil Gkatziouras*
> > > >> >> > > > >>> > > > > > > https://egkatzioura.com/ |
> > > >> >> > > > >>> > > > > > >
> > > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> >> > > > >>> > > > > > > https://github.com/gkatzioura
> > > >> >> > > > >>> > > > > > >
> > > >> >> > > > >>> > > > > > >
> > > >> >> > > > >>> > > > > > > On Sat, 12 Oct 2019 at 02:11, Saikat
> Maitra <
> > > >> >> > > > >>> > > saikat.maitra@gmail.com
> > > >> >> > > > >>> > > > >
> > > >> >> > > > >>> > > > > > > wrote:
> > > >> >> > > > >>> > > > > > >
> > > >> >> > > > >>> > > > > > > > Hello Emmanouil,
> > > >> >> > > > >>> > > > > > > >
> > > >> >> > > > >>> > > > > > > > Can you please share more info about the
> > > >> >> > > implementation?
> > > >> >> > > > >>> > > > > > > >
> > > >> >> > > > >>> > > > > > > > We have similar implementation for data
> > > >> streamers
> > > >> >> > with
> > > >> >> > > > >>> Kafka
> > > >> >> > > > >>> > and
> > > >> >> > > > >>> > > > > > RocketMQ
> > > >> >> > > > >>> > > > > > > >
> > > >> >> > > > >>> > > > > > > >
> > > >> >> > https://apacheignite-mix.readme.io/docs/kafka-streamer
> > > >> >> > > > >>> > > > > > > >
> > > >> >> > > > >>> > > > > > > >
> > > >> >> > > > https://apacheignite-mix.readme.io/docs/rocketmq-streamer
> > > >> >> > > > >>> > > > > > > >
> > > >> >> > > > >>> > > > > > > > Please review and share if you have any
> > > >> questions.
> > > >> >> > > > >>> > > > > > > >
> > > >> >> > > > >>> > > > > > > > Regards,
> > > >> >> > > > >>> > > > > > > > Saikat
> > > >> >> > > > >>> > > > > > > >
> > > >> >> > > > >>> > > > > > > >
> > > >> >> > > > >>> > > > > > > >
> > > >> >> > > > >>> > > > > > > > On Fri, Oct 11, 2019 at 4:56 PM Emmanouil
> > > >> >> > Gkatziouras <
> > > >> >> > > > >>> > > > > > > > gkatzioura@gmail.com>
> > > >> >> > > > >>> > > > > > > > wrote:
> > > >> >> > > > >>> > > > > > > >
> > > >> >> > > > >>> > > > > > > > > Hi all,
> > > >> >> > > > >>> > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > By mentioning Pub/Sub I mean Google
> > Cloud's
> > > >> >> Pub/Sub
> > > >> >> > > [1]
> > > >> >> > > > >>> > > > > > > > > Yes it is indeed cloud specific. It is
> > the
> > > >> >> > messaging
> > > >> >> > > > >>> service
> > > >> >> > > > >>> > > > > provided
> > > >> >> > > > >>> > > > > > > by
> > > >> >> > > > >>> > > > > > > > > GCP.
> > > >> >> > > > >>> > > > > > > > > I have made the integration with it,
> but
> > I
> > > >> >> haven't
> > > >> >> > > > >>> published
> > > >> >> > > > >>> > it
> > > >> >> > > > >>> > > > yet
> > > >> >> > > > >>> > > > > > (I
> > > >> >> > > > >>> > > > > > > am
> > > >> >> > > > >>> > > > > > > > > waiting for the discussion/ticket and
> > > proceed
> > > >> >> with
> > > >> >> > > the
> > > >> >> > > > >>> > > > > contribution).
> > > >> >> > > > >>> > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > Kind regards,
> > > >> >> > > > >>> > > > > > > > > Emmanouil
> > > >> >> > > > >>> > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > [1]
> > > >> >> https://cloud.google.com/pubsub/docs/overview
> > > >> >> > > > >>> > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > *Emmanouil Gkatziouras*
> > > >> >> > > > >>> > > > > > > > > https://egkatzioura.com/ |
> > > >> >> > > > >>> > > > > > > > >
> > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> >> > > > >>> > > > > > > > > https://github.com/gkatzioura
> > > >> >> > > > >>> > > > > > > > >
> > > >> >> > > > >>> > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > On Fri, 11 Oct 2019 at 21:45, Denis
> > Magda <
> > > >> >> > > > >>> dmagda@apache.org
> > > >> >> > > > >>> > >
> > > >> >> > > > >>> > > > > wrote:
> > > >> >> > > > >>> > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > > Hello Emmanouil,
> > > >> >> > > > >>> > > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > > Are you referring to any specific
> > > >> >> implementation
> > > >> >> > > > >>> tailored
> > > >> >> > > > >>> > > for a
> > > >> >> > > > >>> > > > > > > cloud?
> > > >> >> > > > >>> > > > > > > > If
> > > >> >> > > > >>> > > > > > > > > > I'm not mistaken you were mentioning
> > > >> AWS/GCE.
> > > >> >> Is
> > > >> >> > > > there
> > > >> >> > > > >>> any
> > > >> >> > > > >>> > > > Github
> > > >> >> > > > >>> > > > > > > > source
> > > >> >> > > > >>> > > > > > > > > > code?
> > > >> >> > > > >>> > > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > > Here is a list of existing
> integrations
> > > >> with
> > > >> >> > > > streaming
> > > >> >> > > > >>> and
> > > >> >> > > > >>> > > > > pub/sub
> > > >> >> > > > >>> > > > > > > > > > frameworks:
> > > >> >> > > > >>> > > > > > > > > >
> > > >> >> https://apacheignite-mix.readme.io/docs/overview
> > > >> >> > > > >>> > > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > > with the source code in Ignite
> master:
> > > >> >> > > > >>> > > > > > > > > >
> > > >> >> > > https://github.com/apache/ignite/tree/master/modules
> > > >> >> > > > >>> > > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > > Check out those resources to see what
> > are
> > > >> the
> > > >> >> > most
> > > >> >> > > > >>> suitable
> > > >> >> > > > >>> > > > > Ignite
> > > >> >> > > > >>> > > > > > > APIs
> > > >> >> > > > >>> > > > > > > > > for
> > > >> >> > > > >>> > > > > > > > > > such integrations.
> > > >> >> > > > >>> > > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > > @Saikat Maitra <
> > saikat.maitra@gmail.com>
> > > ,
> > > >> >> > @Roman
> > > >> >> > > <
> > > >> >> > > > >>> > > > > > rshtykh@yahoo.com
> > > >> >> > > > >>> > > > > > > >
> > > >> >> > > > >>> > > > > > > > ,
> > > >> >> > > > >>> > > > > > > > > as
> > > >> >> > > > >>> > > > > > > > > > streaming technologies
> > > >> >> contributors/maintainers,
> > > >> >> > > > could
> > > >> >> > > > >>> you
> > > >> >> > > > >>> > > > please
> > > >> >> > > > >>> > > > > > > join
> > > >> >> > > > >>> > > > > > > > > this
> > > >> >> > > > >>> > > > > > > > > > thread?
> > > >> >> > > > >>> > > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > > -
> > > >> >> > > > >>> > > > > > > > > > Denis
> > > >> >> > > > >>> > > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > > On Thu, Oct 10, 2019 at 7:58 AM
> > Emmanouil
> > > >> >> > > > Gkatziouras <
> > > >> >> > > > >>> > > > > > > > > > gkatzioura@gmail.com>
> > > >> >> > > > >>> > > > > > > > > > wrote:
> > > >> >> > > > >>> > > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > > > Hello everyone,
> > > >> >> > > > >>> > > > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > > > I started using Ignite lately. Part
> > of
> > > my
> > > >> >> work
> > > >> >> > > > >>> involves
> > > >> >> > > > >>> > > > > Pub/Sub,
> > > >> >> > > > >>> > > > > > > > thus I
> > > >> >> > > > >>> > > > > > > > > > > created a pub/sub streamer.
> > > >> >> > > > >>> > > > > > > > > > > Pub/Sub is pretty close to Kafka.
> > > >> >> > > > >>> > > > > > > > > > > Here's how the streamer works.
> Every
> > > node
> > > >> >> > creates
> > > >> >> > > > an
> > > >> >> > > > >>> > > instance
> > > >> >> > > > >>> > > > > of
> > > >> >> > > > >>> > > > > > a
> > > >> >> > > > >>> > > > > > > > > > Pub/Sub
> > > >> >> > > > >>> > > > > > > > > > > client.
> > > >> >> > > > >>> > > > > > > > > > > The clients subscribe to the same
> > topic
> > > >> >> using
> > > >> >> > the
> > > >> >> > > > >>> same
> > > >> >> > > > >>> > > > > > subscription
> > > >> >> > > > >>> > > > > > > > > name.
> > > >> >> > > > >>> > > > > > > > > > > A message is being sent and one of
> > the
> > > >> nodes
> > > >> >> > will
> > > >> >> > > > >>> receive
> > > >> >> > > > >>> > > the
> > > >> >> > > > >>> > > > > > > > message,
> > > >> >> > > > >>> > > > > > > > > > > process it and then forward it to
> > > Ignite
> > > >> >> using
> > > >> >> > > the
> > > >> >> > > > >>> > > extractor
> > > >> >> > > > >>> > > > > > > > provided.
> > > >> >> > > > >>> > > > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > > > Any thoughts?
> > > >> >> > > > >>> > > > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > > > *Emmanouil Gkatziouras*
> > > >> >> > > > >>> > > > > > > > > > > https://egkatzioura.com/ |
> > > >> >> > > > >>> > > > > > > > > > >
> > > >> >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > >> >> > > > >>> > > > > > > > > > > https://github.com/gkatzioura
> > > >> >> > > > >>> > > > > > > > > > >
> > > >> >> > > > >>> > > > > > > > > >
> > > >> >> > > > >>> > > > > > > > >
> > > >> >> > > > >>> > > > > > > >
> > > >> >> > > > >>> > > > > > >
> > > >> >> > > > >>> > > > > >
> > > >> >> > > > >>> > > > >
> > > >> >> > > > >>> > > >
> > > >> >> > > > >>> > >
> > > >> >> > > > >>> >
> > > >> >> > > > >>>
> > > >> >> > > > >>
> > > >> >> > > >
> > > >> >> > >
> > > >> >> >
> > > >> >>
> > > >> >
> > > >>
> > > >
> > >
> >
>

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Emmanouil Gkatziouras <gk...@gmail.com>]

Hi all,

I made my first pull request [1]. Since this project is brand new (no
parent poms, licensing), I reckoned it was best to use Saikat's branch on
Flink.
I suppose the Flink branch will be merged first. If not please give me
guidelines on how I should proceed next.

Kind regards
Emmanouil

[1] https://github.com/apache/ignite-extensions/pull/2

*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura


On Fri, 22 Nov 2019 at 20:55, Denis Magda <dm...@apache.org> wrote:

> Awesome, ping us whenever you're ready!
>
> -
> Denis
>
>
> On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <
> gkatzioura@gmail.com>
> wrote:
>
> > Hi all!
> >
> > I am sorry for being late on that. I was trying to refactor the test in
> > order not to be in need of any external tools or spinning up a server.
> > I did forked the new repo and indeed my changes there, so a pull request
> is
> > a matter of time!
> >
> > Kind regards
> >
> > *Emmanouil Gkatziouras*
> > https://egkatzioura.com/ |
> > https://www.linkedin.com/in/gkatziourasemmanouil/
> > https://github.com/gkatzioura
> >
> >
> > On Fri, 22 Nov 2019 at 20:45, Denis Magda <dm...@apache.org> wrote:
> >
> > > Hi Emmanouil,
> > >
> > > Do you have any questions or need any support from the community?
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <saikat.maitra@gmail.com
> >
> > > wrote:
> > >
> > >> Hi Emmanouil,
> > >>
> > >> Can you please take a looks at dev utils, if this is something you are
> > >> looking for
> > >> https://github.com/apache/ignite/tree/master/modules/dev-utils?
> > >>
> > >> IMO, if you can release Pub/Sub server in maven and then use it as
> > >> dependency, that would be great.
> > >>
> > >> Regards,
> > >> Saikat
> > >>
> > >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <
> saikat.maitra@gmail.com>
> > >> wrote:
> > >>
> > >> > Hi Emmanouil,
> > >> >
> > >> > The master branch has been initialized. I have raised a PR to
> migrate
> > >> > Flink module to ignite-extensions.
> > >> >
> > >> > PR https://github.com/apache/ignite-extensions/pull/1
> > >> > Jira : https://issues.apache.org/jira/browse/IGNITE-12356
> > >> >
> > >> > Regards,
> > >> > Saikat
> > >> >
> > >> >
> > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
> > >> > gkatzioura@gmail.com> wrote:
> > >> >
> > >> >> Hi all!
> > >> >>
> > >> >> This is an update. The integration is tested with mocks based on
> the
> > >> >> original Ignite repo [1].
> > >> >> I am currently waiting for the first commits on the new integration
> > >> >> project
> > >> >> to take place in order to create my pull request accordingly to the
> > >> >> initial
> > >> >> commits.
> > >> >> In order to simulate the Pub/Sub server in a test I used a test
> > server
> > >> >> implementation [2]. Unfortunately they don't have this util
> uploaded
> > >> in a
> > >> >> maven repo, thus it needs to be hosted on a maven repo (Apache
> > License
> > >> >> 2.0). Is there a repo for utilities that Ignite uses for the
> > >> test/build?
> > >> >> I have also created a ticket [3]
> > >> >>
> > >> >> Kind regards
> > >> >>
> > >> >> [1]
> https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > >> >> [2] https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> > >> >> [3] https://issues.apache.org/jira/browse/IGNITE-12262
> > >> >>
> > >> >> *Emmanouil Gkatziouras*
> > >> >> https://egkatzioura.com/ |
> > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> >> https://github.com/gkatzioura
> > >> >>
> > >> >>
> > >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <
> saikat.maitra@gmail.com
> > >
> > >> >> wrote:
> > >> >>
> > >> >> > Hello Ilya,
> > >> >> >
> > >> >> > Thank you for your email. Yes, before we remove any integration
> we
> > >> will
> > >> >> > move them to another repository and release.
> > >> >> >
> > >> >> > We are thinking we will start with Pub/Sub Streamer
> implementation
> > >> and
> > >> >> > continue to move other integration as mentioned in the following
> > doc.
> > >> >> >
> > >> >> >
> > >> >> >
> > >> >>
> > >>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > >> >> >
> > >> >> > Regards,
> > >> >> > Saikat
> > >> >> >
> > >> >> >
> > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
> > >> >> > ilya.kasnacheev@gmail.com>
> > >> >> > wrote:
> > >> >> >
> > >> >> > > Hello!
> > >> >> > >
> > >> >> > > My take on this: we want to offload some of integrations, but
> > >> please
> > >> >> > commit
> > >> >> > > and release them to another project first, and only then we
> will
> > >> >> decide
> > >> >> > to
> > >> >> > > remove them from our repository.
> > >> >> > >
> > >> >> > > Please also make sure they are available with same artifact
> name
> > >> and
> > >> >> API.
> > >> >> > > If this has to be changed, let's wait for AI 3.0
> > >> >> > >
> > >> >> > > Regards,
> > >> >> > > --
> > >> >> > > Ilya Kasnacheev
> > >> >> > >
> > >> >> > >
> > >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <
> > >> saikat.maitra@gmail.com>:
> > >> >> > >
> > >> >> > > > Hello Emmanouil,
> > >> >> > > >
> > >> >> > > > As discussed earlier, I discussed with Apache Bahir community
> > and
> > >> >> they
> > >> >> > > are
> > >> >> > > > interested to have Apache Ignite extensions as part of Apache
> > >> Bahir
> > >> >> > > > project.
> > >> >> > > >
> > >> >> > > > Can you please share Pub/Sub streamer implementation details
> > with
> > >> >> > Apache
> > >> >> > > > Bahir community and request for feedback.
> > >> >> > > >
> > >> >> > > > https://bahir.apache.org/
> > >> >> > > >
> > >> >> > > > I have also requested for contributor access in Jira for
> Apache
> > >> >> Bahir
> > >> >> > > > project so that I can create issues and assign to myself.
> > >> >> > > >
> > >> >> > > > I can help with code reviews as well.
> > >> >> > > >
> > >> >> > > > Here is the email thread for reference
> > >> >> > > >
> > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > >> >> > > >
> > >> >> > > > Regards,
> > >> >> > > > Saikat
> > >> >> > > >
> > >> >> > > >
> > >> >> > > >
> > >> >> > > >
> > >> >> > > >
> > >> >> > > >
> > >> >> > > >
> > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <
> > >> >> saikat.maitra@gmail.com
> > >> >> > >
> > >> >> > > > wrote:
> > >> >> > > >
> > >> >> > > > > Hello,
> > >> >> > > > >
> > >> >> > > > > As discussed I have created following discussion threads on
> > our
> > >> >> > > migration
> > >> >> > > > > proposals for Apache Ignite extensions:
> > >> >> > > > >
> > >> >> > > > >
> > >> >> > > > >
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> > >> >> > > > >
> > >> >> > > > >
> > >> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > >> >> > > > >
> > >> >> > > > > I will share update as I hear more information.
> > >> >> > > > >
> > >> >> > > > > Regards,
> > >> >> > > > > Saikat
> > >> >> > > > >
> > >> >> > > > >
> > >> >> > > > >
> > >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra <
> > >> >> > saikat.maitra@gmail.com
> > >> >> > > >
> > >> >> > > > > wrote:
> > >> >> > > > >
> > >> >> > > > >> Hello Denis,
> > >> >> > > > >>
> > >> >> > > > >> Sure, sounds good. I will create a separate discussion
> > thread
> > >> to
> > >> >> get
> > >> >> > > > >> community feedback on whether we
> > >> >> > > > >> should join the Bahir or kick-off "Ignite Extensions"
> > project.
> > >> >> > > > >>
> > >> >> > > > >> Regards,
> > >> >> > > > >> Saikat
> > >> >> > > > >>
> > >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda <
> > >> dmagda@apache.org>
> > >> >> > > wrote:
> > >> >> > > > >>
> > >> >> > > > >>> Folks,
> > >> >> > > > >>>
> > >> >> > > > >>> The concept of Apache Bahir is precisely what we need!
> > >> Saikat,
> > >> >> > thanks
> > >> >> > > > for
> > >> >> > > > >>> doing the research. Why I believe the * idea* fits us
> well:
> > >> >> > > > >>>
> > >> >> > > > >>>    - All integrations can be stored in separate Github
> > >> >> repositories
> > >> >> > > and
> > >> >> > > > >>>    have their dev lifecycles. We've not obliged to couple
> > all
> > >> >> the
> > >> >> > > > >>> integrations
> > >> >> > > > >>>    in a single repo. For instance, Spark can be located
> in
> > a
> > >> >> > > dedicated
> > >> >> > > > >>> repo
> > >> >> > > > >>>    while streaming integrations might be in a single one.
> > >> It's
> > >> >> up
> > >> >> > to
> > >> >> > > > us.
> > >> >> > > > >>>    - All the repositories and integrations belong to ASF.
> > >> We're
> > >> >> not
> > >> >> > > > >>> dumping
> > >> >> > > > >>>    them on Github but rather keep supporting and
> developing
> > >> in
> > >> >> > > > >>> accordance with
> > >> >> > > > >>>    ASF vision and practices.
> > >> >> > > > >>>    - There is a way to reward contributors via
> > committership
> > >> and
> > >> >> > PMC
> > >> >> > > > >>>    membership. You won't get this if a project is just
> one
> > of
> > >> >> the
> > >> >> > > > >>> millions of
> > >> >> > > > >>>    Github projects.
> > >> >> > > > >>>
> > >> >> > > > >>> Saikat, as Ignite PMC member, could you please kick-off a
> > >> >> separate
> > >> >> > > > >>> dev-list
> > >> >> > > > >>> discussion to involve more community members and
> referring
> > to
> > >> >> this
> > >> >> > > > >>> thread?
> > >> >> > > > >>> I think the primary question the community needs to
> answer
> > >> >> whether
> > >> >> > we
> > >> >> > > > >>> should join the Bahir or kick-off "Ignite Extensions"
> > >> project?
> > >> >> > > > >>>
> > >> >> > > > >>> -
> > >> >> > > > >>> Denis
> > >> >> > > > >>>
> > >> >> > > > >>>
> > >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev <
> > >> >> > > > zaleslaw.sin@gmail.com>
> > >> >> > > > >>> wrote:
> > >> >> > > > >>>
> > >> >> > > > >>> > Maybe we could move all our Streaming Integrations
> there,
> > >> but
> > >> >> > what
> > >> >> > > is
> > >> >> > > > >>> about
> > >> >> > > > >>> > maintaining and committer permissions to the new
> > >> repositories?
> > >> >> > > > >>> > I see the list of the committers and PMC members there
> > >> >> > > > >>> > https://bahir.apache.org/community-members/
> > >> >> > > > >>> >
> > >> >> > > > >>> > Could somebody from Ignite community be added to this
> > list
> > >> as
> > >> >> a
> > >> >> > > > >>> > PMC/committer to maintain Ignite integrations?
> > >> >> > > > >>> > If yes, I'd happy to join this project and collaborate
> > with
> > >> >> these
> > >> >> > > > guys
> > >> >> > > > >>> > together
> > >> >> > > > >>> > If no, and we should start from the zero level with
> > >> external
> > >> >> PRs
> > >> >> > -
> > >> >> > > > >>> hmmm,
> > >> >> > > > >>> > it's better to have our own external repository with
> > >> >> ApacheBahirr
> > >> >> > > > >>> ideology.
> > >> >> > > > >>> >
> > >> >> > > > >>> > Also, I agree, that all connectors could be moved there
> > (in
> > >> >> > > > >>> ApacheBahirr
> > >> >> > > > >>> > like repository), but not all modules from the page
> > >> >> > > > >>> >
> > >> >> > > > >>> >
> > >> >> > > > >>>
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > >> >> > > > >>> > because
> > >> >> > > > >>> > they use different parts of core modules and could be
> > >> >> developed
> > >> >> > > with
> > >> >> > > > >>> > different velocity.
> > >> >> > > > >>> >
> > >> >> > > > >>> > In reality, before creation of new repositories we need
> > >> wide
> > >> >> > > > discussion
> > >> >> > > > >>> > based on the document mentioned above.
> > >> >> > > > >>> >
> > >> >> > > > >>> > P.S. Of course, we could experiment in the next release
> > 2.8
> > >> >> with
> > >> >> > > one
> > >> >> > > > >>> or two
> > >> >> > > > >>> > integrations like twitter or ZeroMQ
> > >> >> > > > >>> > Also, I'd like Denis Magda idea for the separate
> > >> repositories
> > >> >> in
> > >> >> > > > Apache
> > >> >> > > > >>> > Github to maintain them by anybody.
> > >> >> > > > >>> >
> > >> >> > > > >>> >
> > >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra <
> > >> >> > > saikat.maitra@gmail.com
> > >> >> > > > >:
> > >> >> > > > >>> >
> > >> >> > > > >>> > > Hi Denis, Emmanouil
> > >> >> > > > >>> > >
> > >> >> > > > >>> > > Thank you for your email. I think creating a separate
> > >> >> > integration
> > >> >> > > > >>> project
> > >> >> > > > >>> > > in github and also proposing it as an apache
> incubator
> > >> >> project
> > >> >> > > will
> > >> >> > > > >>> be a
> > >> >> > > > >>> > > good move. The other separate integration modules
> > >> >> > > > >>> > >
> > >> >> > > > >>> > >
> > >> >> > > > >>> >
> > >> >> > > > >>>
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > >> >> > > > >>> > > can
> > >> >> > > > >>> > > be moved to this new apache incubator project.
> > >> >> > > > >>> > >
> > >> >> > > > >>> > > There are similar integration available for Flink and
> > >> Spark
> > >> >> in
> > >> >> > > > Apache
> > >> >> > > > >>> > Bahir
> > >> >> > > > >>> > > https://bahir.apache.org/
> > >> >> > > > >>> > >
> > >> >> > > > >>> > > Do you think we should reach out to Apache Bahir
> > >> community
> > >> >> > with a
> > >> >> > > > >>> > proposal
> > >> >> > > > >>> > > or we should plan to create a separate Apache
> Incubator
> > >> >> > project?
> > >> >> > > > >>> > >
> > >> >> > > > >>> > > Regards,
> > >> >> > > > >>> > > Saikat
> > >> >> > > > >>> > >
> > >> >> > > > >>> > >
> > >> >> > > > >>> > >
> > >> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil
> Gkatziouras <
> > >> >> > > > >>> > > gkatzioura@gmail.com>
> > >> >> > > > >>> > > wrote:
> > >> >> > > > >>> > >
> > >> >> > > > >>> > > > Hi all!
> > >> >> > > > >>> > > >
> > >> >> > > > >>> > > > Based on the discussions most probably this is
> going
> > to
> > >> >> be on
> > >> >> > > > >>> another
> > >> >> > > > >>> > > > GitHub repo. Therefore I will prepare a standalone
> > >> project
> > >> >> > with
> > >> >> > > > the
> > >> >> > > > >>> > > Pub/Sub
> > >> >> > > > >>> > > > module and once the repository is created a pull
> > >> request
> > >> >> > shall
> > >> >> > > be
> > >> >> > > > >>> > issued.
> > >> >> > > > >>> > > > If there is anything else I can do in order to
> > >> facilitate
> > >> >> > this
> > >> >> > > > >>> please
> > >> >> > > > >>> > let
> > >> >> > > > >>> > > > me know.
> > >> >> > > > >>> > > >
> > >> >> > > > >>> > > > If a ticket is created for this issue, I shall be
> > able
> > >> to
> > >> >> > > create
> > >> >> > > > a
> > >> >> > > > >>> pull
> > >> >> > > > >>> > > > request and thus a build will take place on
> ignite's
> > >> CI.
> > >> >> > > > >>> > > > If everyone is aligned with adding a Pub/Sub
> > >> >> implementation
> > >> >> > > may I
> > >> >> > > > >>> > create
> > >> >> > > > >>> > > a
> > >> >> > > > >>> > > > ticket on JIRA?
> > >> >> > > > >>> > > >
> > >> >> > > > >>> > > > My next question has to do on how Ignite configures
> > the
> > >> >> build
> > >> >> > > > jobs
> > >> >> > > > >>> in
> > >> >> > > > >>> > > Team
> > >> >> > > > >>> > > > City. The unit tests for my implementation need a
> > local
> > >> >> > Pub/Sub
> > >> >> > > > >>> server
> > >> >> > > > >>> > up
> > >> >> > > > >>> > > > and running (they provide one for testing
> purposes).
> > >> Will
> > >> >> > this
> > >> >> > > > take
> > >> >> > > > >>> > > effect
> > >> >> > > > >>> > > > in a form of a build script inside my
> implementation
> > or
> > >> >> is it
> > >> >> > > > >>> something
> > >> >> > > > >>> > > > that should be configured on Team City?
> > >> >> > > > >>> > > >
> > >> >> > > > >>> > > > Kind regards,
> > >> >> > > > >>> > > > Emmanouil
> > >> >> > > > >>> > > >
> > >> >> > > > >>> > > >
> > >> >> > > > >>> > > > *Emmanouil Gkatziouras*
> > >> >> > > > >>> > > > https://egkatzioura.com/ |
> > >> >> > > > >>> > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> >> > > > >>> > > > https://github.com/gkatzioura
> > >> >> > > > >>> > > >
> > >> >> > > > >>> > > >
> > >> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis Magda <
> > >> >> dmagda@apache.org
> > >> >> > >
> > >> >> > > > >>> wrote:
> > >> >> > > > >>> > > >
> > >> >> > > > >>> > > > > Emmanouil, Saikat,
> > >> >> > > > >>> > > > >
> > >> >> > > > >>> > > > > After contemplating on this, let me propose
> another
> > >> way
> > >> >> > with
> > >> >> > > > this
> > >> >> > > > >>> > > > > particular integration that is aligned with our
> > >> >> > > modularization
> > >> >> > > > >>> > > endeavor.
> > >> >> > > > >>> > > > > The modularization [1] defines the Ignite core,
> > >> Ignite
> > >> >> > > modules,
> > >> >> > > > >>> and
> > >> >> > > > >>> > > > > independent integrations. The first two are to be
> > >> >> located
> > >> >> > in
> > >> >> > > > >>> Ignite
> > >> >> > > > >>> > > > > repositories and to be tested/updated by the
> > >> community
> > >> >> for
> > >> >> > > > every
> > >> >> > > > >>> > > release.
> > >> >> > > > >>> > > > > The modules are a subset of existing integrations
> > >> that
> > >> >> are
> > >> >> > > > >>> adopted
> > >> >> > > > >>> > > widely
> > >> >> > > > >>> > > > > and require community attention.
> > >> >> > > > >>> > > > >
> > >> >> > > > >>> > > > > As for the independent integrations [1], those
> are
> > >> >> supposed
> > >> >> > > to
> > >> >> > > > be
> > >> >> > > > >>> > > located
> > >> >> > > > >>> > > > > in a separate GitHub repo. I would advise us to
> > start
> > >> >> > > > >>> implementing
> > >> >> > > > >>> > this
> > >> >> > > > >>> > > > > concept with the Pub/Sub integration. Even though
> > it
> > >> >> will
> > >> >> > be
> > >> >> > > > >>> located
> > >> >> > > > >>> > > in a
> > >> >> > > > >>> > > > > separate Github repo, it will be listed among
> > >> officially
> > >> >> > > > >>> available
> > >> >> > > > >>> > > > > integrations on the Ignite website featuring
> > >> Emmanouil
> > >> >> as a
> > >> >> > > > >>> > > contributor.
> > >> >> > > > >>> > > > >
> > >> >> > > > >>> > > > > What do you think?
> > >> >> > > > >>> > > > >
> > >> >> > > > >>> > > > >
> > >> >> > > > >>> > > > > [1]
> > >> >> > > > >>> > > > >
> > >> >> > > > >>> > > > >
> > >> >> > > > >>> > > >
> > >> >> > > > >>> > >
> > >> >> > > > >>> >
> > >> >> > > > >>>
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > >> >> > > > >>> > > > >
> > >> >> > > > >>> > > > >
> > >> >> > > > >>> > > > >
> > >> >> > > > >>> > > > > -
> > >> >> > > > >>> > > > > Denis
> > >> >> > > > >>> > > > >
> > >> >> > > > >>> > > > >
> > >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM Saikat Maitra <
> > >> >> > > > >>> > saikat.maitra@gmail.com
> > >> >> > > > >>> > > >
> > >> >> > > > >>> > > > > wrote:
> > >> >> > > > >>> > > > >
> > >> >> > > > >>> > > > > > Hi Emmanouil,
> > >> >> > > > >>> > > > > >
> > >> >> > > > >>> > > > > > The changes looks good to me. I wanted to check
> > if
> > >> you
> > >> >> > are
> > >> >> > > > also
> > >> >> > > > >>> > able
> > >> >> > > > >>> > > to
> > >> >> > > > >>> > > > > > validate that the once data is added to Ignite
> > >> cluster
> > >> >> > you
> > >> >> > > > are
> > >> >> > > > >>> also
> > >> >> > > > >>> > > > able
> > >> >> > > > >>> > > > > to
> > >> >> > > > >>> > > > > > access it using another Ignite client or using
> > rest
> > >> >> > > > endpoints.
> > >> >> > > > >>> > > > > >
> > >> >> > > > >>> > > > > > We use Teamcity for CI process, the details are
> > as
> > >> >> > follows
> > >> >> > > > >>> > > > > >
> > >> >> > > > >>> > > > > > Trigger validation of those test suites that
> have
> > >> been
> > >> >> > > > >>> affected by
> > >> >> > > > >>> > > your
> > >> >> > > > >>> > > > > > changes on TeamCity <
> > http://ci.ignite.apache.org/>:
> > >> <
> > >> >> > > > >>> > > > > http://204.14.53.153/
> > >> >> > > > >>> > > > > > >
> > >> >> > > > >>> > > > > >
> > >> >> > > > >>> > > > > >    - Locate a test suite you have to check,
> press
> > >> >> button
> > >> >> > > > named
> > >> >> > > > >>> > "..."
> > >> >> > > > >>> > > > that
> > >> >> > > > >>> > > > > >    is located on the left of "Run" button. "Run
> > >> custom
> > >> >> > > build"
> > >> >> > > > >>> > dialog
> > >> >> > > > >>> > > > will
> > >> >> > > > >>> > > > > >    appear;
> > >> >> > > > >>> > > > > >    - Go to "Changes" tab and choose
> > >> >> > > > >>> > "pull/<pull-request-number>/head"
> > >> >> > > > >>> > > > in
> > >> >> > > > >>> > > > > >    "Build branch" dropdown list;
> > >> >> > > > >>> > > > > >    - Press "Run build" button and monitor tests
> > >> >> results.
> > >> >> > > > >>> > > > > >
> > >> >> > > > >>> > > > > >
> > >> >> > > > >>> > > > > >
> > >> >> > > > >>> > > > > > @Roman rshtykh@yahoo.com - can you also please
> > >> take a
> > >> >> > > look?
> > >> >> > > > >>> > > > > >
> > >> >> > > > >>> > > > > > Regards,
> > >> >> > > > >>> > > > > > Saikat
> > >> >> > > > >>> > > > > >
> > >> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20 PM Emmanouil
> > >> Gkatziouras
> > >> >> <
> > >> >> > > > >>> > > > > > gkatzioura@gmail.com>
> > >> >> > > > >>> > > > > > wrote:
> > >> >> > > > >>> > > > > >
> > >> >> > > > >>> > > > > > > Hi @Saikat Maitra
> > >> >> > > > >>> > > > > > >
> > >> >> > > > >>> > > > > > > I have my implementation on an Ignite fork on
> > my
> > >> >> > personal
> > >> >> > > > >>> GitHub
> > >> >> > > > >>> > > > > account
> > >> >> > > > >>> > > > > > > [1]
> > >> >> > > > >>> > > > > > > It is based on the Kafka implementation which
> > is
> > >> of
> > >> >> a
> > >> >> > > > polling
> > >> >> > > > >>> > > nature.
> > >> >> > > > >>> > > > > > > GCP does not provide a service like Kafka.
> > >> Instead
> > >> >> it
> > >> >> > > > >>> provides us
> > >> >> > > > >>> > > > with
> > >> >> > > > >>> > > > > > > Pub/Sub which is just like Kafka.
> > >> >> > > > >>> > > > > > > Therefore using the Kafka streamer as a
> > guideline
> > >> >> made
> > >> >> > > more
> > >> >> > > > >>> sense
> > >> >> > > > >>> > > to
> > >> >> > > > >>> > > > > me.
> > >> >> > > > >>> > > > > > >
> > >> >> > > > >>> > > > > > > Apart from the implementation I have also
> > added a
> > >> >> unit
> > >> >> > > > test.
> > >> >> > > > >>> > > > > > > It definitely needs some polishing and a
> double
> > >> >> check
> > >> >> > on
> > >> >> > > my
> > >> >> > > > >>> side
> > >> >> > > > >>> > to
> > >> >> > > > >>> > > > > make
> > >> >> > > > >>> > > > > > > the review smooth.
> > >> >> > > > >>> > > > > > > For the tests a local Pub/Sub [2] server has
> to
> > >> be
> > >> >> > setup
> > >> >> > > > >>> however
> > >> >> > > > >>> > I
> > >> >> > > > >>> > > am
> > >> >> > > > >>> > > > > not
> > >> >> > > > >>> > > > > > > sure which is the best way to do so with
> > regards
> > >> to
> > >> >> > your
> > >> >> > > > >>> CI/CD.
> > >> >> > > > >>> > > > > > >
> > >> >> > > > >>> > > > > > > [1]
> > >> >> > > > >>> > >
> > >> >> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > >> >> > > > >>> > > > > > > [2]
> > >> >> > > > >>>
> https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
> > >> >> > > > >>> > > > > > >
> > >> >> > > > >>> > > > > > > *Emmanouil Gkatziouras*
> > >> >> > > > >>> > > > > > > https://egkatzioura.com/ |
> > >> >> > > > >>> > > > > > >
> > >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> >> > > > >>> > > > > > > https://github.com/gkatzioura
> > >> >> > > > >>> > > > > > >
> > >> >> > > > >>> > > > > > >
> > >> >> > > > >>> > > > > > > On Sat, 12 Oct 2019 at 02:11, Saikat Maitra <
> > >> >> > > > >>> > > saikat.maitra@gmail.com
> > >> >> > > > >>> > > > >
> > >> >> > > > >>> > > > > > > wrote:
> > >> >> > > > >>> > > > > > >
> > >> >> > > > >>> > > > > > > > Hello Emmanouil,
> > >> >> > > > >>> > > > > > > >
> > >> >> > > > >>> > > > > > > > Can you please share more info about the
> > >> >> > > implementation?
> > >> >> > > > >>> > > > > > > >
> > >> >> > > > >>> > > > > > > > We have similar implementation for data
> > >> streamers
> > >> >> > with
> > >> >> > > > >>> Kafka
> > >> >> > > > >>> > and
> > >> >> > > > >>> > > > > > RocketMQ
> > >> >> > > > >>> > > > > > > >
> > >> >> > > > >>> > > > > > > >
> > >> >> > https://apacheignite-mix.readme.io/docs/kafka-streamer
> > >> >> > > > >>> > > > > > > >
> > >> >> > > > >>> > > > > > > >
> > >> >> > > > https://apacheignite-mix.readme.io/docs/rocketmq-streamer
> > >> >> > > > >>> > > > > > > >
> > >> >> > > > >>> > > > > > > > Please review and share if you have any
> > >> questions.
> > >> >> > > > >>> > > > > > > >
> > >> >> > > > >>> > > > > > > > Regards,
> > >> >> > > > >>> > > > > > > > Saikat
> > >> >> > > > >>> > > > > > > >
> > >> >> > > > >>> > > > > > > >
> > >> >> > > > >>> > > > > > > >
> > >> >> > > > >>> > > > > > > > On Fri, Oct 11, 2019 at 4:56 PM Emmanouil
> > >> >> > Gkatziouras <
> > >> >> > > > >>> > > > > > > > gkatzioura@gmail.com>
> > >> >> > > > >>> > > > > > > > wrote:
> > >> >> > > > >>> > > > > > > >
> > >> >> > > > >>> > > > > > > > > Hi all,
> > >> >> > > > >>> > > > > > > > >
> > >> >> > > > >>> > > > > > > > > By mentioning Pub/Sub I mean Google
> Cloud's
> > >> >> Pub/Sub
> > >> >> > > [1]
> > >> >> > > > >>> > > > > > > > > Yes it is indeed cloud specific. It is
> the
> > >> >> > messaging
> > >> >> > > > >>> service
> > >> >> > > > >>> > > > > provided
> > >> >> > > > >>> > > > > > > by
> > >> >> > > > >>> > > > > > > > > GCP.
> > >> >> > > > >>> > > > > > > > > I have made the integration with it, but
> I
> > >> >> haven't
> > >> >> > > > >>> published
> > >> >> > > > >>> > it
> > >> >> > > > >>> > > > yet
> > >> >> > > > >>> > > > > > (I
> > >> >> > > > >>> > > > > > > am
> > >> >> > > > >>> > > > > > > > > waiting for the discussion/ticket and
> > proceed
> > >> >> with
> > >> >> > > the
> > >> >> > > > >>> > > > > contribution).
> > >> >> > > > >>> > > > > > > > >
> > >> >> > > > >>> > > > > > > > > Kind regards,
> > >> >> > > > >>> > > > > > > > > Emmanouil
> > >> >> > > > >>> > > > > > > > >
> > >> >> > > > >>> > > > > > > > > [1]
> > >> >> https://cloud.google.com/pubsub/docs/overview
> > >> >> > > > >>> > > > > > > > >
> > >> >> > > > >>> > > > > > > > > *Emmanouil Gkatziouras*
> > >> >> > > > >>> > > > > > > > > https://egkatzioura.com/ |
> > >> >> > > > >>> > > > > > > > >
> > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> >> > > > >>> > > > > > > > > https://github.com/gkatzioura
> > >> >> > > > >>> > > > > > > > >
> > >> >> > > > >>> > > > > > > > >
> > >> >> > > > >>> > > > > > > > > On Fri, 11 Oct 2019 at 21:45, Denis
> Magda <
> > >> >> > > > >>> dmagda@apache.org
> > >> >> > > > >>> > >
> > >> >> > > > >>> > > > > wrote:
> > >> >> > > > >>> > > > > > > > >
> > >> >> > > > >>> > > > > > > > > > Hello Emmanouil,
> > >> >> > > > >>> > > > > > > > > >
> > >> >> > > > >>> > > > > > > > > > Are you referring to any specific
> > >> >> implementation
> > >> >> > > > >>> tailored
> > >> >> > > > >>> > > for a
> > >> >> > > > >>> > > > > > > cloud?
> > >> >> > > > >>> > > > > > > > If
> > >> >> > > > >>> > > > > > > > > > I'm not mistaken you were mentioning
> > >> AWS/GCE.
> > >> >> Is
> > >> >> > > > there
> > >> >> > > > >>> any
> > >> >> > > > >>> > > > Github
> > >> >> > > > >>> > > > > > > > source
> > >> >> > > > >>> > > > > > > > > > code?
> > >> >> > > > >>> > > > > > > > > >
> > >> >> > > > >>> > > > > > > > > > Here is a list of existing integrations
> > >> with
> > >> >> > > > streaming
> > >> >> > > > >>> and
> > >> >> > > > >>> > > > > pub/sub
> > >> >> > > > >>> > > > > > > > > > frameworks:
> > >> >> > > > >>> > > > > > > > > >
> > >> >> https://apacheignite-mix.readme.io/docs/overview
> > >> >> > > > >>> > > > > > > > > >
> > >> >> > > > >>> > > > > > > > > > with the source code in Ignite master:
> > >> >> > > > >>> > > > > > > > > >
> > >> >> > > https://github.com/apache/ignite/tree/master/modules
> > >> >> > > > >>> > > > > > > > > >
> > >> >> > > > >>> > > > > > > > > > Check out those resources to see what
> are
> > >> the
> > >> >> > most
> > >> >> > > > >>> suitable
> > >> >> > > > >>> > > > > Ignite
> > >> >> > > > >>> > > > > > > APIs
> > >> >> > > > >>> > > > > > > > > for
> > >> >> > > > >>> > > > > > > > > > such integrations.
> > >> >> > > > >>> > > > > > > > > >
> > >> >> > > > >>> > > > > > > > > > @Saikat Maitra <
> saikat.maitra@gmail.com>
> > ,
> > >> >> > @Roman
> > >> >> > > <
> > >> >> > > > >>> > > > > > rshtykh@yahoo.com
> > >> >> > > > >>> > > > > > > >
> > >> >> > > > >>> > > > > > > > ,
> > >> >> > > > >>> > > > > > > > > as
> > >> >> > > > >>> > > > > > > > > > streaming technologies
> > >> >> contributors/maintainers,
> > >> >> > > > could
> > >> >> > > > >>> you
> > >> >> > > > >>> > > > please
> > >> >> > > > >>> > > > > > > join
> > >> >> > > > >>> > > > > > > > > this
> > >> >> > > > >>> > > > > > > > > > thread?
> > >> >> > > > >>> > > > > > > > > >
> > >> >> > > > >>> > > > > > > > > > -
> > >> >> > > > >>> > > > > > > > > > Denis
> > >> >> > > > >>> > > > > > > > > >
> > >> >> > > > >>> > > > > > > > > >
> > >> >> > > > >>> > > > > > > > > > On Thu, Oct 10, 2019 at 7:58 AM
> Emmanouil
> > >> >> > > > Gkatziouras <
> > >> >> > > > >>> > > > > > > > > > gkatzioura@gmail.com>
> > >> >> > > > >>> > > > > > > > > > wrote:
> > >> >> > > > >>> > > > > > > > > >
> > >> >> > > > >>> > > > > > > > > > > Hello everyone,
> > >> >> > > > >>> > > > > > > > > > >
> > >> >> > > > >>> > > > > > > > > > > I started using Ignite lately. Part
> of
> > my
> > >> >> work
> > >> >> > > > >>> involves
> > >> >> > > > >>> > > > > Pub/Sub,
> > >> >> > > > >>> > > > > > > > thus I
> > >> >> > > > >>> > > > > > > > > > > created a pub/sub streamer.
> > >> >> > > > >>> > > > > > > > > > > Pub/Sub is pretty close to Kafka.
> > >> >> > > > >>> > > > > > > > > > > Here's how the streamer works. Every
> > node
> > >> >> > creates
> > >> >> > > > an
> > >> >> > > > >>> > > instance
> > >> >> > > > >>> > > > > of
> > >> >> > > > >>> > > > > > a
> > >> >> > > > >>> > > > > > > > > > Pub/Sub
> > >> >> > > > >>> > > > > > > > > > > client.
> > >> >> > > > >>> > > > > > > > > > > The clients subscribe to the same
> topic
> > >> >> using
> > >> >> > the
> > >> >> > > > >>> same
> > >> >> > > > >>> > > > > > subscription
> > >> >> > > > >>> > > > > > > > > name.
> > >> >> > > > >>> > > > > > > > > > > A message is being sent and one of
> the
> > >> nodes
> > >> >> > will
> > >> >> > > > >>> receive
> > >> >> > > > >>> > > the
> > >> >> > > > >>> > > > > > > > message,
> > >> >> > > > >>> > > > > > > > > > > process it and then forward it to
> > Ignite
> > >> >> using
> > >> >> > > the
> > >> >> > > > >>> > > extractor
> > >> >> > > > >>> > > > > > > > provided.
> > >> >> > > > >>> > > > > > > > > > >
> > >> >> > > > >>> > > > > > > > > > > Any thoughts?
> > >> >> > > > >>> > > > > > > > > > >
> > >> >> > > > >>> > > > > > > > > > > *Emmanouil Gkatziouras*
> > >> >> > > > >>> > > > > > > > > > > https://egkatzioura.com/ |
> > >> >> > > > >>> > > > > > > > > > >
> > >> >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > >> >> > > > >>> > > > > > > > > > > https://github.com/gkatzioura
> > >> >> > > > >>> > > > > > > > > > >
> > >> >> > > > >>> > > > > > > > > >
> > >> >> > > > >>> > > > > > > > >
> > >> >> > > > >>> > > > > > > >
> > >> >> > > > >>> > > > > > >
> > >> >> > > > >>> > > > > >
> > >> >> > > > >>> > > > >
> > >> >> > > > >>> > > >
> > >> >> > > > >>> > >
> > >> >> > > > >>> >
> > >> >> > > > >>>
> > >> >> > > > >>
> > >> >> > > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >> >
> > >>
> > >
> >
>

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Denis Magda <dm...@apache.org>]

Awesome, ping us whenever you're ready!

-
Denis


On Fri, Nov 22, 2019 at 12:52 PM Emmanouil Gkatziouras <gk...@gmail.com>
wrote:

> Hi all!
>
> I am sorry for being late on that. I was trying to refactor the test in
> order not to be in need of any external tools or spinning up a server.
> I did forked the new repo and indeed my changes there, so a pull request is
> a matter of time!
>
> Kind regards
>
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>
>
> On Fri, 22 Nov 2019 at 20:45, Denis Magda <dm...@apache.org> wrote:
>
> > Hi Emmanouil,
> >
> > Do you have any questions or need any support from the community?
> >
> > -
> > Denis
> >
> >
> > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <sa...@gmail.com>
> > wrote:
> >
> >> Hi Emmanouil,
> >>
> >> Can you please take a looks at dev utils, if this is something you are
> >> looking for
> >> https://github.com/apache/ignite/tree/master/modules/dev-utils?
> >>
> >> IMO, if you can release Pub/Sub server in maven and then use it as
> >> dependency, that would be great.
> >>
> >> Regards,
> >> Saikat
> >>
> >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <sa...@gmail.com>
> >> wrote:
> >>
> >> > Hi Emmanouil,
> >> >
> >> > The master branch has been initialized. I have raised a PR to migrate
> >> > Flink module to ignite-extensions.
> >> >
> >> > PR https://github.com/apache/ignite-extensions/pull/1
> >> > Jira : https://issues.apache.org/jira/browse/IGNITE-12356
> >> >
> >> > Regards,
> >> > Saikat
> >> >
> >> >
> >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
> >> > gkatzioura@gmail.com> wrote:
> >> >
> >> >> Hi all!
> >> >>
> >> >> This is an update. The integration is tested with mocks based on the
> >> >> original Ignite repo [1].
> >> >> I am currently waiting for the first commits on the new integration
> >> >> project
> >> >> to take place in order to create my pull request accordingly to the
> >> >> initial
> >> >> commits.
> >> >> In order to simulate the Pub/Sub server in a test I used a test
> server
> >> >> implementation [2]. Unfortunately they don't have this util uploaded
> >> in a
> >> >> maven repo, thus it needs to be hosted on a maven repo (Apache
> License
> >> >> 2.0). Is there a repo for utilities that Ignite uses for the
> >> test/build?
> >> >> I have also created a ticket [3]
> >> >>
> >> >> Kind regards
> >> >>
> >> >> [1] https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> >> >> [2] https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> >> >> [3] https://issues.apache.org/jira/browse/IGNITE-12262
> >> >>
> >> >> *Emmanouil Gkatziouras*
> >> >> https://egkatzioura.com/ |
> >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> >> https://github.com/gkatzioura
> >> >>
> >> >>
> >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <saikat.maitra@gmail.com
> >
> >> >> wrote:
> >> >>
> >> >> > Hello Ilya,
> >> >> >
> >> >> > Thank you for your email. Yes, before we remove any integration we
> >> will
> >> >> > move them to another repository and release.
> >> >> >
> >> >> > We are thinking we will start with Pub/Sub Streamer implementation
> >> and
> >> >> > continue to move other integration as mentioned in the following
> doc.
> >> >> >
> >> >> >
> >> >> >
> >> >>
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >> >> >
> >> >> > Regards,
> >> >> > Saikat
> >> >> >
> >> >> >
> >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
> >> >> > ilya.kasnacheev@gmail.com>
> >> >> > wrote:
> >> >> >
> >> >> > > Hello!
> >> >> > >
> >> >> > > My take on this: we want to offload some of integrations, but
> >> please
> >> >> > commit
> >> >> > > and release them to another project first, and only then we will
> >> >> decide
> >> >> > to
> >> >> > > remove them from our repository.
> >> >> > >
> >> >> > > Please also make sure they are available with same artifact name
> >> and
> >> >> API.
> >> >> > > If this has to be changed, let's wait for AI 3.0
> >> >> > >
> >> >> > > Regards,
> >> >> > > --
> >> >> > > Ilya Kasnacheev
> >> >> > >
> >> >> > >
> >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <
> >> saikat.maitra@gmail.com>:
> >> >> > >
> >> >> > > > Hello Emmanouil,
> >> >> > > >
> >> >> > > > As discussed earlier, I discussed with Apache Bahir community
> and
> >> >> they
> >> >> > > are
> >> >> > > > interested to have Apache Ignite extensions as part of Apache
> >> Bahir
> >> >> > > > project.
> >> >> > > >
> >> >> > > > Can you please share Pub/Sub streamer implementation details
> with
> >> >> > Apache
> >> >> > > > Bahir community and request for feedback.
> >> >> > > >
> >> >> > > > https://bahir.apache.org/
> >> >> > > >
> >> >> > > > I have also requested for contributor access in Jira for Apache
> >> >> Bahir
> >> >> > > > project so that I can create issues and assign to myself.
> >> >> > > >
> >> >> > > > I can help with code reviews as well.
> >> >> > > >
> >> >> > > > Here is the email thread for reference
> >> >> > > >
> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> >> >> > > >
> >> >> > > > Regards,
> >> >> > > > Saikat
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <
> >> >> saikat.maitra@gmail.com
> >> >> > >
> >> >> > > > wrote:
> >> >> > > >
> >> >> > > > > Hello,
> >> >> > > > >
> >> >> > > > > As discussed I have created following discussion threads on
> our
> >> >> > > migration
> >> >> > > > > proposals for Apache Ignite extensions:
> >> >> > > > >
> >> >> > > > >
> >> >> > > > >
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> >> >> > > > >
> >> >> > > > >
> >> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> >> >> > > > >
> >> >> > > > > I will share update as I hear more information.
> >> >> > > > >
> >> >> > > > > Regards,
> >> >> > > > > Saikat
> >> >> > > > >
> >> >> > > > >
> >> >> > > > >
> >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra <
> >> >> > saikat.maitra@gmail.com
> >> >> > > >
> >> >> > > > > wrote:
> >> >> > > > >
> >> >> > > > >> Hello Denis,
> >> >> > > > >>
> >> >> > > > >> Sure, sounds good. I will create a separate discussion
> thread
> >> to
> >> >> get
> >> >> > > > >> community feedback on whether we
> >> >> > > > >> should join the Bahir or kick-off "Ignite Extensions"
> project.
> >> >> > > > >>
> >> >> > > > >> Regards,
> >> >> > > > >> Saikat
> >> >> > > > >>
> >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda <
> >> dmagda@apache.org>
> >> >> > > wrote:
> >> >> > > > >>
> >> >> > > > >>> Folks,
> >> >> > > > >>>
> >> >> > > > >>> The concept of Apache Bahir is precisely what we need!
> >> Saikat,
> >> >> > thanks
> >> >> > > > for
> >> >> > > > >>> doing the research. Why I believe the * idea* fits us well:
> >> >> > > > >>>
> >> >> > > > >>>    - All integrations can be stored in separate Github
> >> >> repositories
> >> >> > > and
> >> >> > > > >>>    have their dev lifecycles. We've not obliged to couple
> all
> >> >> the
> >> >> > > > >>> integrations
> >> >> > > > >>>    in a single repo. For instance, Spark can be located in
> a
> >> >> > > dedicated
> >> >> > > > >>> repo
> >> >> > > > >>>    while streaming integrations might be in a single one.
> >> It's
> >> >> up
> >> >> > to
> >> >> > > > us.
> >> >> > > > >>>    - All the repositories and integrations belong to ASF.
> >> We're
> >> >> not
> >> >> > > > >>> dumping
> >> >> > > > >>>    them on Github but rather keep supporting and developing
> >> in
> >> >> > > > >>> accordance with
> >> >> > > > >>>    ASF vision and practices.
> >> >> > > > >>>    - There is a way to reward contributors via
> committership
> >> and
> >> >> > PMC
> >> >> > > > >>>    membership. You won't get this if a project is just one
> of
> >> >> the
> >> >> > > > >>> millions of
> >> >> > > > >>>    Github projects.
> >> >> > > > >>>
> >> >> > > > >>> Saikat, as Ignite PMC member, could you please kick-off a
> >> >> separate
> >> >> > > > >>> dev-list
> >> >> > > > >>> discussion to involve more community members and referring
> to
> >> >> this
> >> >> > > > >>> thread?
> >> >> > > > >>> I think the primary question the community needs to answer
> >> >> whether
> >> >> > we
> >> >> > > > >>> should join the Bahir or kick-off "Ignite Extensions"
> >> project?
> >> >> > > > >>>
> >> >> > > > >>> -
> >> >> > > > >>> Denis
> >> >> > > > >>>
> >> >> > > > >>>
> >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev <
> >> >> > > > zaleslaw.sin@gmail.com>
> >> >> > > > >>> wrote:
> >> >> > > > >>>
> >> >> > > > >>> > Maybe we could move all our Streaming Integrations there,
> >> but
> >> >> > what
> >> >> > > is
> >> >> > > > >>> about
> >> >> > > > >>> > maintaining and committer permissions to the new
> >> repositories?
> >> >> > > > >>> > I see the list of the committers and PMC members there
> >> >> > > > >>> > https://bahir.apache.org/community-members/
> >> >> > > > >>> >
> >> >> > > > >>> > Could somebody from Ignite community be added to this
> list
> >> as
> >> >> a
> >> >> > > > >>> > PMC/committer to maintain Ignite integrations?
> >> >> > > > >>> > If yes, I'd happy to join this project and collaborate
> with
> >> >> these
> >> >> > > > guys
> >> >> > > > >>> > together
> >> >> > > > >>> > If no, and we should start from the zero level with
> >> external
> >> >> PRs
> >> >> > -
> >> >> > > > >>> hmmm,
> >> >> > > > >>> > it's better to have our own external repository with
> >> >> ApacheBahirr
> >> >> > > > >>> ideology.
> >> >> > > > >>> >
> >> >> > > > >>> > Also, I agree, that all connectors could be moved there
> (in
> >> >> > > > >>> ApacheBahirr
> >> >> > > > >>> > like repository), but not all modules from the page
> >> >> > > > >>> >
> >> >> > > > >>> >
> >> >> > > > >>>
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >> >> > > > >>> > because
> >> >> > > > >>> > they use different parts of core modules and could be
> >> >> developed
> >> >> > > with
> >> >> > > > >>> > different velocity.
> >> >> > > > >>> >
> >> >> > > > >>> > In reality, before creation of new repositories we need
> >> wide
> >> >> > > > discussion
> >> >> > > > >>> > based on the document mentioned above.
> >> >> > > > >>> >
> >> >> > > > >>> > P.S. Of course, we could experiment in the next release
> 2.8
> >> >> with
> >> >> > > one
> >> >> > > > >>> or two
> >> >> > > > >>> > integrations like twitter or ZeroMQ
> >> >> > > > >>> > Also, I'd like Denis Magda idea for the separate
> >> repositories
> >> >> in
> >> >> > > > Apache
> >> >> > > > >>> > Github to maintain them by anybody.
> >> >> > > > >>> >
> >> >> > > > >>> >
> >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra <
> >> >> > > saikat.maitra@gmail.com
> >> >> > > > >:
> >> >> > > > >>> >
> >> >> > > > >>> > > Hi Denis, Emmanouil
> >> >> > > > >>> > >
> >> >> > > > >>> > > Thank you for your email. I think creating a separate
> >> >> > integration
> >> >> > > > >>> project
> >> >> > > > >>> > > in github and also proposing it as an apache incubator
> >> >> project
> >> >> > > will
> >> >> > > > >>> be a
> >> >> > > > >>> > > good move. The other separate integration modules
> >> >> > > > >>> > >
> >> >> > > > >>> > >
> >> >> > > > >>> >
> >> >> > > > >>>
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >> >> > > > >>> > > can
> >> >> > > > >>> > > be moved to this new apache incubator project.
> >> >> > > > >>> > >
> >> >> > > > >>> > > There are similar integration available for Flink and
> >> Spark
> >> >> in
> >> >> > > > Apache
> >> >> > > > >>> > Bahir
> >> >> > > > >>> > > https://bahir.apache.org/
> >> >> > > > >>> > >
> >> >> > > > >>> > > Do you think we should reach out to Apache Bahir
> >> community
> >> >> > with a
> >> >> > > > >>> > proposal
> >> >> > > > >>> > > or we should plan to create a separate Apache Incubator
> >> >> > project?
> >> >> > > > >>> > >
> >> >> > > > >>> > > Regards,
> >> >> > > > >>> > > Saikat
> >> >> > > > >>> > >
> >> >> > > > >>> > >
> >> >> > > > >>> > >
> >> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil Gkatziouras <
> >> >> > > > >>> > > gkatzioura@gmail.com>
> >> >> > > > >>> > > wrote:
> >> >> > > > >>> > >
> >> >> > > > >>> > > > Hi all!
> >> >> > > > >>> > > >
> >> >> > > > >>> > > > Based on the discussions most probably this is going
> to
> >> >> be on
> >> >> > > > >>> another
> >> >> > > > >>> > > > GitHub repo. Therefore I will prepare a standalone
> >> project
> >> >> > with
> >> >> > > > the
> >> >> > > > >>> > > Pub/Sub
> >> >> > > > >>> > > > module and once the repository is created a pull
> >> request
> >> >> > shall
> >> >> > > be
> >> >> > > > >>> > issued.
> >> >> > > > >>> > > > If there is anything else I can do in order to
> >> facilitate
> >> >> > this
> >> >> > > > >>> please
> >> >> > > > >>> > let
> >> >> > > > >>> > > > me know.
> >> >> > > > >>> > > >
> >> >> > > > >>> > > > If a ticket is created for this issue, I shall be
> able
> >> to
> >> >> > > create
> >> >> > > > a
> >> >> > > > >>> pull
> >> >> > > > >>> > > > request and thus a build will take place on ignite's
> >> CI.
> >> >> > > > >>> > > > If everyone is aligned with adding a Pub/Sub
> >> >> implementation
> >> >> > > may I
> >> >> > > > >>> > create
> >> >> > > > >>> > > a
> >> >> > > > >>> > > > ticket on JIRA?
> >> >> > > > >>> > > >
> >> >> > > > >>> > > > My next question has to do on how Ignite configures
> the
> >> >> build
> >> >> > > > jobs
> >> >> > > > >>> in
> >> >> > > > >>> > > Team
> >> >> > > > >>> > > > City. The unit tests for my implementation need a
> local
> >> >> > Pub/Sub
> >> >> > > > >>> server
> >> >> > > > >>> > up
> >> >> > > > >>> > > > and running (they provide one for testing purposes).
> >> Will
> >> >> > this
> >> >> > > > take
> >> >> > > > >>> > > effect
> >> >> > > > >>> > > > in a form of a build script inside my implementation
> or
> >> >> is it
> >> >> > > > >>> something
> >> >> > > > >>> > > > that should be configured on Team City?
> >> >> > > > >>> > > >
> >> >> > > > >>> > > > Kind regards,
> >> >> > > > >>> > > > Emmanouil
> >> >> > > > >>> > > >
> >> >> > > > >>> > > >
> >> >> > > > >>> > > > *Emmanouil Gkatziouras*
> >> >> > > > >>> > > > https://egkatzioura.com/ |
> >> >> > > > >>> > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >> >> > > > >>> > > > https://github.com/gkatzioura
> >> >> > > > >>> > > >
> >> >> > > > >>> > > >
> >> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis Magda <
> >> >> dmagda@apache.org
> >> >> > >
> >> >> > > > >>> wrote:
> >> >> > > > >>> > > >
> >> >> > > > >>> > > > > Emmanouil, Saikat,
> >> >> > > > >>> > > > >
> >> >> > > > >>> > > > > After contemplating on this, let me propose another
> >> way
> >> >> > with
> >> >> > > > this
> >> >> > > > >>> > > > > particular integration that is aligned with our
> >> >> > > modularization
> >> >> > > > >>> > > endeavor.
> >> >> > > > >>> > > > > The modularization [1] defines the Ignite core,
> >> Ignite
> >> >> > > modules,
> >> >> > > > >>> and
> >> >> > > > >>> > > > > independent integrations. The first two are to be
> >> >> located
> >> >> > in
> >> >> > > > >>> Ignite
> >> >> > > > >>> > > > > repositories and to be tested/updated by the
> >> community
> >> >> for
> >> >> > > > every
> >> >> > > > >>> > > release.
> >> >> > > > >>> > > > > The modules are a subset of existing integrations
> >> that
> >> >> are
> >> >> > > > >>> adopted
> >> >> > > > >>> > > widely
> >> >> > > > >>> > > > > and require community attention.
> >> >> > > > >>> > > > >
> >> >> > > > >>> > > > > As for the independent integrations [1], those are
> >> >> supposed
> >> >> > > to
> >> >> > > > be
> >> >> > > > >>> > > located
> >> >> > > > >>> > > > > in a separate GitHub repo. I would advise us to
> start
> >> >> > > > >>> implementing
> >> >> > > > >>> > this
> >> >> > > > >>> > > > > concept with the Pub/Sub integration. Even though
> it
> >> >> will
> >> >> > be
> >> >> > > > >>> located
> >> >> > > > >>> > > in a
> >> >> > > > >>> > > > > separate Github repo, it will be listed among
> >> officially
> >> >> > > > >>> available
> >> >> > > > >>> > > > > integrations on the Ignite website featuring
> >> Emmanouil
> >> >> as a
> >> >> > > > >>> > > contributor.
> >> >> > > > >>> > > > >
> >> >> > > > >>> > > > > What do you think?
> >> >> > > > >>> > > > >
> >> >> > > > >>> > > > >
> >> >> > > > >>> > > > > [1]
> >> >> > > > >>> > > > >
> >> >> > > > >>> > > > >
> >> >> > > > >>> > > >
> >> >> > > > >>> > >
> >> >> > > > >>> >
> >> >> > > > >>>
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >> >> > > > >>> > > > >
> >> >> > > > >>> > > > >
> >> >> > > > >>> > > > >
> >> >> > > > >>> > > > > -
> >> >> > > > >>> > > > > Denis
> >> >> > > > >>> > > > >
> >> >> > > > >>> > > > >
> >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM Saikat Maitra <
> >> >> > > > >>> > saikat.maitra@gmail.com
> >> >> > > > >>> > > >
> >> >> > > > >>> > > > > wrote:
> >> >> > > > >>> > > > >
> >> >> > > > >>> > > > > > Hi Emmanouil,
> >> >> > > > >>> > > > > >
> >> >> > > > >>> > > > > > The changes looks good to me. I wanted to check
> if
> >> you
> >> >> > are
> >> >> > > > also
> >> >> > > > >>> > able
> >> >> > > > >>> > > to
> >> >> > > > >>> > > > > > validate that the once data is added to Ignite
> >> cluster
> >> >> > you
> >> >> > > > are
> >> >> > > > >>> also
> >> >> > > > >>> > > > able
> >> >> > > > >>> > > > > to
> >> >> > > > >>> > > > > > access it using another Ignite client or using
> rest
> >> >> > > > endpoints.
> >> >> > > > >>> > > > > >
> >> >> > > > >>> > > > > > We use Teamcity for CI process, the details are
> as
> >> >> > follows
> >> >> > > > >>> > > > > >
> >> >> > > > >>> > > > > > Trigger validation of those test suites that have
> >> been
> >> >> > > > >>> affected by
> >> >> > > > >>> > > your
> >> >> > > > >>> > > > > > changes on TeamCity <
> http://ci.ignite.apache.org/>:
> >> <
> >> >> > > > >>> > > > > http://204.14.53.153/
> >> >> > > > >>> > > > > > >
> >> >> > > > >>> > > > > >
> >> >> > > > >>> > > > > >    - Locate a test suite you have to check, press
> >> >> button
> >> >> > > > named
> >> >> > > > >>> > "..."
> >> >> > > > >>> > > > that
> >> >> > > > >>> > > > > >    is located on the left of "Run" button. "Run
> >> custom
> >> >> > > build"
> >> >> > > > >>> > dialog
> >> >> > > > >>> > > > will
> >> >> > > > >>> > > > > >    appear;
> >> >> > > > >>> > > > > >    - Go to "Changes" tab and choose
> >> >> > > > >>> > "pull/<pull-request-number>/head"
> >> >> > > > >>> > > > in
> >> >> > > > >>> > > > > >    "Build branch" dropdown list;
> >> >> > > > >>> > > > > >    - Press "Run build" button and monitor tests
> >> >> results.
> >> >> > > > >>> > > > > >
> >> >> > > > >>> > > > > >
> >> >> > > > >>> > > > > >
> >> >> > > > >>> > > > > > @Roman rshtykh@yahoo.com - can you also please
> >> take a
> >> >> > > look?
> >> >> > > > >>> > > > > >
> >> >> > > > >>> > > > > > Regards,
> >> >> > > > >>> > > > > > Saikat
> >> >> > > > >>> > > > > >
> >> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20 PM Emmanouil
> >> Gkatziouras
> >> >> <
> >> >> > > > >>> > > > > > gkatzioura@gmail.com>
> >> >> > > > >>> > > > > > wrote:
> >> >> > > > >>> > > > > >
> >> >> > > > >>> > > > > > > Hi @Saikat Maitra
> >> >> > > > >>> > > > > > >
> >> >> > > > >>> > > > > > > I have my implementation on an Ignite fork on
> my
> >> >> > personal
> >> >> > > > >>> GitHub
> >> >> > > > >>> > > > > account
> >> >> > > > >>> > > > > > > [1]
> >> >> > > > >>> > > > > > > It is based on the Kafka implementation which
> is
> >> of
> >> >> a
> >> >> > > > polling
> >> >> > > > >>> > > nature.
> >> >> > > > >>> > > > > > > GCP does not provide a service like Kafka.
> >> Instead
> >> >> it
> >> >> > > > >>> provides us
> >> >> > > > >>> > > > with
> >> >> > > > >>> > > > > > > Pub/Sub which is just like Kafka.
> >> >> > > > >>> > > > > > > Therefore using the Kafka streamer as a
> guideline
> >> >> made
> >> >> > > more
> >> >> > > > >>> sense
> >> >> > > > >>> > > to
> >> >> > > > >>> > > > > me.
> >> >> > > > >>> > > > > > >
> >> >> > > > >>> > > > > > > Apart from the implementation I have also
> added a
> >> >> unit
> >> >> > > > test.
> >> >> > > > >>> > > > > > > It definitely needs some polishing and a double
> >> >> check
> >> >> > on
> >> >> > > my
> >> >> > > > >>> side
> >> >> > > > >>> > to
> >> >> > > > >>> > > > > make
> >> >> > > > >>> > > > > > > the review smooth.
> >> >> > > > >>> > > > > > > For the tests a local Pub/Sub [2] server has to
> >> be
> >> >> > setup
> >> >> > > > >>> however
> >> >> > > > >>> > I
> >> >> > > > >>> > > am
> >> >> > > > >>> > > > > not
> >> >> > > > >>> > > > > > > sure which is the best way to do so with
> regards
> >> to
> >> >> > your
> >> >> > > > >>> CI/CD.
> >> >> > > > >>> > > > > > >
> >> >> > > > >>> > > > > > > [1]
> >> >> > > > >>> > >
> >> >> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> >> >> > > > >>> > > > > > > [2]
> >> >> > > > >>> https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
> >> >> > > > >>> > > > > > >
> >> >> > > > >>> > > > > > > *Emmanouil Gkatziouras*
> >> >> > > > >>> > > > > > > https://egkatzioura.com/ |
> >> >> > > > >>> > > > > > >
> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> >> > > > >>> > > > > > > https://github.com/gkatzioura
> >> >> > > > >>> > > > > > >
> >> >> > > > >>> > > > > > >
> >> >> > > > >>> > > > > > > On Sat, 12 Oct 2019 at 02:11, Saikat Maitra <
> >> >> > > > >>> > > saikat.maitra@gmail.com
> >> >> > > > >>> > > > >
> >> >> > > > >>> > > > > > > wrote:
> >> >> > > > >>> > > > > > >
> >> >> > > > >>> > > > > > > > Hello Emmanouil,
> >> >> > > > >>> > > > > > > >
> >> >> > > > >>> > > > > > > > Can you please share more info about the
> >> >> > > implementation?
> >> >> > > > >>> > > > > > > >
> >> >> > > > >>> > > > > > > > We have similar implementation for data
> >> streamers
> >> >> > with
> >> >> > > > >>> Kafka
> >> >> > > > >>> > and
> >> >> > > > >>> > > > > > RocketMQ
> >> >> > > > >>> > > > > > > >
> >> >> > > > >>> > > > > > > >
> >> >> > https://apacheignite-mix.readme.io/docs/kafka-streamer
> >> >> > > > >>> > > > > > > >
> >> >> > > > >>> > > > > > > >
> >> >> > > > https://apacheignite-mix.readme.io/docs/rocketmq-streamer
> >> >> > > > >>> > > > > > > >
> >> >> > > > >>> > > > > > > > Please review and share if you have any
> >> questions.
> >> >> > > > >>> > > > > > > >
> >> >> > > > >>> > > > > > > > Regards,
> >> >> > > > >>> > > > > > > > Saikat
> >> >> > > > >>> > > > > > > >
> >> >> > > > >>> > > > > > > >
> >> >> > > > >>> > > > > > > >
> >> >> > > > >>> > > > > > > > On Fri, Oct 11, 2019 at 4:56 PM Emmanouil
> >> >> > Gkatziouras <
> >> >> > > > >>> > > > > > > > gkatzioura@gmail.com>
> >> >> > > > >>> > > > > > > > wrote:
> >> >> > > > >>> > > > > > > >
> >> >> > > > >>> > > > > > > > > Hi all,
> >> >> > > > >>> > > > > > > > >
> >> >> > > > >>> > > > > > > > > By mentioning Pub/Sub I mean Google Cloud's
> >> >> Pub/Sub
> >> >> > > [1]
> >> >> > > > >>> > > > > > > > > Yes it is indeed cloud specific. It is the
> >> >> > messaging
> >> >> > > > >>> service
> >> >> > > > >>> > > > > provided
> >> >> > > > >>> > > > > > > by
> >> >> > > > >>> > > > > > > > > GCP.
> >> >> > > > >>> > > > > > > > > I have made the integration with it, but I
> >> >> haven't
> >> >> > > > >>> published
> >> >> > > > >>> > it
> >> >> > > > >>> > > > yet
> >> >> > > > >>> > > > > > (I
> >> >> > > > >>> > > > > > > am
> >> >> > > > >>> > > > > > > > > waiting for the discussion/ticket and
> proceed
> >> >> with
> >> >> > > the
> >> >> > > > >>> > > > > contribution).
> >> >> > > > >>> > > > > > > > >
> >> >> > > > >>> > > > > > > > > Kind regards,
> >> >> > > > >>> > > > > > > > > Emmanouil
> >> >> > > > >>> > > > > > > > >
> >> >> > > > >>> > > > > > > > > [1]
> >> >> https://cloud.google.com/pubsub/docs/overview
> >> >> > > > >>> > > > > > > > >
> >> >> > > > >>> > > > > > > > > *Emmanouil Gkatziouras*
> >> >> > > > >>> > > > > > > > > https://egkatzioura.com/ |
> >> >> > > > >>> > > > > > > > >
> >> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> >> > > > >>> > > > > > > > > https://github.com/gkatzioura
> >> >> > > > >>> > > > > > > > >
> >> >> > > > >>> > > > > > > > >
> >> >> > > > >>> > > > > > > > > On Fri, 11 Oct 2019 at 21:45, Denis Magda <
> >> >> > > > >>> dmagda@apache.org
> >> >> > > > >>> > >
> >> >> > > > >>> > > > > wrote:
> >> >> > > > >>> > > > > > > > >
> >> >> > > > >>> > > > > > > > > > Hello Emmanouil,
> >> >> > > > >>> > > > > > > > > >
> >> >> > > > >>> > > > > > > > > > Are you referring to any specific
> >> >> implementation
> >> >> > > > >>> tailored
> >> >> > > > >>> > > for a
> >> >> > > > >>> > > > > > > cloud?
> >> >> > > > >>> > > > > > > > If
> >> >> > > > >>> > > > > > > > > > I'm not mistaken you were mentioning
> >> AWS/GCE.
> >> >> Is
> >> >> > > > there
> >> >> > > > >>> any
> >> >> > > > >>> > > > Github
> >> >> > > > >>> > > > > > > > source
> >> >> > > > >>> > > > > > > > > > code?
> >> >> > > > >>> > > > > > > > > >
> >> >> > > > >>> > > > > > > > > > Here is a list of existing integrations
> >> with
> >> >> > > > streaming
> >> >> > > > >>> and
> >> >> > > > >>> > > > > pub/sub
> >> >> > > > >>> > > > > > > > > > frameworks:
> >> >> > > > >>> > > > > > > > > >
> >> >> https://apacheignite-mix.readme.io/docs/overview
> >> >> > > > >>> > > > > > > > > >
> >> >> > > > >>> > > > > > > > > > with the source code in Ignite master:
> >> >> > > > >>> > > > > > > > > >
> >> >> > > https://github.com/apache/ignite/tree/master/modules
> >> >> > > > >>> > > > > > > > > >
> >> >> > > > >>> > > > > > > > > > Check out those resources to see what are
> >> the
> >> >> > most
> >> >> > > > >>> suitable
> >> >> > > > >>> > > > > Ignite
> >> >> > > > >>> > > > > > > APIs
> >> >> > > > >>> > > > > > > > > for
> >> >> > > > >>> > > > > > > > > > such integrations.
> >> >> > > > >>> > > > > > > > > >
> >> >> > > > >>> > > > > > > > > > @Saikat Maitra <sa...@gmail.com>
> ,
> >> >> > @Roman
> >> >> > > <
> >> >> > > > >>> > > > > > rshtykh@yahoo.com
> >> >> > > > >>> > > > > > > >
> >> >> > > > >>> > > > > > > > ,
> >> >> > > > >>> > > > > > > > > as
> >> >> > > > >>> > > > > > > > > > streaming technologies
> >> >> contributors/maintainers,
> >> >> > > > could
> >> >> > > > >>> you
> >> >> > > > >>> > > > please
> >> >> > > > >>> > > > > > > join
> >> >> > > > >>> > > > > > > > > this
> >> >> > > > >>> > > > > > > > > > thread?
> >> >> > > > >>> > > > > > > > > >
> >> >> > > > >>> > > > > > > > > > -
> >> >> > > > >>> > > > > > > > > > Denis
> >> >> > > > >>> > > > > > > > > >
> >> >> > > > >>> > > > > > > > > >
> >> >> > > > >>> > > > > > > > > > On Thu, Oct 10, 2019 at 7:58 AM Emmanouil
> >> >> > > > Gkatziouras <
> >> >> > > > >>> > > > > > > > > > gkatzioura@gmail.com>
> >> >> > > > >>> > > > > > > > > > wrote:
> >> >> > > > >>> > > > > > > > > >
> >> >> > > > >>> > > > > > > > > > > Hello everyone,
> >> >> > > > >>> > > > > > > > > > >
> >> >> > > > >>> > > > > > > > > > > I started using Ignite lately. Part of
> my
> >> >> work
> >> >> > > > >>> involves
> >> >> > > > >>> > > > > Pub/Sub,
> >> >> > > > >>> > > > > > > > thus I
> >> >> > > > >>> > > > > > > > > > > created a pub/sub streamer.
> >> >> > > > >>> > > > > > > > > > > Pub/Sub is pretty close to Kafka.
> >> >> > > > >>> > > > > > > > > > > Here's how the streamer works. Every
> node
> >> >> > creates
> >> >> > > > an
> >> >> > > > >>> > > instance
> >> >> > > > >>> > > > > of
> >> >> > > > >>> > > > > > a
> >> >> > > > >>> > > > > > > > > > Pub/Sub
> >> >> > > > >>> > > > > > > > > > > client.
> >> >> > > > >>> > > > > > > > > > > The clients subscribe to the same topic
> >> >> using
> >> >> > the
> >> >> > > > >>> same
> >> >> > > > >>> > > > > > subscription
> >> >> > > > >>> > > > > > > > > name.
> >> >> > > > >>> > > > > > > > > > > A message is being sent and one of the
> >> nodes
> >> >> > will
> >> >> > > > >>> receive
> >> >> > > > >>> > > the
> >> >> > > > >>> > > > > > > > message,
> >> >> > > > >>> > > > > > > > > > > process it and then forward it to
> Ignite
> >> >> using
> >> >> > > the
> >> >> > > > >>> > > extractor
> >> >> > > > >>> > > > > > > > provided.
> >> >> > > > >>> > > > > > > > > > >
> >> >> > > > >>> > > > > > > > > > > Any thoughts?
> >> >> > > > >>> > > > > > > > > > >
> >> >> > > > >>> > > > > > > > > > > *Emmanouil Gkatziouras*
> >> >> > > > >>> > > > > > > > > > > https://egkatzioura.com/ |
> >> >> > > > >>> > > > > > > > > > >
> >> >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >> >> > > > >>> > > > > > > > > > > https://github.com/gkatzioura
> >> >> > > > >>> > > > > > > > > > >
> >> >> > > > >>> > > > > > > > > >
> >> >> > > > >>> > > > > > > > >
> >> >> > > > >>> > > > > > > >
> >> >> > > > >>> > > > > > >
> >> >> > > > >>> > > > > >
> >> >> > > > >>> > > > >
> >> >> > > > >>> > > >
> >> >> > > > >>> > >
> >> >> > > > >>> >
> >> >> > > > >>>
> >> >> > > > >>
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >> >
> >>
> >
>

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Emmanouil Gkatziouras <gk...@gmail.com>]

Hi all!

I am sorry for being late on that. I was trying to refactor the test in
order not to be in need of any external tools or spinning up a server.
I did forked the new repo and indeed my changes there, so a pull request is
a matter of time!

Kind regards

*Emmanouil Gkatziouras*
https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/
https://github.com/gkatzioura


On Fri, 22 Nov 2019 at 20:45, Denis Magda <dm...@apache.org> wrote:

> Hi Emmanouil,
>
> Do you have any questions or need any support from the community?
>
> -
> Denis
>
>
> On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <sa...@gmail.com>
> wrote:
>
>> Hi Emmanouil,
>>
>> Can you please take a looks at dev utils, if this is something you are
>> looking for
>> https://github.com/apache/ignite/tree/master/modules/dev-utils?
>>
>> IMO, if you can release Pub/Sub server in maven and then use it as
>> dependency, that would be great.
>>
>> Regards,
>> Saikat
>>
>> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <sa...@gmail.com>
>> wrote:
>>
>> > Hi Emmanouil,
>> >
>> > The master branch has been initialized. I have raised a PR to migrate
>> > Flink module to ignite-extensions.
>> >
>> > PR https://github.com/apache/ignite-extensions/pull/1
>> > Jira : https://issues.apache.org/jira/browse/IGNITE-12356
>> >
>> > Regards,
>> > Saikat
>> >
>> >
>> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
>> > gkatzioura@gmail.com> wrote:
>> >
>> >> Hi all!
>> >>
>> >> This is an update. The integration is tested with mocks based on the
>> >> original Ignite repo [1].
>> >> I am currently waiting for the first commits on the new integration
>> >> project
>> >> to take place in order to create my pull request accordingly to the
>> >> initial
>> >> commits.
>> >> In order to simulate the Pub/Sub server in a test I used a test server
>> >> implementation [2]. Unfortunately they don't have this util uploaded
>> in a
>> >> maven repo, thus it needs to be hosted on a maven repo (Apache License
>> >> 2.0). Is there a repo for utilities that Ignite uses for the
>> test/build?
>> >> I have also created a ticket [3]
>> >>
>> >> Kind regards
>> >>
>> >> [1] https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
>> >> [2] https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
>> >> [3] https://issues.apache.org/jira/browse/IGNITE-12262
>> >>
>> >> *Emmanouil Gkatziouras*
>> >> https://egkatzioura.com/ |
>> >> https://www.linkedin.com/in/gkatziourasemmanouil/
>> >> https://github.com/gkatzioura
>> >>
>> >>
>> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <sa...@gmail.com>
>> >> wrote:
>> >>
>> >> > Hello Ilya,
>> >> >
>> >> > Thank you for your email. Yes, before we remove any integration we
>> will
>> >> > move them to another repository and release.
>> >> >
>> >> > We are thinking we will start with Pub/Sub Streamer implementation
>> and
>> >> > continue to move other integration as mentioned in the following doc.
>> >> >
>> >> >
>> >> >
>> >>
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> >> >
>> >> > Regards,
>> >> > Saikat
>> >> >
>> >> >
>> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
>> >> > ilya.kasnacheev@gmail.com>
>> >> > wrote:
>> >> >
>> >> > > Hello!
>> >> > >
>> >> > > My take on this: we want to offload some of integrations, but
>> please
>> >> > commit
>> >> > > and release them to another project first, and only then we will
>> >> decide
>> >> > to
>> >> > > remove them from our repository.
>> >> > >
>> >> > > Please also make sure they are available with same artifact name
>> and
>> >> API.
>> >> > > If this has to be changed, let's wait for AI 3.0
>> >> > >
>> >> > > Regards,
>> >> > > --
>> >> > > Ilya Kasnacheev
>> >> > >
>> >> > >
>> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <
>> saikat.maitra@gmail.com>:
>> >> > >
>> >> > > > Hello Emmanouil,
>> >> > > >
>> >> > > > As discussed earlier, I discussed with Apache Bahir community and
>> >> they
>> >> > > are
>> >> > > > interested to have Apache Ignite extensions as part of Apache
>> Bahir
>> >> > > > project.
>> >> > > >
>> >> > > > Can you please share Pub/Sub streamer implementation details with
>> >> > Apache
>> >> > > > Bahir community and request for feedback.
>> >> > > >
>> >> > > > https://bahir.apache.org/
>> >> > > >
>> >> > > > I have also requested for contributor access in Jira for Apache
>> >> Bahir
>> >> > > > project so that I can create issues and assign to myself.
>> >> > > >
>> >> > > > I can help with code reviews as well.
>> >> > > >
>> >> > > > Here is the email thread for reference
>> >> > > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
>> >> > > >
>> >> > > > Regards,
>> >> > > > Saikat
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <
>> >> saikat.maitra@gmail.com
>> >> > >
>> >> > > > wrote:
>> >> > > >
>> >> > > > > Hello,
>> >> > > > >
>> >> > > > > As discussed I have created following discussion threads on our
>> >> > > migration
>> >> > > > > proposals for Apache Ignite extensions:
>> >> > > > >
>> >> > > > >
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
>> >> > > > >
>> >> > > > >
>> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
>> >> > > > >
>> >> > > > > I will share update as I hear more information.
>> >> > > > >
>> >> > > > > Regards,
>> >> > > > > Saikat
>> >> > > > >
>> >> > > > >
>> >> > > > >
>> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra <
>> >> > saikat.maitra@gmail.com
>> >> > > >
>> >> > > > > wrote:
>> >> > > > >
>> >> > > > >> Hello Denis,
>> >> > > > >>
>> >> > > > >> Sure, sounds good. I will create a separate discussion thread
>> to
>> >> get
>> >> > > > >> community feedback on whether we
>> >> > > > >> should join the Bahir or kick-off "Ignite Extensions" project.
>> >> > > > >>
>> >> > > > >> Regards,
>> >> > > > >> Saikat
>> >> > > > >>
>> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda <
>> dmagda@apache.org>
>> >> > > wrote:
>> >> > > > >>
>> >> > > > >>> Folks,
>> >> > > > >>>
>> >> > > > >>> The concept of Apache Bahir is precisely what we need!
>> Saikat,
>> >> > thanks
>> >> > > > for
>> >> > > > >>> doing the research. Why I believe the * idea* fits us well:
>> >> > > > >>>
>> >> > > > >>>    - All integrations can be stored in separate Github
>> >> repositories
>> >> > > and
>> >> > > > >>>    have their dev lifecycles. We've not obliged to couple all
>> >> the
>> >> > > > >>> integrations
>> >> > > > >>>    in a single repo. For instance, Spark can be located in a
>> >> > > dedicated
>> >> > > > >>> repo
>> >> > > > >>>    while streaming integrations might be in a single one.
>> It's
>> >> up
>> >> > to
>> >> > > > us.
>> >> > > > >>>    - All the repositories and integrations belong to ASF.
>> We're
>> >> not
>> >> > > > >>> dumping
>> >> > > > >>>    them on Github but rather keep supporting and developing
>> in
>> >> > > > >>> accordance with
>> >> > > > >>>    ASF vision and practices.
>> >> > > > >>>    - There is a way to reward contributors via committership
>> and
>> >> > PMC
>> >> > > > >>>    membership. You won't get this if a project is just one of
>> >> the
>> >> > > > >>> millions of
>> >> > > > >>>    Github projects.
>> >> > > > >>>
>> >> > > > >>> Saikat, as Ignite PMC member, could you please kick-off a
>> >> separate
>> >> > > > >>> dev-list
>> >> > > > >>> discussion to involve more community members and referring to
>> >> this
>> >> > > > >>> thread?
>> >> > > > >>> I think the primary question the community needs to answer
>> >> whether
>> >> > we
>> >> > > > >>> should join the Bahir or kick-off "Ignite Extensions"
>> project?
>> >> > > > >>>
>> >> > > > >>> -
>> >> > > > >>> Denis
>> >> > > > >>>
>> >> > > > >>>
>> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev <
>> >> > > > zaleslaw.sin@gmail.com>
>> >> > > > >>> wrote:
>> >> > > > >>>
>> >> > > > >>> > Maybe we could move all our Streaming Integrations there,
>> but
>> >> > what
>> >> > > is
>> >> > > > >>> about
>> >> > > > >>> > maintaining and committer permissions to the new
>> repositories?
>> >> > > > >>> > I see the list of the committers and PMC members there
>> >> > > > >>> > https://bahir.apache.org/community-members/
>> >> > > > >>> >
>> >> > > > >>> > Could somebody from Ignite community be added to this list
>> as
>> >> a
>> >> > > > >>> > PMC/committer to maintain Ignite integrations?
>> >> > > > >>> > If yes, I'd happy to join this project and collaborate with
>> >> these
>> >> > > > guys
>> >> > > > >>> > together
>> >> > > > >>> > If no, and we should start from the zero level with
>> external
>> >> PRs
>> >> > -
>> >> > > > >>> hmmm,
>> >> > > > >>> > it's better to have our own external repository with
>> >> ApacheBahirr
>> >> > > > >>> ideology.
>> >> > > > >>> >
>> >> > > > >>> > Also, I agree, that all connectors could be moved there (in
>> >> > > > >>> ApacheBahirr
>> >> > > > >>> > like repository), but not all modules from the page
>> >> > > > >>> >
>> >> > > > >>> >
>> >> > > > >>>
>> >> > > >
>> >> > >
>> >> >
>> >>
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> >> > > > >>> > because
>> >> > > > >>> > they use different parts of core modules and could be
>> >> developed
>> >> > > with
>> >> > > > >>> > different velocity.
>> >> > > > >>> >
>> >> > > > >>> > In reality, before creation of new repositories we need
>> wide
>> >> > > > discussion
>> >> > > > >>> > based on the document mentioned above.
>> >> > > > >>> >
>> >> > > > >>> > P.S. Of course, we could experiment in the next release 2.8
>> >> with
>> >> > > one
>> >> > > > >>> or two
>> >> > > > >>> > integrations like twitter or ZeroMQ
>> >> > > > >>> > Also, I'd like Denis Magda idea for the separate
>> repositories
>> >> in
>> >> > > > Apache
>> >> > > > >>> > Github to maintain them by anybody.
>> >> > > > >>> >
>> >> > > > >>> >
>> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra <
>> >> > > saikat.maitra@gmail.com
>> >> > > > >:
>> >> > > > >>> >
>> >> > > > >>> > > Hi Denis, Emmanouil
>> >> > > > >>> > >
>> >> > > > >>> > > Thank you for your email. I think creating a separate
>> >> > integration
>> >> > > > >>> project
>> >> > > > >>> > > in github and also proposing it as an apache incubator
>> >> project
>> >> > > will
>> >> > > > >>> be a
>> >> > > > >>> > > good move. The other separate integration modules
>> >> > > > >>> > >
>> >> > > > >>> > >
>> >> > > > >>> >
>> >> > > > >>>
>> >> > > >
>> >> > >
>> >> >
>> >>
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> >> > > > >>> > > can
>> >> > > > >>> > > be moved to this new apache incubator project.
>> >> > > > >>> > >
>> >> > > > >>> > > There are similar integration available for Flink and
>> Spark
>> >> in
>> >> > > > Apache
>> >> > > > >>> > Bahir
>> >> > > > >>> > > https://bahir.apache.org/
>> >> > > > >>> > >
>> >> > > > >>> > > Do you think we should reach out to Apache Bahir
>> community
>> >> > with a
>> >> > > > >>> > proposal
>> >> > > > >>> > > or we should plan to create a separate Apache Incubator
>> >> > project?
>> >> > > > >>> > >
>> >> > > > >>> > > Regards,
>> >> > > > >>> > > Saikat
>> >> > > > >>> > >
>> >> > > > >>> > >
>> >> > > > >>> > >
>> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil Gkatziouras <
>> >> > > > >>> > > gkatzioura@gmail.com>
>> >> > > > >>> > > wrote:
>> >> > > > >>> > >
>> >> > > > >>> > > > Hi all!
>> >> > > > >>> > > >
>> >> > > > >>> > > > Based on the discussions most probably this is going to
>> >> be on
>> >> > > > >>> another
>> >> > > > >>> > > > GitHub repo. Therefore I will prepare a standalone
>> project
>> >> > with
>> >> > > > the
>> >> > > > >>> > > Pub/Sub
>> >> > > > >>> > > > module and once the repository is created a pull
>> request
>> >> > shall
>> >> > > be
>> >> > > > >>> > issued.
>> >> > > > >>> > > > If there is anything else I can do in order to
>> facilitate
>> >> > this
>> >> > > > >>> please
>> >> > > > >>> > let
>> >> > > > >>> > > > me know.
>> >> > > > >>> > > >
>> >> > > > >>> > > > If a ticket is created for this issue, I shall be able
>> to
>> >> > > create
>> >> > > > a
>> >> > > > >>> pull
>> >> > > > >>> > > > request and thus a build will take place on ignite's
>> CI.
>> >> > > > >>> > > > If everyone is aligned with adding a Pub/Sub
>> >> implementation
>> >> > > may I
>> >> > > > >>> > create
>> >> > > > >>> > > a
>> >> > > > >>> > > > ticket on JIRA?
>> >> > > > >>> > > >
>> >> > > > >>> > > > My next question has to do on how Ignite configures the
>> >> build
>> >> > > > jobs
>> >> > > > >>> in
>> >> > > > >>> > > Team
>> >> > > > >>> > > > City. The unit tests for my implementation need a local
>> >> > Pub/Sub
>> >> > > > >>> server
>> >> > > > >>> > up
>> >> > > > >>> > > > and running (they provide one for testing purposes).
>> Will
>> >> > this
>> >> > > > take
>> >> > > > >>> > > effect
>> >> > > > >>> > > > in a form of a build script inside my implementation or
>> >> is it
>> >> > > > >>> something
>> >> > > > >>> > > > that should be configured on Team City?
>> >> > > > >>> > > >
>> >> > > > >>> > > > Kind regards,
>> >> > > > >>> > > > Emmanouil
>> >> > > > >>> > > >
>> >> > > > >>> > > >
>> >> > > > >>> > > > *Emmanouil Gkatziouras*
>> >> > > > >>> > > > https://egkatzioura.com/ |
>> >> > > > >>> > > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> >> > > > >>> > > > https://github.com/gkatzioura
>> >> > > > >>> > > >
>> >> > > > >>> > > >
>> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis Magda <
>> >> dmagda@apache.org
>> >> > >
>> >> > > > >>> wrote:
>> >> > > > >>> > > >
>> >> > > > >>> > > > > Emmanouil, Saikat,
>> >> > > > >>> > > > >
>> >> > > > >>> > > > > After contemplating on this, let me propose another
>> way
>> >> > with
>> >> > > > this
>> >> > > > >>> > > > > particular integration that is aligned with our
>> >> > > modularization
>> >> > > > >>> > > endeavor.
>> >> > > > >>> > > > > The modularization [1] defines the Ignite core,
>> Ignite
>> >> > > modules,
>> >> > > > >>> and
>> >> > > > >>> > > > > independent integrations. The first two are to be
>> >> located
>> >> > in
>> >> > > > >>> Ignite
>> >> > > > >>> > > > > repositories and to be tested/updated by the
>> community
>> >> for
>> >> > > > every
>> >> > > > >>> > > release.
>> >> > > > >>> > > > > The modules are a subset of existing integrations
>> that
>> >> are
>> >> > > > >>> adopted
>> >> > > > >>> > > widely
>> >> > > > >>> > > > > and require community attention.
>> >> > > > >>> > > > >
>> >> > > > >>> > > > > As for the independent integrations [1], those are
>> >> supposed
>> >> > > to
>> >> > > > be
>> >> > > > >>> > > located
>> >> > > > >>> > > > > in a separate GitHub repo. I would advise us to start
>> >> > > > >>> implementing
>> >> > > > >>> > this
>> >> > > > >>> > > > > concept with the Pub/Sub integration. Even though it
>> >> will
>> >> > be
>> >> > > > >>> located
>> >> > > > >>> > > in a
>> >> > > > >>> > > > > separate Github repo, it will be listed among
>> officially
>> >> > > > >>> available
>> >> > > > >>> > > > > integrations on the Ignite website featuring
>> Emmanouil
>> >> as a
>> >> > > > >>> > > contributor.
>> >> > > > >>> > > > >
>> >> > > > >>> > > > > What do you think?
>> >> > > > >>> > > > >
>> >> > > > >>> > > > >
>> >> > > > >>> > > > > [1]
>> >> > > > >>> > > > >
>> >> > > > >>> > > > >
>> >> > > > >>> > > >
>> >> > > > >>> > >
>> >> > > > >>> >
>> >> > > > >>>
>> >> > > >
>> >> > >
>> >> >
>> >>
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> >> > > > >>> > > > >
>> >> > > > >>> > > > >
>> >> > > > >>> > > > >
>> >> > > > >>> > > > > -
>> >> > > > >>> > > > > Denis
>> >> > > > >>> > > > >
>> >> > > > >>> > > > >
>> >> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM Saikat Maitra <
>> >> > > > >>> > saikat.maitra@gmail.com
>> >> > > > >>> > > >
>> >> > > > >>> > > > > wrote:
>> >> > > > >>> > > > >
>> >> > > > >>> > > > > > Hi Emmanouil,
>> >> > > > >>> > > > > >
>> >> > > > >>> > > > > > The changes looks good to me. I wanted to check if
>> you
>> >> > are
>> >> > > > also
>> >> > > > >>> > able
>> >> > > > >>> > > to
>> >> > > > >>> > > > > > validate that the once data is added to Ignite
>> cluster
>> >> > you
>> >> > > > are
>> >> > > > >>> also
>> >> > > > >>> > > > able
>> >> > > > >>> > > > > to
>> >> > > > >>> > > > > > access it using another Ignite client or using rest
>> >> > > > endpoints.
>> >> > > > >>> > > > > >
>> >> > > > >>> > > > > > We use Teamcity for CI process, the details are as
>> >> > follows
>> >> > > > >>> > > > > >
>> >> > > > >>> > > > > > Trigger validation of those test suites that have
>> been
>> >> > > > >>> affected by
>> >> > > > >>> > > your
>> >> > > > >>> > > > > > changes on TeamCity <http://ci.ignite.apache.org/>:
>> <
>> >> > > > >>> > > > > http://204.14.53.153/
>> >> > > > >>> > > > > > >
>> >> > > > >>> > > > > >
>> >> > > > >>> > > > > >    - Locate a test suite you have to check, press
>> >> button
>> >> > > > named
>> >> > > > >>> > "..."
>> >> > > > >>> > > > that
>> >> > > > >>> > > > > >    is located on the left of "Run" button. "Run
>> custom
>> >> > > build"
>> >> > > > >>> > dialog
>> >> > > > >>> > > > will
>> >> > > > >>> > > > > >    appear;
>> >> > > > >>> > > > > >    - Go to "Changes" tab and choose
>> >> > > > >>> > "pull/<pull-request-number>/head"
>> >> > > > >>> > > > in
>> >> > > > >>> > > > > >    "Build branch" dropdown list;
>> >> > > > >>> > > > > >    - Press "Run build" button and monitor tests
>> >> results.
>> >> > > > >>> > > > > >
>> >> > > > >>> > > > > >
>> >> > > > >>> > > > > >
>> >> > > > >>> > > > > > @Roman rshtykh@yahoo.com - can you also please
>> take a
>> >> > > look?
>> >> > > > >>> > > > > >
>> >> > > > >>> > > > > > Regards,
>> >> > > > >>> > > > > > Saikat
>> >> > > > >>> > > > > >
>> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20 PM Emmanouil
>> Gkatziouras
>> >> <
>> >> > > > >>> > > > > > gkatzioura@gmail.com>
>> >> > > > >>> > > > > > wrote:
>> >> > > > >>> > > > > >
>> >> > > > >>> > > > > > > Hi @Saikat Maitra
>> >> > > > >>> > > > > > >
>> >> > > > >>> > > > > > > I have my implementation on an Ignite fork on my
>> >> > personal
>> >> > > > >>> GitHub
>> >> > > > >>> > > > > account
>> >> > > > >>> > > > > > > [1]
>> >> > > > >>> > > > > > > It is based on the Kafka implementation which is
>> of
>> >> a
>> >> > > > polling
>> >> > > > >>> > > nature.
>> >> > > > >>> > > > > > > GCP does not provide a service like Kafka.
>> Instead
>> >> it
>> >> > > > >>> provides us
>> >> > > > >>> > > > with
>> >> > > > >>> > > > > > > Pub/Sub which is just like Kafka.
>> >> > > > >>> > > > > > > Therefore using the Kafka streamer as a guideline
>> >> made
>> >> > > more
>> >> > > > >>> sense
>> >> > > > >>> > > to
>> >> > > > >>> > > > > me.
>> >> > > > >>> > > > > > >
>> >> > > > >>> > > > > > > Apart from the implementation I have also added a
>> >> unit
>> >> > > > test.
>> >> > > > >>> > > > > > > It definitely needs some polishing and a double
>> >> check
>> >> > on
>> >> > > my
>> >> > > > >>> side
>> >> > > > >>> > to
>> >> > > > >>> > > > > make
>> >> > > > >>> > > > > > > the review smooth.
>> >> > > > >>> > > > > > > For the tests a local Pub/Sub [2] server has to
>> be
>> >> > setup
>> >> > > > >>> however
>> >> > > > >>> > I
>> >> > > > >>> > > am
>> >> > > > >>> > > > > not
>> >> > > > >>> > > > > > > sure which is the best way to do so with regards
>> to
>> >> > your
>> >> > > > >>> CI/CD.
>> >> > > > >>> > > > > > >
>> >> > > > >>> > > > > > > [1]
>> >> > > > >>> > >
>> >> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
>> >> > > > >>> > > > > > > [2]
>> >> > > > >>> https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
>> >> > > > >>> > > > > > >
>> >> > > > >>> > > > > > > *Emmanouil Gkatziouras*
>> >> > > > >>> > > > > > > https://egkatzioura.com/ |
>> >> > > > >>> > > > > > >
>> https://www.linkedin.com/in/gkatziourasemmanouil/
>> >> > > > >>> > > > > > > https://github.com/gkatzioura
>> >> > > > >>> > > > > > >
>> >> > > > >>> > > > > > >
>> >> > > > >>> > > > > > > On Sat, 12 Oct 2019 at 02:11, Saikat Maitra <
>> >> > > > >>> > > saikat.maitra@gmail.com
>> >> > > > >>> > > > >
>> >> > > > >>> > > > > > > wrote:
>> >> > > > >>> > > > > > >
>> >> > > > >>> > > > > > > > Hello Emmanouil,
>> >> > > > >>> > > > > > > >
>> >> > > > >>> > > > > > > > Can you please share more info about the
>> >> > > implementation?
>> >> > > > >>> > > > > > > >
>> >> > > > >>> > > > > > > > We have similar implementation for data
>> streamers
>> >> > with
>> >> > > > >>> Kafka
>> >> > > > >>> > and
>> >> > > > >>> > > > > > RocketMQ
>> >> > > > >>> > > > > > > >
>> >> > > > >>> > > > > > > >
>> >> > https://apacheignite-mix.readme.io/docs/kafka-streamer
>> >> > > > >>> > > > > > > >
>> >> > > > >>> > > > > > > >
>> >> > > > https://apacheignite-mix.readme.io/docs/rocketmq-streamer
>> >> > > > >>> > > > > > > >
>> >> > > > >>> > > > > > > > Please review and share if you have any
>> questions.
>> >> > > > >>> > > > > > > >
>> >> > > > >>> > > > > > > > Regards,
>> >> > > > >>> > > > > > > > Saikat
>> >> > > > >>> > > > > > > >
>> >> > > > >>> > > > > > > >
>> >> > > > >>> > > > > > > >
>> >> > > > >>> > > > > > > > On Fri, Oct 11, 2019 at 4:56 PM Emmanouil
>> >> > Gkatziouras <
>> >> > > > >>> > > > > > > > gkatzioura@gmail.com>
>> >> > > > >>> > > > > > > > wrote:
>> >> > > > >>> > > > > > > >
>> >> > > > >>> > > > > > > > > Hi all,
>> >> > > > >>> > > > > > > > >
>> >> > > > >>> > > > > > > > > By mentioning Pub/Sub I mean Google Cloud's
>> >> Pub/Sub
>> >> > > [1]
>> >> > > > >>> > > > > > > > > Yes it is indeed cloud specific. It is the
>> >> > messaging
>> >> > > > >>> service
>> >> > > > >>> > > > > provided
>> >> > > > >>> > > > > > > by
>> >> > > > >>> > > > > > > > > GCP.
>> >> > > > >>> > > > > > > > > I have made the integration with it, but I
>> >> haven't
>> >> > > > >>> published
>> >> > > > >>> > it
>> >> > > > >>> > > > yet
>> >> > > > >>> > > > > > (I
>> >> > > > >>> > > > > > > am
>> >> > > > >>> > > > > > > > > waiting for the discussion/ticket and proceed
>> >> with
>> >> > > the
>> >> > > > >>> > > > > contribution).
>> >> > > > >>> > > > > > > > >
>> >> > > > >>> > > > > > > > > Kind regards,
>> >> > > > >>> > > > > > > > > Emmanouil
>> >> > > > >>> > > > > > > > >
>> >> > > > >>> > > > > > > > > [1]
>> >> https://cloud.google.com/pubsub/docs/overview
>> >> > > > >>> > > > > > > > >
>> >> > > > >>> > > > > > > > > *Emmanouil Gkatziouras*
>> >> > > > >>> > > > > > > > > https://egkatzioura.com/ |
>> >> > > > >>> > > > > > > > >
>> >> https://www.linkedin.com/in/gkatziourasemmanouil/
>> >> > > > >>> > > > > > > > > https://github.com/gkatzioura
>> >> > > > >>> > > > > > > > >
>> >> > > > >>> > > > > > > > >
>> >> > > > >>> > > > > > > > > On Fri, 11 Oct 2019 at 21:45, Denis Magda <
>> >> > > > >>> dmagda@apache.org
>> >> > > > >>> > >
>> >> > > > >>> > > > > wrote:
>> >> > > > >>> > > > > > > > >
>> >> > > > >>> > > > > > > > > > Hello Emmanouil,
>> >> > > > >>> > > > > > > > > >
>> >> > > > >>> > > > > > > > > > Are you referring to any specific
>> >> implementation
>> >> > > > >>> tailored
>> >> > > > >>> > > for a
>> >> > > > >>> > > > > > > cloud?
>> >> > > > >>> > > > > > > > If
>> >> > > > >>> > > > > > > > > > I'm not mistaken you were mentioning
>> AWS/GCE.
>> >> Is
>> >> > > > there
>> >> > > > >>> any
>> >> > > > >>> > > > Github
>> >> > > > >>> > > > > > > > source
>> >> > > > >>> > > > > > > > > > code?
>> >> > > > >>> > > > > > > > > >
>> >> > > > >>> > > > > > > > > > Here is a list of existing integrations
>> with
>> >> > > > streaming
>> >> > > > >>> and
>> >> > > > >>> > > > > pub/sub
>> >> > > > >>> > > > > > > > > > frameworks:
>> >> > > > >>> > > > > > > > > >
>> >> https://apacheignite-mix.readme.io/docs/overview
>> >> > > > >>> > > > > > > > > >
>> >> > > > >>> > > > > > > > > > with the source code in Ignite master:
>> >> > > > >>> > > > > > > > > >
>> >> > > https://github.com/apache/ignite/tree/master/modules
>> >> > > > >>> > > > > > > > > >
>> >> > > > >>> > > > > > > > > > Check out those resources to see what are
>> the
>> >> > most
>> >> > > > >>> suitable
>> >> > > > >>> > > > > Ignite
>> >> > > > >>> > > > > > > APIs
>> >> > > > >>> > > > > > > > > for
>> >> > > > >>> > > > > > > > > > such integrations.
>> >> > > > >>> > > > > > > > > >
>> >> > > > >>> > > > > > > > > > @Saikat Maitra <sa...@gmail.com> ,
>> >> > @Roman
>> >> > > <
>> >> > > > >>> > > > > > rshtykh@yahoo.com
>> >> > > > >>> > > > > > > >
>> >> > > > >>> > > > > > > > ,
>> >> > > > >>> > > > > > > > > as
>> >> > > > >>> > > > > > > > > > streaming technologies
>> >> contributors/maintainers,
>> >> > > > could
>> >> > > > >>> you
>> >> > > > >>> > > > please
>> >> > > > >>> > > > > > > join
>> >> > > > >>> > > > > > > > > this
>> >> > > > >>> > > > > > > > > > thread?
>> >> > > > >>> > > > > > > > > >
>> >> > > > >>> > > > > > > > > > -
>> >> > > > >>> > > > > > > > > > Denis
>> >> > > > >>> > > > > > > > > >
>> >> > > > >>> > > > > > > > > >
>> >> > > > >>> > > > > > > > > > On Thu, Oct 10, 2019 at 7:58 AM Emmanouil
>> >> > > > Gkatziouras <
>> >> > > > >>> > > > > > > > > > gkatzioura@gmail.com>
>> >> > > > >>> > > > > > > > > > wrote:
>> >> > > > >>> > > > > > > > > >
>> >> > > > >>> > > > > > > > > > > Hello everyone,
>> >> > > > >>> > > > > > > > > > >
>> >> > > > >>> > > > > > > > > > > I started using Ignite lately. Part of my
>> >> work
>> >> > > > >>> involves
>> >> > > > >>> > > > > Pub/Sub,
>> >> > > > >>> > > > > > > > thus I
>> >> > > > >>> > > > > > > > > > > created a pub/sub streamer.
>> >> > > > >>> > > > > > > > > > > Pub/Sub is pretty close to Kafka.
>> >> > > > >>> > > > > > > > > > > Here's how the streamer works. Every node
>> >> > creates
>> >> > > > an
>> >> > > > >>> > > instance
>> >> > > > >>> > > > > of
>> >> > > > >>> > > > > > a
>> >> > > > >>> > > > > > > > > > Pub/Sub
>> >> > > > >>> > > > > > > > > > > client.
>> >> > > > >>> > > > > > > > > > > The clients subscribe to the same topic
>> >> using
>> >> > the
>> >> > > > >>> same
>> >> > > > >>> > > > > > subscription
>> >> > > > >>> > > > > > > > > name.
>> >> > > > >>> > > > > > > > > > > A message is being sent and one of the
>> nodes
>> >> > will
>> >> > > > >>> receive
>> >> > > > >>> > > the
>> >> > > > >>> > > > > > > > message,
>> >> > > > >>> > > > > > > > > > > process it and then forward it to Ignite
>> >> using
>> >> > > the
>> >> > > > >>> > > extractor
>> >> > > > >>> > > > > > > > provided.
>> >> > > > >>> > > > > > > > > > >
>> >> > > > >>> > > > > > > > > > > Any thoughts?
>> >> > > > >>> > > > > > > > > > >
>> >> > > > >>> > > > > > > > > > > *Emmanouil Gkatziouras*
>> >> > > > >>> > > > > > > > > > > https://egkatzioura.com/ |
>> >> > > > >>> > > > > > > > > > >
>> >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> >> > > > >>> > > > > > > > > > > https://github.com/gkatzioura
>> >> > > > >>> > > > > > > > > > >
>> >> > > > >>> > > > > > > > > >
>> >> > > > >>> > > > > > > > >
>> >> > > > >>> > > > > > > >
>> >> > > > >>> > > > > > >
>> >> > > > >>> > > > > >
>> >> > > > >>> > > > >
>> >> > > > >>> > > >
>> >> > > > >>> > >
>> >> > > > >>> >
>> >> > > > >>>
>> >> > > > >>
>> >> > > >
>> >> > >
>> >> >
>> >>
>> >
>>
>

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Denis Magda <dm...@apache.org>]

Hi Emmanouil,

Do you have any questions or need any support from the community?

-
Denis


On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <sa...@gmail.com>
wrote:

> Hi Emmanouil,
>
> Can you please take a looks at dev utils, if this is something you are
> looking for https://github.com/apache/ignite/tree/master/modules/dev-utils
> ?
>
> IMO, if you can release Pub/Sub server in maven and then use it as
> dependency, that would be great.
>
> Regards,
> Saikat
>
> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <sa...@gmail.com>
> wrote:
>
> > Hi Emmanouil,
> >
> > The master branch has been initialized. I have raised a PR to migrate
> > Flink module to ignite-extensions.
> >
> > PR https://github.com/apache/ignite-extensions/pull/1
> > Jira : https://issues.apache.org/jira/browse/IGNITE-12356
> >
> > Regards,
> > Saikat
> >
> >
> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
> > gkatzioura@gmail.com> wrote:
> >
> >> Hi all!
> >>
> >> This is an update. The integration is tested with mocks based on the
> >> original Ignite repo [1].
> >> I am currently waiting for the first commits on the new integration
> >> project
> >> to take place in order to create my pull request accordingly to the
> >> initial
> >> commits.
> >> In order to simulate the Pub/Sub server in a test I used a test server
> >> implementation [2]. Unfortunately they don't have this util uploaded in
> a
> >> maven repo, thus it needs to be hosted on a maven repo (Apache License
> >> 2.0). Is there a repo for utilities that Ignite uses for the test/build?
> >> I have also created a ticket [3]
> >>
> >> Kind regards
> >>
> >> [1] https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> >> [2] https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> >> [3] https://issues.apache.org/jira/browse/IGNITE-12262
> >>
> >> *Emmanouil Gkatziouras*
> >> https://egkatzioura.com/ |
> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> https://github.com/gkatzioura
> >>
> >>
> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <sa...@gmail.com>
> >> wrote:
> >>
> >> > Hello Ilya,
> >> >
> >> > Thank you for your email. Yes, before we remove any integration we
> will
> >> > move them to another repository and release.
> >> >
> >> > We are thinking we will start with Pub/Sub Streamer implementation and
> >> > continue to move other integration as mentioned in the following doc.
> >> >
> >> >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >> >
> >> > Regards,
> >> > Saikat
> >> >
> >> >
> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
> >> > ilya.kasnacheev@gmail.com>
> >> > wrote:
> >> >
> >> > > Hello!
> >> > >
> >> > > My take on this: we want to offload some of integrations, but please
> >> > commit
> >> > > and release them to another project first, and only then we will
> >> decide
> >> > to
> >> > > remove them from our repository.
> >> > >
> >> > > Please also make sure they are available with same artifact name and
> >> API.
> >> > > If this has to be changed, let's wait for AI 3.0
> >> > >
> >> > > Regards,
> >> > > --
> >> > > Ilya Kasnacheev
> >> > >
> >> > >
> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <saikat.maitra@gmail.com
> >:
> >> > >
> >> > > > Hello Emmanouil,
> >> > > >
> >> > > > As discussed earlier, I discussed with Apache Bahir community and
> >> they
> >> > > are
> >> > > > interested to have Apache Ignite extensions as part of Apache
> Bahir
> >> > > > project.
> >> > > >
> >> > > > Can you please share Pub/Sub streamer implementation details with
> >> > Apache
> >> > > > Bahir community and request for feedback.
> >> > > >
> >> > > > https://bahir.apache.org/
> >> > > >
> >> > > > I have also requested for contributor access in Jira for Apache
> >> Bahir
> >> > > > project so that I can create issues and assign to myself.
> >> > > >
> >> > > > I can help with code reviews as well.
> >> > > >
> >> > > > Here is the email thread for reference
> >> > > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> >> > > >
> >> > > > Regards,
> >> > > > Saikat
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <
> >> saikat.maitra@gmail.com
> >> > >
> >> > > > wrote:
> >> > > >
> >> > > > > Hello,
> >> > > > >
> >> > > > > As discussed I have created following discussion threads on our
> >> > > migration
> >> > > > > proposals for Apache Ignite extensions:
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> >> > > > >
> >> > > > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> >> > > > >
> >> > > > > I will share update as I hear more information.
> >> > > > >
> >> > > > > Regards,
> >> > > > > Saikat
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra <
> >> > saikat.maitra@gmail.com
> >> > > >
> >> > > > > wrote:
> >> > > > >
> >> > > > >> Hello Denis,
> >> > > > >>
> >> > > > >> Sure, sounds good. I will create a separate discussion thread
> to
> >> get
> >> > > > >> community feedback on whether we
> >> > > > >> should join the Bahir or kick-off "Ignite Extensions" project.
> >> > > > >>
> >> > > > >> Regards,
> >> > > > >> Saikat
> >> > > > >>
> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda <dmagda@apache.org
> >
> >> > > wrote:
> >> > > > >>
> >> > > > >>> Folks,
> >> > > > >>>
> >> > > > >>> The concept of Apache Bahir is precisely what we need! Saikat,
> >> > thanks
> >> > > > for
> >> > > > >>> doing the research. Why I believe the * idea* fits us well:
> >> > > > >>>
> >> > > > >>>    - All integrations can be stored in separate Github
> >> repositories
> >> > > and
> >> > > > >>>    have their dev lifecycles. We've not obliged to couple all
> >> the
> >> > > > >>> integrations
> >> > > > >>>    in a single repo. For instance, Spark can be located in a
> >> > > dedicated
> >> > > > >>> repo
> >> > > > >>>    while streaming integrations might be in a single one. It's
> >> up
> >> > to
> >> > > > us.
> >> > > > >>>    - All the repositories and integrations belong to ASF.
> We're
> >> not
> >> > > > >>> dumping
> >> > > > >>>    them on Github but rather keep supporting and developing in
> >> > > > >>> accordance with
> >> > > > >>>    ASF vision and practices.
> >> > > > >>>    - There is a way to reward contributors via committership
> and
> >> > PMC
> >> > > > >>>    membership. You won't get this if a project is just one of
> >> the
> >> > > > >>> millions of
> >> > > > >>>    Github projects.
> >> > > > >>>
> >> > > > >>> Saikat, as Ignite PMC member, could you please kick-off a
> >> separate
> >> > > > >>> dev-list
> >> > > > >>> discussion to involve more community members and referring to
> >> this
> >> > > > >>> thread?
> >> > > > >>> I think the primary question the community needs to answer
> >> whether
> >> > we
> >> > > > >>> should join the Bahir or kick-off "Ignite Extensions" project?
> >> > > > >>>
> >> > > > >>> -
> >> > > > >>> Denis
> >> > > > >>>
> >> > > > >>>
> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev <
> >> > > > zaleslaw.sin@gmail.com>
> >> > > > >>> wrote:
> >> > > > >>>
> >> > > > >>> > Maybe we could move all our Streaming Integrations there,
> but
> >> > what
> >> > > is
> >> > > > >>> about
> >> > > > >>> > maintaining and committer permissions to the new
> repositories?
> >> > > > >>> > I see the list of the committers and PMC members there
> >> > > > >>> > https://bahir.apache.org/community-members/
> >> > > > >>> >
> >> > > > >>> > Could somebody from Ignite community be added to this list
> as
> >> a
> >> > > > >>> > PMC/committer to maintain Ignite integrations?
> >> > > > >>> > If yes, I'd happy to join this project and collaborate with
> >> these
> >> > > > guys
> >> > > > >>> > together
> >> > > > >>> > If no, and we should start from the zero level with external
> >> PRs
> >> > -
> >> > > > >>> hmmm,
> >> > > > >>> > it's better to have our own external repository with
> >> ApacheBahirr
> >> > > > >>> ideology.
> >> > > > >>> >
> >> > > > >>> > Also, I agree, that all connectors could be moved there (in
> >> > > > >>> ApacheBahirr
> >> > > > >>> > like repository), but not all modules from the page
> >> > > > >>> >
> >> > > > >>> >
> >> > > > >>>
> >> > > >
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >> > > > >>> > because
> >> > > > >>> > they use different parts of core modules and could be
> >> developed
> >> > > with
> >> > > > >>> > different velocity.
> >> > > > >>> >
> >> > > > >>> > In reality, before creation of new repositories we need wide
> >> > > > discussion
> >> > > > >>> > based on the document mentioned above.
> >> > > > >>> >
> >> > > > >>> > P.S. Of course, we could experiment in the next release 2.8
> >> with
> >> > > one
> >> > > > >>> or two
> >> > > > >>> > integrations like twitter or ZeroMQ
> >> > > > >>> > Also, I'd like Denis Magda idea for the separate
> repositories
> >> in
> >> > > > Apache
> >> > > > >>> > Github to maintain them by anybody.
> >> > > > >>> >
> >> > > > >>> >
> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra <
> >> > > saikat.maitra@gmail.com
> >> > > > >:
> >> > > > >>> >
> >> > > > >>> > > Hi Denis, Emmanouil
> >> > > > >>> > >
> >> > > > >>> > > Thank you for your email. I think creating a separate
> >> > integration
> >> > > > >>> project
> >> > > > >>> > > in github and also proposing it as an apache incubator
> >> project
> >> > > will
> >> > > > >>> be a
> >> > > > >>> > > good move. The other separate integration modules
> >> > > > >>> > >
> >> > > > >>> > >
> >> > > > >>> >
> >> > > > >>>
> >> > > >
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >> > > > >>> > > can
> >> > > > >>> > > be moved to this new apache incubator project.
> >> > > > >>> > >
> >> > > > >>> > > There are similar integration available for Flink and
> Spark
> >> in
> >> > > > Apache
> >> > > > >>> > Bahir
> >> > > > >>> > > https://bahir.apache.org/
> >> > > > >>> > >
> >> > > > >>> > > Do you think we should reach out to Apache Bahir community
> >> > with a
> >> > > > >>> > proposal
> >> > > > >>> > > or we should plan to create a separate Apache Incubator
> >> > project?
> >> > > > >>> > >
> >> > > > >>> > > Regards,
> >> > > > >>> > > Saikat
> >> > > > >>> > >
> >> > > > >>> > >
> >> > > > >>> > >
> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil Gkatziouras <
> >> > > > >>> > > gkatzioura@gmail.com>
> >> > > > >>> > > wrote:
> >> > > > >>> > >
> >> > > > >>> > > > Hi all!
> >> > > > >>> > > >
> >> > > > >>> > > > Based on the discussions most probably this is going to
> >> be on
> >> > > > >>> another
> >> > > > >>> > > > GitHub repo. Therefore I will prepare a standalone
> project
> >> > with
> >> > > > the
> >> > > > >>> > > Pub/Sub
> >> > > > >>> > > > module and once the repository is created a pull request
> >> > shall
> >> > > be
> >> > > > >>> > issued.
> >> > > > >>> > > > If there is anything else I can do in order to
> facilitate
> >> > this
> >> > > > >>> please
> >> > > > >>> > let
> >> > > > >>> > > > me know.
> >> > > > >>> > > >
> >> > > > >>> > > > If a ticket is created for this issue, I shall be able
> to
> >> > > create
> >> > > > a
> >> > > > >>> pull
> >> > > > >>> > > > request and thus a build will take place on ignite's CI.
> >> > > > >>> > > > If everyone is aligned with adding a Pub/Sub
> >> implementation
> >> > > may I
> >> > > > >>> > create
> >> > > > >>> > > a
> >> > > > >>> > > > ticket on JIRA?
> >> > > > >>> > > >
> >> > > > >>> > > > My next question has to do on how Ignite configures the
> >> build
> >> > > > jobs
> >> > > > >>> in
> >> > > > >>> > > Team
> >> > > > >>> > > > City. The unit tests for my implementation need a local
> >> > Pub/Sub
> >> > > > >>> server
> >> > > > >>> > up
> >> > > > >>> > > > and running (they provide one for testing purposes).
> Will
> >> > this
> >> > > > take
> >> > > > >>> > > effect
> >> > > > >>> > > > in a form of a build script inside my implementation or
> >> is it
> >> > > > >>> something
> >> > > > >>> > > > that should be configured on Team City?
> >> > > > >>> > > >
> >> > > > >>> > > > Kind regards,
> >> > > > >>> > > > Emmanouil
> >> > > > >>> > > >
> >> > > > >>> > > >
> >> > > > >>> > > > *Emmanouil Gkatziouras*
> >> > > > >>> > > > https://egkatzioura.com/ |
> >> > > > >>> > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > > >>> > > > https://github.com/gkatzioura
> >> > > > >>> > > >
> >> > > > >>> > > >
> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis Magda <
> >> dmagda@apache.org
> >> > >
> >> > > > >>> wrote:
> >> > > > >>> > > >
> >> > > > >>> > > > > Emmanouil, Saikat,
> >> > > > >>> > > > >
> >> > > > >>> > > > > After contemplating on this, let me propose another
> way
> >> > with
> >> > > > this
> >> > > > >>> > > > > particular integration that is aligned with our
> >> > > modularization
> >> > > > >>> > > endeavor.
> >> > > > >>> > > > > The modularization [1] defines the Ignite core, Ignite
> >> > > modules,
> >> > > > >>> and
> >> > > > >>> > > > > independent integrations. The first two are to be
> >> located
> >> > in
> >> > > > >>> Ignite
> >> > > > >>> > > > > repositories and to be tested/updated by the community
> >> for
> >> > > > every
> >> > > > >>> > > release.
> >> > > > >>> > > > > The modules are a subset of existing integrations that
> >> are
> >> > > > >>> adopted
> >> > > > >>> > > widely
> >> > > > >>> > > > > and require community attention.
> >> > > > >>> > > > >
> >> > > > >>> > > > > As for the independent integrations [1], those are
> >> supposed
> >> > > to
> >> > > > be
> >> > > > >>> > > located
> >> > > > >>> > > > > in a separate GitHub repo. I would advise us to start
> >> > > > >>> implementing
> >> > > > >>> > this
> >> > > > >>> > > > > concept with the Pub/Sub integration. Even though it
> >> will
> >> > be
> >> > > > >>> located
> >> > > > >>> > > in a
> >> > > > >>> > > > > separate Github repo, it will be listed among
> officially
> >> > > > >>> available
> >> > > > >>> > > > > integrations on the Ignite website featuring Emmanouil
> >> as a
> >> > > > >>> > > contributor.
> >> > > > >>> > > > >
> >> > > > >>> > > > > What do you think?
> >> > > > >>> > > > >
> >> > > > >>> > > > >
> >> > > > >>> > > > > [1]
> >> > > > >>> > > > >
> >> > > > >>> > > > >
> >> > > > >>> > > >
> >> > > > >>> > >
> >> > > > >>> >
> >> > > > >>>
> >> > > >
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >> > > > >>> > > > >
> >> > > > >>> > > > >
> >> > > > >>> > > > >
> >> > > > >>> > > > > -
> >> > > > >>> > > > > Denis
> >> > > > >>> > > > >
> >> > > > >>> > > > >
> >> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM Saikat Maitra <
> >> > > > >>> > saikat.maitra@gmail.com
> >> > > > >>> > > >
> >> > > > >>> > > > > wrote:
> >> > > > >>> > > > >
> >> > > > >>> > > > > > Hi Emmanouil,
> >> > > > >>> > > > > >
> >> > > > >>> > > > > > The changes looks good to me. I wanted to check if
> you
> >> > are
> >> > > > also
> >> > > > >>> > able
> >> > > > >>> > > to
> >> > > > >>> > > > > > validate that the once data is added to Ignite
> cluster
> >> > you
> >> > > > are
> >> > > > >>> also
> >> > > > >>> > > > able
> >> > > > >>> > > > > to
> >> > > > >>> > > > > > access it using another Ignite client or using rest
> >> > > > endpoints.
> >> > > > >>> > > > > >
> >> > > > >>> > > > > > We use Teamcity for CI process, the details are as
> >> > follows
> >> > > > >>> > > > > >
> >> > > > >>> > > > > > Trigger validation of those test suites that have
> been
> >> > > > >>> affected by
> >> > > > >>> > > your
> >> > > > >>> > > > > > changes on TeamCity <http://ci.ignite.apache.org/>:
> <
> >> > > > >>> > > > > http://204.14.53.153/
> >> > > > >>> > > > > > >
> >> > > > >>> > > > > >
> >> > > > >>> > > > > >    - Locate a test suite you have to check, press
> >> button
> >> > > > named
> >> > > > >>> > "..."
> >> > > > >>> > > > that
> >> > > > >>> > > > > >    is located on the left of "Run" button. "Run
> custom
> >> > > build"
> >> > > > >>> > dialog
> >> > > > >>> > > > will
> >> > > > >>> > > > > >    appear;
> >> > > > >>> > > > > >    - Go to "Changes" tab and choose
> >> > > > >>> > "pull/<pull-request-number>/head"
> >> > > > >>> > > > in
> >> > > > >>> > > > > >    "Build branch" dropdown list;
> >> > > > >>> > > > > >    - Press "Run build" button and monitor tests
> >> results.
> >> > > > >>> > > > > >
> >> > > > >>> > > > > >
> >> > > > >>> > > > > >
> >> > > > >>> > > > > > @Roman rshtykh@yahoo.com - can you also please
> take a
> >> > > look?
> >> > > > >>> > > > > >
> >> > > > >>> > > > > > Regards,
> >> > > > >>> > > > > > Saikat
> >> > > > >>> > > > > >
> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20 PM Emmanouil
> Gkatziouras
> >> <
> >> > > > >>> > > > > > gkatzioura@gmail.com>
> >> > > > >>> > > > > > wrote:
> >> > > > >>> > > > > >
> >> > > > >>> > > > > > > Hi @Saikat Maitra
> >> > > > >>> > > > > > >
> >> > > > >>> > > > > > > I have my implementation on an Ignite fork on my
> >> > personal
> >> > > > >>> GitHub
> >> > > > >>> > > > > account
> >> > > > >>> > > > > > > [1]
> >> > > > >>> > > > > > > It is based on the Kafka implementation which is
> of
> >> a
> >> > > > polling
> >> > > > >>> > > nature.
> >> > > > >>> > > > > > > GCP does not provide a service like Kafka. Instead
> >> it
> >> > > > >>> provides us
> >> > > > >>> > > > with
> >> > > > >>> > > > > > > Pub/Sub which is just like Kafka.
> >> > > > >>> > > > > > > Therefore using the Kafka streamer as a guideline
> >> made
> >> > > more
> >> > > > >>> sense
> >> > > > >>> > > to
> >> > > > >>> > > > > me.
> >> > > > >>> > > > > > >
> >> > > > >>> > > > > > > Apart from the implementation I have also added a
> >> unit
> >> > > > test.
> >> > > > >>> > > > > > > It definitely needs some polishing and a double
> >> check
> >> > on
> >> > > my
> >> > > > >>> side
> >> > > > >>> > to
> >> > > > >>> > > > > make
> >> > > > >>> > > > > > > the review smooth.
> >> > > > >>> > > > > > > For the tests a local Pub/Sub [2] server has to be
> >> > setup
> >> > > > >>> however
> >> > > > >>> > I
> >> > > > >>> > > am
> >> > > > >>> > > > > not
> >> > > > >>> > > > > > > sure which is the best way to do so with regards
> to
> >> > your
> >> > > > >>> CI/CD.
> >> > > > >>> > > > > > >
> >> > > > >>> > > > > > > [1]
> >> > > > >>> > >
> >> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> >> > > > >>> > > > > > > [2]
> >> > > > >>> https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
> >> > > > >>> > > > > > >
> >> > > > >>> > > > > > > *Emmanouil Gkatziouras*
> >> > > > >>> > > > > > > https://egkatzioura.com/ |
> >> > > > >>> > > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > > >>> > > > > > > https://github.com/gkatzioura
> >> > > > >>> > > > > > >
> >> > > > >>> > > > > > >
> >> > > > >>> > > > > > > On Sat, 12 Oct 2019 at 02:11, Saikat Maitra <
> >> > > > >>> > > saikat.maitra@gmail.com
> >> > > > >>> > > > >
> >> > > > >>> > > > > > > wrote:
> >> > > > >>> > > > > > >
> >> > > > >>> > > > > > > > Hello Emmanouil,
> >> > > > >>> > > > > > > >
> >> > > > >>> > > > > > > > Can you please share more info about the
> >> > > implementation?
> >> > > > >>> > > > > > > >
> >> > > > >>> > > > > > > > We have similar implementation for data
> streamers
> >> > with
> >> > > > >>> Kafka
> >> > > > >>> > and
> >> > > > >>> > > > > > RocketMQ
> >> > > > >>> > > > > > > >
> >> > > > >>> > > > > > > >
> >> > https://apacheignite-mix.readme.io/docs/kafka-streamer
> >> > > > >>> > > > > > > >
> >> > > > >>> > > > > > > >
> >> > > > https://apacheignite-mix.readme.io/docs/rocketmq-streamer
> >> > > > >>> > > > > > > >
> >> > > > >>> > > > > > > > Please review and share if you have any
> questions.
> >> > > > >>> > > > > > > >
> >> > > > >>> > > > > > > > Regards,
> >> > > > >>> > > > > > > > Saikat
> >> > > > >>> > > > > > > >
> >> > > > >>> > > > > > > >
> >> > > > >>> > > > > > > >
> >> > > > >>> > > > > > > > On Fri, Oct 11, 2019 at 4:56 PM Emmanouil
> >> > Gkatziouras <
> >> > > > >>> > > > > > > > gkatzioura@gmail.com>
> >> > > > >>> > > > > > > > wrote:
> >> > > > >>> > > > > > > >
> >> > > > >>> > > > > > > > > Hi all,
> >> > > > >>> > > > > > > > >
> >> > > > >>> > > > > > > > > By mentioning Pub/Sub I mean Google Cloud's
> >> Pub/Sub
> >> > > [1]
> >> > > > >>> > > > > > > > > Yes it is indeed cloud specific. It is the
> >> > messaging
> >> > > > >>> service
> >> > > > >>> > > > > provided
> >> > > > >>> > > > > > > by
> >> > > > >>> > > > > > > > > GCP.
> >> > > > >>> > > > > > > > > I have made the integration with it, but I
> >> haven't
> >> > > > >>> published
> >> > > > >>> > it
> >> > > > >>> > > > yet
> >> > > > >>> > > > > > (I
> >> > > > >>> > > > > > > am
> >> > > > >>> > > > > > > > > waiting for the discussion/ticket and proceed
> >> with
> >> > > the
> >> > > > >>> > > > > contribution).
> >> > > > >>> > > > > > > > >
> >> > > > >>> > > > > > > > > Kind regards,
> >> > > > >>> > > > > > > > > Emmanouil
> >> > > > >>> > > > > > > > >
> >> > > > >>> > > > > > > > > [1]
> >> https://cloud.google.com/pubsub/docs/overview
> >> > > > >>> > > > > > > > >
> >> > > > >>> > > > > > > > > *Emmanouil Gkatziouras*
> >> > > > >>> > > > > > > > > https://egkatzioura.com/ |
> >> > > > >>> > > > > > > > >
> >> https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > > >>> > > > > > > > > https://github.com/gkatzioura
> >> > > > >>> > > > > > > > >
> >> > > > >>> > > > > > > > >
> >> > > > >>> > > > > > > > > On Fri, 11 Oct 2019 at 21:45, Denis Magda <
> >> > > > >>> dmagda@apache.org
> >> > > > >>> > >
> >> > > > >>> > > > > wrote:
> >> > > > >>> > > > > > > > >
> >> > > > >>> > > > > > > > > > Hello Emmanouil,
> >> > > > >>> > > > > > > > > >
> >> > > > >>> > > > > > > > > > Are you referring to any specific
> >> implementation
> >> > > > >>> tailored
> >> > > > >>> > > for a
> >> > > > >>> > > > > > > cloud?
> >> > > > >>> > > > > > > > If
> >> > > > >>> > > > > > > > > > I'm not mistaken you were mentioning
> AWS/GCE.
> >> Is
> >> > > > there
> >> > > > >>> any
> >> > > > >>> > > > Github
> >> > > > >>> > > > > > > > source
> >> > > > >>> > > > > > > > > > code?
> >> > > > >>> > > > > > > > > >
> >> > > > >>> > > > > > > > > > Here is a list of existing integrations with
> >> > > > streaming
> >> > > > >>> and
> >> > > > >>> > > > > pub/sub
> >> > > > >>> > > > > > > > > > frameworks:
> >> > > > >>> > > > > > > > > >
> >> https://apacheignite-mix.readme.io/docs/overview
> >> > > > >>> > > > > > > > > >
> >> > > > >>> > > > > > > > > > with the source code in Ignite master:
> >> > > > >>> > > > > > > > > >
> >> > > https://github.com/apache/ignite/tree/master/modules
> >> > > > >>> > > > > > > > > >
> >> > > > >>> > > > > > > > > > Check out those resources to see what are
> the
> >> > most
> >> > > > >>> suitable
> >> > > > >>> > > > > Ignite
> >> > > > >>> > > > > > > APIs
> >> > > > >>> > > > > > > > > for
> >> > > > >>> > > > > > > > > > such integrations.
> >> > > > >>> > > > > > > > > >
> >> > > > >>> > > > > > > > > > @Saikat Maitra <sa...@gmail.com> ,
> >> > @Roman
> >> > > <
> >> > > > >>> > > > > > rshtykh@yahoo.com
> >> > > > >>> > > > > > > >
> >> > > > >>> > > > > > > > ,
> >> > > > >>> > > > > > > > > as
> >> > > > >>> > > > > > > > > > streaming technologies
> >> contributors/maintainers,
> >> > > > could
> >> > > > >>> you
> >> > > > >>> > > > please
> >> > > > >>> > > > > > > join
> >> > > > >>> > > > > > > > > this
> >> > > > >>> > > > > > > > > > thread?
> >> > > > >>> > > > > > > > > >
> >> > > > >>> > > > > > > > > > -
> >> > > > >>> > > > > > > > > > Denis
> >> > > > >>> > > > > > > > > >
> >> > > > >>> > > > > > > > > >
> >> > > > >>> > > > > > > > > > On Thu, Oct 10, 2019 at 7:58 AM Emmanouil
> >> > > > Gkatziouras <
> >> > > > >>> > > > > > > > > > gkatzioura@gmail.com>
> >> > > > >>> > > > > > > > > > wrote:
> >> > > > >>> > > > > > > > > >
> >> > > > >>> > > > > > > > > > > Hello everyone,
> >> > > > >>> > > > > > > > > > >
> >> > > > >>> > > > > > > > > > > I started using Ignite lately. Part of my
> >> work
> >> > > > >>> involves
> >> > > > >>> > > > > Pub/Sub,
> >> > > > >>> > > > > > > > thus I
> >> > > > >>> > > > > > > > > > > created a pub/sub streamer.
> >> > > > >>> > > > > > > > > > > Pub/Sub is pretty close to Kafka.
> >> > > > >>> > > > > > > > > > > Here's how the streamer works. Every node
> >> > creates
> >> > > > an
> >> > > > >>> > > instance
> >> > > > >>> > > > > of
> >> > > > >>> > > > > > a
> >> > > > >>> > > > > > > > > > Pub/Sub
> >> > > > >>> > > > > > > > > > > client.
> >> > > > >>> > > > > > > > > > > The clients subscribe to the same topic
> >> using
> >> > the
> >> > > > >>> same
> >> > > > >>> > > > > > subscription
> >> > > > >>> > > > > > > > > name.
> >> > > > >>> > > > > > > > > > > A message is being sent and one of the
> nodes
> >> > will
> >> > > > >>> receive
> >> > > > >>> > > the
> >> > > > >>> > > > > > > > message,
> >> > > > >>> > > > > > > > > > > process it and then forward it to Ignite
> >> using
> >> > > the
> >> > > > >>> > > extractor
> >> > > > >>> > > > > > > > provided.
> >> > > > >>> > > > > > > > > > >
> >> > > > >>> > > > > > > > > > > Any thoughts?
> >> > > > >>> > > > > > > > > > >
> >> > > > >>> > > > > > > > > > > *Emmanouil Gkatziouras*
> >> > > > >>> > > > > > > > > > > https://egkatzioura.com/ |
> >> > > > >>> > > > > > > > > > >
> >> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> >> > > > >>> > > > > > > > > > > https://github.com/gkatzioura
> >> > > > >>> > > > > > > > > > >
> >> > > > >>> > > > > > > > > >
> >> > > > >>> > > > > > > > >
> >> > > > >>> > > > > > > >
> >> > > > >>> > > > > > >
> >> > > > >>> > > > > >
> >> > > > >>> > > > >
> >> > > > >>> > > >
> >> > > > >>> > >
> >> > > > >>> >
> >> > > > >>>
> >> > > > >>
> >> > > >
> >> > >
> >> >
> >>
> >
>

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra <sa...@gmail.com>]

Hi Emmanouil,

Can you please take a looks at dev utils, if this is something you are
looking for https://github.com/apache/ignite/tree/master/modules/dev-utils?

IMO, if you can release Pub/Sub server in maven and then use it as
dependency, that would be great.

Regards,
Saikat

On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <sa...@gmail.com>
wrote:

> Hi Emmanouil,
>
> The master branch has been initialized. I have raised a PR to migrate
> Flink module to ignite-extensions.
>
> PR https://github.com/apache/ignite-extensions/pull/1
> Jira : https://issues.apache.org/jira/browse/IGNITE-12356
>
> Regards,
> Saikat
>
>
> On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <
> gkatzioura@gmail.com> wrote:
>
>> Hi all!
>>
>> This is an update. The integration is tested with mocks based on the
>> original Ignite repo [1].
>> I am currently waiting for the first commits on the new integration
>> project
>> to take place in order to create my pull request accordingly to the
>> initial
>> commits.
>> In order to simulate the Pub/Sub server in a test I used a test server
>> implementation [2]. Unfortunately they don't have this util uploaded in a
>> maven repo, thus it needs to be hosted on a maven repo (Apache License
>> 2.0). Is there a repo for utilities that Ignite uses for the test/build?
>> I have also created a ticket [3]
>>
>> Kind regards
>>
>> [1] https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
>> [2] https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
>> [3] https://issues.apache.org/jira/browse/IGNITE-12262
>>
>> *Emmanouil Gkatziouras*
>> https://egkatzioura.com/ |
>> https://www.linkedin.com/in/gkatziourasemmanouil/
>> https://github.com/gkatzioura
>>
>>
>> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <sa...@gmail.com>
>> wrote:
>>
>> > Hello Ilya,
>> >
>> > Thank you for your email. Yes, before we remove any integration we will
>> > move them to another repository and release.
>> >
>> > We are thinking we will start with Pub/Sub Streamer implementation and
>> > continue to move other integration as mentioned in the following doc.
>> >
>> >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> >
>> > Regards,
>> > Saikat
>> >
>> >
>> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
>> > ilya.kasnacheev@gmail.com>
>> > wrote:
>> >
>> > > Hello!
>> > >
>> > > My take on this: we want to offload some of integrations, but please
>> > commit
>> > > and release them to another project first, and only then we will
>> decide
>> > to
>> > > remove them from our repository.
>> > >
>> > > Please also make sure they are available with same artifact name and
>> API.
>> > > If this has to be changed, let's wait for AI 3.0
>> > >
>> > > Regards,
>> > > --
>> > > Ilya Kasnacheev
>> > >
>> > >
>> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <sa...@gmail.com>:
>> > >
>> > > > Hello Emmanouil,
>> > > >
>> > > > As discussed earlier, I discussed with Apache Bahir community and
>> they
>> > > are
>> > > > interested to have Apache Ignite extensions as part of Apache Bahir
>> > > > project.
>> > > >
>> > > > Can you please share Pub/Sub streamer implementation details with
>> > Apache
>> > > > Bahir community and request for feedback.
>> > > >
>> > > > https://bahir.apache.org/
>> > > >
>> > > > I have also requested for contributor access in Jira for Apache
>> Bahir
>> > > > project so that I can create issues and assign to myself.
>> > > >
>> > > > I can help with code reviews as well.
>> > > >
>> > > > Here is the email thread for reference
>> > > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
>> > > >
>> > > > Regards,
>> > > > Saikat
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <
>> saikat.maitra@gmail.com
>> > >
>> > > > wrote:
>> > > >
>> > > > > Hello,
>> > > > >
>> > > > > As discussed I have created following discussion threads on our
>> > > migration
>> > > > > proposals for Apache Ignite extensions:
>> > > > >
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
>> > > > >
>> > > > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
>> > > > >
>> > > > > I will share update as I hear more information.
>> > > > >
>> > > > > Regards,
>> > > > > Saikat
>> > > > >
>> > > > >
>> > > > >
>> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra <
>> > saikat.maitra@gmail.com
>> > > >
>> > > > > wrote:
>> > > > >
>> > > > >> Hello Denis,
>> > > > >>
>> > > > >> Sure, sounds good. I will create a separate discussion thread to
>> get
>> > > > >> community feedback on whether we
>> > > > >> should join the Bahir or kick-off "Ignite Extensions" project.
>> > > > >>
>> > > > >> Regards,
>> > > > >> Saikat
>> > > > >>
>> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda <dm...@apache.org>
>> > > wrote:
>> > > > >>
>> > > > >>> Folks,
>> > > > >>>
>> > > > >>> The concept of Apache Bahir is precisely what we need! Saikat,
>> > thanks
>> > > > for
>> > > > >>> doing the research. Why I believe the * idea* fits us well:
>> > > > >>>
>> > > > >>>    - All integrations can be stored in separate Github
>> repositories
>> > > and
>> > > > >>>    have their dev lifecycles. We've not obliged to couple all
>> the
>> > > > >>> integrations
>> > > > >>>    in a single repo. For instance, Spark can be located in a
>> > > dedicated
>> > > > >>> repo
>> > > > >>>    while streaming integrations might be in a single one. It's
>> up
>> > to
>> > > > us.
>> > > > >>>    - All the repositories and integrations belong to ASF. We're
>> not
>> > > > >>> dumping
>> > > > >>>    them on Github but rather keep supporting and developing in
>> > > > >>> accordance with
>> > > > >>>    ASF vision and practices.
>> > > > >>>    - There is a way to reward contributors via committership and
>> > PMC
>> > > > >>>    membership. You won't get this if a project is just one of
>> the
>> > > > >>> millions of
>> > > > >>>    Github projects.
>> > > > >>>
>> > > > >>> Saikat, as Ignite PMC member, could you please kick-off a
>> separate
>> > > > >>> dev-list
>> > > > >>> discussion to involve more community members and referring to
>> this
>> > > > >>> thread?
>> > > > >>> I think the primary question the community needs to answer
>> whether
>> > we
>> > > > >>> should join the Bahir or kick-off "Ignite Extensions" project?
>> > > > >>>
>> > > > >>> -
>> > > > >>> Denis
>> > > > >>>
>> > > > >>>
>> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev <
>> > > > zaleslaw.sin@gmail.com>
>> > > > >>> wrote:
>> > > > >>>
>> > > > >>> > Maybe we could move all our Streaming Integrations there, but
>> > what
>> > > is
>> > > > >>> about
>> > > > >>> > maintaining and committer permissions to the new repositories?
>> > > > >>> > I see the list of the committers and PMC members there
>> > > > >>> > https://bahir.apache.org/community-members/
>> > > > >>> >
>> > > > >>> > Could somebody from Ignite community be added to this list as
>> a
>> > > > >>> > PMC/committer to maintain Ignite integrations?
>> > > > >>> > If yes, I'd happy to join this project and collaborate with
>> these
>> > > > guys
>> > > > >>> > together
>> > > > >>> > If no, and we should start from the zero level with external
>> PRs
>> > -
>> > > > >>> hmmm,
>> > > > >>> > it's better to have our own external repository with
>> ApacheBahirr
>> > > > >>> ideology.
>> > > > >>> >
>> > > > >>> > Also, I agree, that all connectors could be moved there (in
>> > > > >>> ApacheBahirr
>> > > > >>> > like repository), but not all modules from the page
>> > > > >>> >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> > > > >>> > because
>> > > > >>> > they use different parts of core modules and could be
>> developed
>> > > with
>> > > > >>> > different velocity.
>> > > > >>> >
>> > > > >>> > In reality, before creation of new repositories we need wide
>> > > > discussion
>> > > > >>> > based on the document mentioned above.
>> > > > >>> >
>> > > > >>> > P.S. Of course, we could experiment in the next release 2.8
>> with
>> > > one
>> > > > >>> or two
>> > > > >>> > integrations like twitter or ZeroMQ
>> > > > >>> > Also, I'd like Denis Magda idea for the separate repositories
>> in
>> > > > Apache
>> > > > >>> > Github to maintain them by anybody.
>> > > > >>> >
>> > > > >>> >
>> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra <
>> > > saikat.maitra@gmail.com
>> > > > >:
>> > > > >>> >
>> > > > >>> > > Hi Denis, Emmanouil
>> > > > >>> > >
>> > > > >>> > > Thank you for your email. I think creating a separate
>> > integration
>> > > > >>> project
>> > > > >>> > > in github and also proposing it as an apache incubator
>> project
>> > > will
>> > > > >>> be a
>> > > > >>> > > good move. The other separate integration modules
>> > > > >>> > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> > > > >>> > > can
>> > > > >>> > > be moved to this new apache incubator project.
>> > > > >>> > >
>> > > > >>> > > There are similar integration available for Flink and Spark
>> in
>> > > > Apache
>> > > > >>> > Bahir
>> > > > >>> > > https://bahir.apache.org/
>> > > > >>> > >
>> > > > >>> > > Do you think we should reach out to Apache Bahir community
>> > with a
>> > > > >>> > proposal
>> > > > >>> > > or we should plan to create a separate Apache Incubator
>> > project?
>> > > > >>> > >
>> > > > >>> > > Regards,
>> > > > >>> > > Saikat
>> > > > >>> > >
>> > > > >>> > >
>> > > > >>> > >
>> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil Gkatziouras <
>> > > > >>> > > gkatzioura@gmail.com>
>> > > > >>> > > wrote:
>> > > > >>> > >
>> > > > >>> > > > Hi all!
>> > > > >>> > > >
>> > > > >>> > > > Based on the discussions most probably this is going to
>> be on
>> > > > >>> another
>> > > > >>> > > > GitHub repo. Therefore I will prepare a standalone project
>> > with
>> > > > the
>> > > > >>> > > Pub/Sub
>> > > > >>> > > > module and once the repository is created a pull request
>> > shall
>> > > be
>> > > > >>> > issued.
>> > > > >>> > > > If there is anything else I can do in order to facilitate
>> > this
>> > > > >>> please
>> > > > >>> > let
>> > > > >>> > > > me know.
>> > > > >>> > > >
>> > > > >>> > > > If a ticket is created for this issue, I shall be able to
>> > > create
>> > > > a
>> > > > >>> pull
>> > > > >>> > > > request and thus a build will take place on ignite's CI.
>> > > > >>> > > > If everyone is aligned with adding a Pub/Sub
>> implementation
>> > > may I
>> > > > >>> > create
>> > > > >>> > > a
>> > > > >>> > > > ticket on JIRA?
>> > > > >>> > > >
>> > > > >>> > > > My next question has to do on how Ignite configures the
>> build
>> > > > jobs
>> > > > >>> in
>> > > > >>> > > Team
>> > > > >>> > > > City. The unit tests for my implementation need a local
>> > Pub/Sub
>> > > > >>> server
>> > > > >>> > up
>> > > > >>> > > > and running (they provide one for testing purposes). Will
>> > this
>> > > > take
>> > > > >>> > > effect
>> > > > >>> > > > in a form of a build script inside my implementation or
>> is it
>> > > > >>> something
>> > > > >>> > > > that should be configured on Team City?
>> > > > >>> > > >
>> > > > >>> > > > Kind regards,
>> > > > >>> > > > Emmanouil
>> > > > >>> > > >
>> > > > >>> > > >
>> > > > >>> > > > *Emmanouil Gkatziouras*
>> > > > >>> > > > https://egkatzioura.com/ |
>> > > > >>> > > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > >>> > > > https://github.com/gkatzioura
>> > > > >>> > > >
>> > > > >>> > > >
>> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis Magda <
>> dmagda@apache.org
>> > >
>> > > > >>> wrote:
>> > > > >>> > > >
>> > > > >>> > > > > Emmanouil, Saikat,
>> > > > >>> > > > >
>> > > > >>> > > > > After contemplating on this, let me propose another way
>> > with
>> > > > this
>> > > > >>> > > > > particular integration that is aligned with our
>> > > modularization
>> > > > >>> > > endeavor.
>> > > > >>> > > > > The modularization [1] defines the Ignite core, Ignite
>> > > modules,
>> > > > >>> and
>> > > > >>> > > > > independent integrations. The first two are to be
>> located
>> > in
>> > > > >>> Ignite
>> > > > >>> > > > > repositories and to be tested/updated by the community
>> for
>> > > > every
>> > > > >>> > > release.
>> > > > >>> > > > > The modules are a subset of existing integrations that
>> are
>> > > > >>> adopted
>> > > > >>> > > widely
>> > > > >>> > > > > and require community attention.
>> > > > >>> > > > >
>> > > > >>> > > > > As for the independent integrations [1], those are
>> supposed
>> > > to
>> > > > be
>> > > > >>> > > located
>> > > > >>> > > > > in a separate GitHub repo. I would advise us to start
>> > > > >>> implementing
>> > > > >>> > this
>> > > > >>> > > > > concept with the Pub/Sub integration. Even though it
>> will
>> > be
>> > > > >>> located
>> > > > >>> > > in a
>> > > > >>> > > > > separate Github repo, it will be listed among officially
>> > > > >>> available
>> > > > >>> > > > > integrations on the Ignite website featuring Emmanouil
>> as a
>> > > > >>> > > contributor.
>> > > > >>> > > > >
>> > > > >>> > > > > What do you think?
>> > > > >>> > > > >
>> > > > >>> > > > >
>> > > > >>> > > > > [1]
>> > > > >>> > > > >
>> > > > >>> > > > >
>> > > > >>> > > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
>> > > > >>> > > > >
>> > > > >>> > > > >
>> > > > >>> > > > >
>> > > > >>> > > > > -
>> > > > >>> > > > > Denis
>> > > > >>> > > > >
>> > > > >>> > > > >
>> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM Saikat Maitra <
>> > > > >>> > saikat.maitra@gmail.com
>> > > > >>> > > >
>> > > > >>> > > > > wrote:
>> > > > >>> > > > >
>> > > > >>> > > > > > Hi Emmanouil,
>> > > > >>> > > > > >
>> > > > >>> > > > > > The changes looks good to me. I wanted to check if you
>> > are
>> > > > also
>> > > > >>> > able
>> > > > >>> > > to
>> > > > >>> > > > > > validate that the once data is added to Ignite cluster
>> > you
>> > > > are
>> > > > >>> also
>> > > > >>> > > > able
>> > > > >>> > > > > to
>> > > > >>> > > > > > access it using another Ignite client or using rest
>> > > > endpoints.
>> > > > >>> > > > > >
>> > > > >>> > > > > > We use Teamcity for CI process, the details are as
>> > follows
>> > > > >>> > > > > >
>> > > > >>> > > > > > Trigger validation of those test suites that have been
>> > > > >>> affected by
>> > > > >>> > > your
>> > > > >>> > > > > > changes on TeamCity <http://ci.ignite.apache.org/>: <
>> > > > >>> > > > > http://204.14.53.153/
>> > > > >>> > > > > > >
>> > > > >>> > > > > >
>> > > > >>> > > > > >    - Locate a test suite you have to check, press
>> button
>> > > > named
>> > > > >>> > "..."
>> > > > >>> > > > that
>> > > > >>> > > > > >    is located on the left of "Run" button. "Run custom
>> > > build"
>> > > > >>> > dialog
>> > > > >>> > > > will
>> > > > >>> > > > > >    appear;
>> > > > >>> > > > > >    - Go to "Changes" tab and choose
>> > > > >>> > "pull/<pull-request-number>/head"
>> > > > >>> > > > in
>> > > > >>> > > > > >    "Build branch" dropdown list;
>> > > > >>> > > > > >    - Press "Run build" button and monitor tests
>> results.
>> > > > >>> > > > > >
>> > > > >>> > > > > >
>> > > > >>> > > > > >
>> > > > >>> > > > > > @Roman rshtykh@yahoo.com - can you also please take a
>> > > look?
>> > > > >>> > > > > >
>> > > > >>> > > > > > Regards,
>> > > > >>> > > > > > Saikat
>> > > > >>> > > > > >
>> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20 PM Emmanouil Gkatziouras
>> <
>> > > > >>> > > > > > gkatzioura@gmail.com>
>> > > > >>> > > > > > wrote:
>> > > > >>> > > > > >
>> > > > >>> > > > > > > Hi @Saikat Maitra
>> > > > >>> > > > > > >
>> > > > >>> > > > > > > I have my implementation on an Ignite fork on my
>> > personal
>> > > > >>> GitHub
>> > > > >>> > > > > account
>> > > > >>> > > > > > > [1]
>> > > > >>> > > > > > > It is based on the Kafka implementation which is of
>> a
>> > > > polling
>> > > > >>> > > nature.
>> > > > >>> > > > > > > GCP does not provide a service like Kafka. Instead
>> it
>> > > > >>> provides us
>> > > > >>> > > > with
>> > > > >>> > > > > > > Pub/Sub which is just like Kafka.
>> > > > >>> > > > > > > Therefore using the Kafka streamer as a guideline
>> made
>> > > more
>> > > > >>> sense
>> > > > >>> > > to
>> > > > >>> > > > > me.
>> > > > >>> > > > > > >
>> > > > >>> > > > > > > Apart from the implementation I have also added a
>> unit
>> > > > test.
>> > > > >>> > > > > > > It definitely needs some polishing and a double
>> check
>> > on
>> > > my
>> > > > >>> side
>> > > > >>> > to
>> > > > >>> > > > > make
>> > > > >>> > > > > > > the review smooth.
>> > > > >>> > > > > > > For the tests a local Pub/Sub [2] server has to be
>> > setup
>> > > > >>> however
>> > > > >>> > I
>> > > > >>> > > am
>> > > > >>> > > > > not
>> > > > >>> > > > > > > sure which is the best way to do so with regards to
>> > your
>> > > > >>> CI/CD.
>> > > > >>> > > > > > >
>> > > > >>> > > > > > > [1]
>> > > > >>> > >
>> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
>> > > > >>> > > > > > > [2]
>> > > > >>> https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
>> > > > >>> > > > > > >
>> > > > >>> > > > > > > *Emmanouil Gkatziouras*
>> > > > >>> > > > > > > https://egkatzioura.com/ |
>> > > > >>> > > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > >>> > > > > > > https://github.com/gkatzioura
>> > > > >>> > > > > > >
>> > > > >>> > > > > > >
>> > > > >>> > > > > > > On Sat, 12 Oct 2019 at 02:11, Saikat Maitra <
>> > > > >>> > > saikat.maitra@gmail.com
>> > > > >>> > > > >
>> > > > >>> > > > > > > wrote:
>> > > > >>> > > > > > >
>> > > > >>> > > > > > > > Hello Emmanouil,
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > > Can you please share more info about the
>> > > implementation?
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > > We have similar implementation for data streamers
>> > with
>> > > > >>> Kafka
>> > > > >>> > and
>> > > > >>> > > > > > RocketMQ
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > >
>> > https://apacheignite-mix.readme.io/docs/kafka-streamer
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > >
>> > > > https://apacheignite-mix.readme.io/docs/rocketmq-streamer
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > > Please review and share if you have any questions.
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > > Regards,
>> > > > >>> > > > > > > > Saikat
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > > On Fri, Oct 11, 2019 at 4:56 PM Emmanouil
>> > Gkatziouras <
>> > > > >>> > > > > > > > gkatzioura@gmail.com>
>> > > > >>> > > > > > > > wrote:
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > > > Hi all,
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > > By mentioning Pub/Sub I mean Google Cloud's
>> Pub/Sub
>> > > [1]
>> > > > >>> > > > > > > > > Yes it is indeed cloud specific. It is the
>> > messaging
>> > > > >>> service
>> > > > >>> > > > > provided
>> > > > >>> > > > > > > by
>> > > > >>> > > > > > > > > GCP.
>> > > > >>> > > > > > > > > I have made the integration with it, but I
>> haven't
>> > > > >>> published
>> > > > >>> > it
>> > > > >>> > > > yet
>> > > > >>> > > > > > (I
>> > > > >>> > > > > > > am
>> > > > >>> > > > > > > > > waiting for the discussion/ticket and proceed
>> with
>> > > the
>> > > > >>> > > > > contribution).
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > > Kind regards,
>> > > > >>> > > > > > > > > Emmanouil
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > > [1]
>> https://cloud.google.com/pubsub/docs/overview
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > > *Emmanouil Gkatziouras*
>> > > > >>> > > > > > > > > https://egkatzioura.com/ |
>> > > > >>> > > > > > > > >
>> https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > >>> > > > > > > > > https://github.com/gkatzioura
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > > On Fri, 11 Oct 2019 at 21:45, Denis Magda <
>> > > > >>> dmagda@apache.org
>> > > > >>> > >
>> > > > >>> > > > > wrote:
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > > > Hello Emmanouil,
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > > Are you referring to any specific
>> implementation
>> > > > >>> tailored
>> > > > >>> > > for a
>> > > > >>> > > > > > > cloud?
>> > > > >>> > > > > > > > If
>> > > > >>> > > > > > > > > > I'm not mistaken you were mentioning AWS/GCE.
>> Is
>> > > > there
>> > > > >>> any
>> > > > >>> > > > Github
>> > > > >>> > > > > > > > source
>> > > > >>> > > > > > > > > > code?
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > > Here is a list of existing integrations with
>> > > > streaming
>> > > > >>> and
>> > > > >>> > > > > pub/sub
>> > > > >>> > > > > > > > > > frameworks:
>> > > > >>> > > > > > > > > >
>> https://apacheignite-mix.readme.io/docs/overview
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > > with the source code in Ignite master:
>> > > > >>> > > > > > > > > >
>> > > https://github.com/apache/ignite/tree/master/modules
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > > Check out those resources to see what are the
>> > most
>> > > > >>> suitable
>> > > > >>> > > > > Ignite
>> > > > >>> > > > > > > APIs
>> > > > >>> > > > > > > > > for
>> > > > >>> > > > > > > > > > such integrations.
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > > @Saikat Maitra <sa...@gmail.com> ,
>> > @Roman
>> > > <
>> > > > >>> > > > > > rshtykh@yahoo.com
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > > ,
>> > > > >>> > > > > > > > > as
>> > > > >>> > > > > > > > > > streaming technologies
>> contributors/maintainers,
>> > > > could
>> > > > >>> you
>> > > > >>> > > > please
>> > > > >>> > > > > > > join
>> > > > >>> > > > > > > > > this
>> > > > >>> > > > > > > > > > thread?
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > > -
>> > > > >>> > > > > > > > > > Denis
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > > On Thu, Oct 10, 2019 at 7:58 AM Emmanouil
>> > > > Gkatziouras <
>> > > > >>> > > > > > > > > > gkatzioura@gmail.com>
>> > > > >>> > > > > > > > > > wrote:
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > > > Hello everyone,
>> > > > >>> > > > > > > > > > >
>> > > > >>> > > > > > > > > > > I started using Ignite lately. Part of my
>> work
>> > > > >>> involves
>> > > > >>> > > > > Pub/Sub,
>> > > > >>> > > > > > > > thus I
>> > > > >>> > > > > > > > > > > created a pub/sub streamer.
>> > > > >>> > > > > > > > > > > Pub/Sub is pretty close to Kafka.
>> > > > >>> > > > > > > > > > > Here's how the streamer works. Every node
>> > creates
>> > > > an
>> > > > >>> > > instance
>> > > > >>> > > > > of
>> > > > >>> > > > > > a
>> > > > >>> > > > > > > > > > Pub/Sub
>> > > > >>> > > > > > > > > > > client.
>> > > > >>> > > > > > > > > > > The clients subscribe to the same topic
>> using
>> > the
>> > > > >>> same
>> > > > >>> > > > > > subscription
>> > > > >>> > > > > > > > > name.
>> > > > >>> > > > > > > > > > > A message is being sent and one of the nodes
>> > will
>> > > > >>> receive
>> > > > >>> > > the
>> > > > >>> > > > > > > > message,
>> > > > >>> > > > > > > > > > > process it and then forward it to Ignite
>> using
>> > > the
>> > > > >>> > > extractor
>> > > > >>> > > > > > > > provided.
>> > > > >>> > > > > > > > > > >
>> > > > >>> > > > > > > > > > > Any thoughts?
>> > > > >>> > > > > > > > > > >
>> > > > >>> > > > > > > > > > > *Emmanouil Gkatziouras*
>> > > > >>> > > > > > > > > > > https://egkatzioura.com/ |
>> > > > >>> > > > > > > > > > >
>> > > https://www.linkedin.com/in/gkatziourasemmanouil/
>> > > > >>> > > > > > > > > > > https://github.com/gkatzioura
>> > > > >>> > > > > > > > > > >
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > >
>> > > > >>> > > > > >
>> > > > >>> > > > >
>> > > > >>> > > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > > >>
>> > > >
>> > >
>> >
>>
>

Re: [DISCUSS] Pub/Sub Streamer Implementation

[Saikat Maitra <sa...@gmail.com>]

Hi Emmanouil,

The master branch has been initialized. I have raised a PR to migrate Flink
module to ignite-extensions.

PR https://github.com/apache/ignite-extensions/pull/1
Jira : https://issues.apache.org/jira/browse/IGNITE-12356

Regards,
Saikat


On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras <gk...@gmail.com>
wrote:

> Hi all!
>
> This is an update. The integration is tested with mocks based on the
> original Ignite repo [1].
> I am currently waiting for the first commits on the new integration project
> to take place in order to create my pull request accordingly to the initial
> commits.
> In order to simulate the Pub/Sub server in a test I used a test server
> implementation [2]. Unfortunately they don't have this util uploaded in a
> maven repo, thus it needs to be hosted on a maven repo (Apache License
> 2.0). Is there a repo for utilities that Ignite uses for the test/build?
> I have also created a ticket [3]
>
> Kind regards
>
> [1] https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> [2] https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator
> [3] https://issues.apache.org/jira/browse/IGNITE-12262
>
> *Emmanouil Gkatziouras*
> https://egkatzioura.com/ |
> https://www.linkedin.com/in/gkatziourasemmanouil/
> https://github.com/gkatzioura
>
>
> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <sa...@gmail.com>
> wrote:
>
> > Hello Ilya,
> >
> > Thank you for your email. Yes, before we remove any integration we will
> > move them to another repository and release.
> >
> > We are thinking we will start with Pub/Sub Streamer implementation and
> > continue to move other integration as mentioned in the following doc.
> >
> >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> >
> > Regards,
> > Saikat
> >
> >
> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev <
> > ilya.kasnacheev@gmail.com>
> > wrote:
> >
> > > Hello!
> > >
> > > My take on this: we want to offload some of integrations, but please
> > commit
> > > and release them to another project first, and only then we will decide
> > to
> > > remove them from our repository.
> > >
> > > Please also make sure they are available with same artifact name and
> API.
> > > If this has to be changed, let's wait for AI 3.0
> > >
> > > Regards,
> > > --
> > > Ilya Kasnacheev
> > >
> > >
> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra <sa...@gmail.com>:
> > >
> > > > Hello Emmanouil,
> > > >
> > > > As discussed earlier, I discussed with Apache Bahir community and
> they
> > > are
> > > > interested to have Apache Ignite extensions as part of Apache Bahir
> > > > project.
> > > >
> > > > Can you please share Pub/Sub streamer implementation details with
> > Apache
> > > > Bahir community and request for feedback.
> > > >
> > > > https://bahir.apache.org/
> > > >
> > > > I have also requested for contributor access in Jira for Apache Bahir
> > > > project so that I can create issues and assign to myself.
> > > >
> > > > I can help with code reviews as well.
> > > >
> > > > Here is the email thread for reference
> > > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > > >
> > > > Regards,
> > > > Saikat
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra <
> saikat.maitra@gmail.com
> > >
> > > > wrote:
> > > >
> > > > > Hello,
> > > > >
> > > > > As discussed I have created following discussion threads on our
> > > migration
> > > > > proposals for Apache Ignite extensions:
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html
> > > > >
> > > > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html
> > > > >
> > > > > I will share update as I hear more information.
> > > > >
> > > > > Regards,
> > > > > Saikat
> > > > >
> > > > >
> > > > >
> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra <
> > saikat.maitra@gmail.com
> > > >
> > > > > wrote:
> > > > >
> > > > >> Hello Denis,
> > > > >>
> > > > >> Sure, sounds good. I will create a separate discussion thread to
> get
> > > > >> community feedback on whether we
> > > > >> should join the Bahir or kick-off "Ignite Extensions" project.
> > > > >>
> > > > >> Regards,
> > > > >> Saikat
> > > > >>
> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda <dm...@apache.org>
> > > wrote:
> > > > >>
> > > > >>> Folks,
> > > > >>>
> > > > >>> The concept of Apache Bahir is precisely what we need! Saikat,
> > thanks
> > > > for
> > > > >>> doing the research. Why I believe the * idea* fits us well:
> > > > >>>
> > > > >>>    - All integrations can be stored in separate Github
> repositories
> > > and
> > > > >>>    have their dev lifecycles. We've not obliged to couple all the
> > > > >>> integrations
> > > > >>>    in a single repo. For instance, Spark can be located in a
> > > dedicated
> > > > >>> repo
> > > > >>>    while streaming integrations might be in a single one. It's up
> > to
> > > > us.
> > > > >>>    - All the repositories and integrations belong to ASF. We're
> not
> > > > >>> dumping
> > > > >>>    them on Github but rather keep supporting and developing in
> > > > >>> accordance with
> > > > >>>    ASF vision and practices.
> > > > >>>    - There is a way to reward contributors via committership and
> > PMC
> > > > >>>    membership. You won't get this if a project is just one of the
> > > > >>> millions of
> > > > >>>    Github projects.
> > > > >>>
> > > > >>> Saikat, as Ignite PMC member, could you please kick-off a
> separate
> > > > >>> dev-list
> > > > >>> discussion to involve more community members and referring to
> this
> > > > >>> thread?
> > > > >>> I think the primary question the community needs to answer
> whether
> > we
> > > > >>> should join the Bahir or kick-off "Ignite Extensions" project?
> > > > >>>
> > > > >>> -
> > > > >>> Denis
> > > > >>>
> > > > >>>
> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev <
> > > > zaleslaw.sin@gmail.com>
> > > > >>> wrote:
> > > > >>>
> > > > >>> > Maybe we could move all our Streaming Integrations there, but
> > what
> > > is
> > > > >>> about
> > > > >>> > maintaining and committer permissions to the new repositories?
> > > > >>> > I see the list of the committers and PMC members there
> > > > >>> > https://bahir.apache.org/community-members/
> > > > >>> >
> > > > >>> > Could somebody from Ignite community be added to this list as a
> > > > >>> > PMC/committer to maintain Ignite integrations?
> > > > >>> > If yes, I'd happy to join this project and collaborate with
> these
> > > > guys
> > > > >>> > together
> > > > >>> > If no, and we should start from the zero level with external
> PRs
> > -
> > > > >>> hmmm,
> > > > >>> > it's better to have our own external repository with
> ApacheBahirr
> > > > >>> ideology.
> > > > >>> >
> > > > >>> > Also, I agree, that all connectors could be moved there (in
> > > > >>> ApacheBahirr
> > > > >>> > like repository), but not all modules from the page
> > > > >>> >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > >>> > because
> > > > >>> > they use different parts of core modules and could be developed
> > > with
> > > > >>> > different velocity.
> > > > >>> >
> > > > >>> > In reality, before creation of new repositories we need wide
> > > > discussion
> > > > >>> > based on the document mentioned above.
> > > > >>> >
> > > > >>> > P.S. Of course, we could experiment in the next release 2.8
> with
> > > one
> > > > >>> or two
> > > > >>> > integrations like twitter or ZeroMQ
> > > > >>> > Also, I'd like Denis Magda idea for the separate repositories
> in
> > > > Apache
> > > > >>> > Github to maintain them by anybody.
> > > > >>> >
> > > > >>> >
> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra <
> > > saikat.maitra@gmail.com
> > > > >:
> > > > >>> >
> > > > >>> > > Hi Denis, Emmanouil
> > > > >>> > >
> > > > >>> > > Thank you for your email. I think creating a separate
> > integration
> > > > >>> project
> > > > >>> > > in github and also proposing it as an apache incubator
> project
> > > will
> > > > >>> be a
> > > > >>> > > good move. The other separate integration modules
> > > > >>> > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > >>> > > can
> > > > >>> > > be moved to this new apache incubator project.
> > > > >>> > >
> > > > >>> > > There are similar integration available for Flink and Spark
> in
> > > > Apache
> > > > >>> > Bahir
> > > > >>> > > https://bahir.apache.org/
> > > > >>> > >
> > > > >>> > > Do you think we should reach out to Apache Bahir community
> > with a
> > > > >>> > proposal
> > > > >>> > > or we should plan to create a separate Apache Incubator
> > project?
> > > > >>> > >
> > > > >>> > > Regards,
> > > > >>> > > Saikat
> > > > >>> > >
> > > > >>> > >
> > > > >>> > >
> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil Gkatziouras <
> > > > >>> > > gkatzioura@gmail.com>
> > > > >>> > > wrote:
> > > > >>> > >
> > > > >>> > > > Hi all!
> > > > >>> > > >
> > > > >>> > > > Based on the discussions most probably this is going to be
> on
> > > > >>> another
> > > > >>> > > > GitHub repo. Therefore I will prepare a standalone project
> > with
> > > > the
> > > > >>> > > Pub/Sub
> > > > >>> > > > module and once the repository is created a pull request
> > shall
> > > be
> > > > >>> > issued.
> > > > >>> > > > If there is anything else I can do in order to facilitate
> > this
> > > > >>> please
> > > > >>> > let
> > > > >>> > > > me know.
> > > > >>> > > >
> > > > >>> > > > If a ticket is created for this issue, I shall be able to
> > > create
> > > > a
> > > > >>> pull
> > > > >>> > > > request and thus a build will take place on ignite's CI.
> > > > >>> > > > If everyone is aligned with adding a Pub/Sub implementation
> > > may I
> > > > >>> > create
> > > > >>> > > a
> > > > >>> > > > ticket on JIRA?
> > > > >>> > > >
> > > > >>> > > > My next question has to do on how Ignite configures the
> build
> > > > jobs
> > > > >>> in
> > > > >>> > > Team
> > > > >>> > > > City. The unit tests for my implementation need a local
> > Pub/Sub
> > > > >>> server
> > > > >>> > up
> > > > >>> > > > and running (they provide one for testing purposes). Will
> > this
> > > > take
> > > > >>> > > effect
> > > > >>> > > > in a form of a build script inside my implementation or is
> it
> > > > >>> something
> > > > >>> > > > that should be configured on Team City?
> > > > >>> > > >
> > > > >>> > > > Kind regards,
> > > > >>> > > > Emmanouil
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > > > *Emmanouil Gkatziouras*
> > > > >>> > > > https://egkatzioura.com/ |
> > > > >>> > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > >>> > > > https://github.com/gkatzioura
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis Magda <
> dmagda@apache.org
> > >
> > > > >>> wrote:
> > > > >>> > > >
> > > > >>> > > > > Emmanouil, Saikat,
> > > > >>> > > > >
> > > > >>> > > > > After contemplating on this, let me propose another way
> > with
> > > > this
> > > > >>> > > > > particular integration that is aligned with our
> > > modularization
> > > > >>> > > endeavor.
> > > > >>> > > > > The modularization [1] defines the Ignite core, Ignite
> > > modules,
> > > > >>> and
> > > > >>> > > > > independent integrations. The first two are to be located
> > in
> > > > >>> Ignite
> > > > >>> > > > > repositories and to be tested/updated by the community
> for
> > > > every
> > > > >>> > > release.
> > > > >>> > > > > The modules are a subset of existing integrations that
> are
> > > > >>> adopted
> > > > >>> > > widely
> > > > >>> > > > > and require community attention.
> > > > >>> > > > >
> > > > >>> > > > > As for the independent integrations [1], those are
> supposed
> > > to
> > > > be
> > > > >>> > > located
> > > > >>> > > > > in a separate GitHub repo. I would advise us to start
> > > > >>> implementing
> > > > >>> > this
> > > > >>> > > > > concept with the Pub/Sub integration. Even though it will
> > be
> > > > >>> located
> > > > >>> > > in a
> > > > >>> > > > > separate Github repo, it will be listed among officially
> > > > >>> available
> > > > >>> > > > > integrations on the Ignite website featuring Emmanouil
> as a
> > > > >>> > > contributor.
> > > > >>> > > > >
> > > > >>> > > > > What do you think?
> > > > >>> > > > >
> > > > >>> > > > >
> > > > >>> > > > > [1]
> > > > >>> > > > >
> > > > >>> > > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations
> > > > >>> > > > >
> > > > >>> > > > >
> > > > >>> > > > >
> > > > >>> > > > > -
> > > > >>> > > > > Denis
> > > > >>> > > > >
> > > > >>> > > > >
> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM Saikat Maitra <
> > > > >>> > saikat.maitra@gmail.com
> > > > >>> > > >
> > > > >>> > > > > wrote:
> > > > >>> > > > >
> > > > >>> > > > > > Hi Emmanouil,
> > > > >>> > > > > >
> > > > >>> > > > > > The changes looks good to me. I wanted to check if you
> > are
> > > > also
> > > > >>> > able
> > > > >>> > > to
> > > > >>> > > > > > validate that the once data is added to Ignite cluster
> > you
> > > > are
> > > > >>> also
> > > > >>> > > > able
> > > > >>> > > > > to
> > > > >>> > > > > > access it using another Ignite client or using rest
> > > > endpoints.
> > > > >>> > > > > >
> > > > >>> > > > > > We use Teamcity for CI process, the details are as
> > follows
> > > > >>> > > > > >
> > > > >>> > > > > > Trigger validation of those test suites that have been
> > > > >>> affected by
> > > > >>> > > your
> > > > >>> > > > > > changes on TeamCity <http://ci.ignite.apache.org/>: <
> > > > >>> > > > > http://204.14.53.153/
> > > > >>> > > > > > >
> > > > >>> > > > > >
> > > > >>> > > > > >    - Locate a test suite you have to check, press
> button
> > > > named
> > > > >>> > "..."
> > > > >>> > > > that
> > > > >>> > > > > >    is located on the left of "Run" button. "Run custom
> > > build"
> > > > >>> > dialog
> > > > >>> > > > will
> > > > >>> > > > > >    appear;
> > > > >>> > > > > >    - Go to "Changes" tab and choose
> > > > >>> > "pull/<pull-request-number>/head"
> > > > >>> > > > in
> > > > >>> > > > > >    "Build branch" dropdown list;
> > > > >>> > > > > >    - Press "Run build" button and monitor tests
> results.
> > > > >>> > > > > >
> > > > >>> > > > > >
> > > > >>> > > > > >
> > > > >>> > > > > > @Roman rshtykh@yahoo.com - can you also please take a
> > > look?
> > > > >>> > > > > >
> > > > >>> > > > > > Regards,
> > > > >>> > > > > > Saikat
> > > > >>> > > > > >
> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20 PM Emmanouil Gkatziouras <
> > > > >>> > > > > > gkatzioura@gmail.com>
> > > > >>> > > > > > wrote:
> > > > >>> > > > > >
> > > > >>> > > > > > > Hi @Saikat Maitra
> > > > >>> > > > > > >
> > > > >>> > > > > > > I have my implementation on an Ignite fork on my
> > personal
> > > > >>> GitHub
> > > > >>> > > > > account
> > > > >>> > > > > > > [1]
> > > > >>> > > > > > > It is based on the Kafka implementation which is of a
> > > > polling
> > > > >>> > > nature.
> > > > >>> > > > > > > GCP does not provide a service like Kafka. Instead it
> > > > >>> provides us
> > > > >>> > > > with
> > > > >>> > > > > > > Pub/Sub which is just like Kafka.
> > > > >>> > > > > > > Therefore using the Kafka streamer as a guideline
> made
> > > more
> > > > >>> sense
> > > > >>> > > to
> > > > >>> > > > > me.
> > > > >>> > > > > > >
> > > > >>> > > > > > > Apart from the implementation I have also added a
> unit
> > > > test.
> > > > >>> > > > > > > It definitely needs some polishing and a double check
> > on
> > > my
> > > > >>> side
> > > > >>> > to
> > > > >>> > > > > make
> > > > >>> > > > > > > the review smooth.
> > > > >>> > > > > > > For the tests a local Pub/Sub [2] server has to be
> > setup
> > > > >>> however
> > > > >>> > I
> > > > >>> > > am
> > > > >>> > > > > not
> > > > >>> > > > > > > sure which is the best way to do so with regards to
> > your
> > > > >>> CI/CD.
> > > > >>> > > > > > >
> > > > >>> > > > > > > [1]
> > > > >>> > >
> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub
> > > > >>> > > > > > > [2]
> > > > >>> https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/
> > > > >>> > > > > > >
> > > > >>> > > > > > > *Emmanouil Gkatziouras*
> > > > >>> > > > > > > https://egkatzioura.com/ |
> > > > >>> > > > > > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > >>> > > > > > > https://github.com/gkatzioura
> > > > >>> > > > > > >
> > > > >>> > > > > > >
> > > > >>> > > > > > > On Sat, 12 Oct 2019 at 02:11, Saikat Maitra <
> > > > >>> > > saikat.maitra@gmail.com
> > > > >>> > > > >
> > > > >>> > > > > > > wrote:
> > > > >>> > > > > > >
> > > > >>> > > > > > > > Hello Emmanouil,
> > > > >>> > > > > > > >
> > > > >>> > > > > > > > Can you please share more info about the
> > > implementation?
> > > > >>> > > > > > > >
> > > > >>> > > > > > > > We have similar implementation for data streamers
> > with
> > > > >>> Kafka
> > > > >>> > and
> > > > >>> > > > > > RocketMQ
> > > > >>> > > > > > > >
> > > > >>> > > > > > > >
> > https://apacheignite-mix.readme.io/docs/kafka-streamer
> > > > >>> > > > > > > >
> > > > >>> > > > > > > >
> > > > https://apacheignite-mix.readme.io/docs/rocketmq-streamer
> > > > >>> > > > > > > >
> > > > >>> > > > > > > > Please review and share if you have any questions.
> > > > >>> > > > > > > >
> > > > >>> > > > > > > > Regards,
> > > > >>> > > > > > > > Saikat
> > > > >>> > > > > > > >
> > > > >>> > > > > > > >
> > > > >>> > > > > > > >
> > > > >>> > > > > > > > On Fri, Oct 11, 2019 at 4:56 PM Emmanouil
> > Gkatziouras <
> > > > >>> > > > > > > > gkatzioura@gmail.com>
> > > > >>> > > > > > > > wrote:
> > > > >>> > > > > > > >
> > > > >>> > > > > > > > > Hi all,
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > > By mentioning Pub/Sub I mean Google Cloud's
> Pub/Sub
> > > [1]
> > > > >>> > > > > > > > > Yes it is indeed cloud specific. It is the
> > messaging
> > > > >>> service
> > > > >>> > > > > provided
> > > > >>> > > > > > > by
> > > > >>> > > > > > > > > GCP.
> > > > >>> > > > > > > > > I have made the integration with it, but I
> haven't
> > > > >>> published
> > > > >>> > it
> > > > >>> > > > yet
> > > > >>> > > > > > (I
> > > > >>> > > > > > > am
> > > > >>> > > > > > > > > waiting for the discussion/ticket and proceed
> with
> > > the
> > > > >>> > > > > contribution).
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > > Kind regards,
> > > > >>> > > > > > > > > Emmanouil
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > > [1]
> https://cloud.google.com/pubsub/docs/overview
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > > *Emmanouil Gkatziouras*
> > > > >>> > > > > > > > > https://egkatzioura.com/ |
> > > > >>> > > > > > > > >
> https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > >>> > > > > > > > > https://github.com/gkatzioura
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > > On Fri, 11 Oct 2019 at 21:45, Denis Magda <
> > > > >>> dmagda@apache.org
> > > > >>> > >
> > > > >>> > > > > wrote:
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > > > Hello Emmanouil,
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > > Are you referring to any specific
> implementation
> > > > >>> tailored
> > > > >>> > > for a
> > > > >>> > > > > > > cloud?
> > > > >>> > > > > > > > If
> > > > >>> > > > > > > > > > I'm not mistaken you were mentioning AWS/GCE.
> Is
> > > > there
> > > > >>> any
> > > > >>> > > > Github
> > > > >>> > > > > > > > source
> > > > >>> > > > > > > > > > code?
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > > Here is a list of existing integrations with
> > > > streaming
> > > > >>> and
> > > > >>> > > > > pub/sub
> > > > >>> > > > > > > > > > frameworks:
> > > > >>> > > > > > > > > >
> https://apacheignite-mix.readme.io/docs/overview
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > > with the source code in Ignite master:
> > > > >>> > > > > > > > > >
> > > https://github.com/apache/ignite/tree/master/modules
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > > Check out those resources to see what are the
> > most
> > > > >>> suitable
> > > > >>> > > > > Ignite
> > > > >>> > > > > > > APIs
> > > > >>> > > > > > > > > for
> > > > >>> > > > > > > > > > such integrations.
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > > @Saikat Maitra <sa...@gmail.com> ,
> > @Roman
> > > <
> > > > >>> > > > > > rshtykh@yahoo.com
> > > > >>> > > > > > > >
> > > > >>> > > > > > > > ,
> > > > >>> > > > > > > > > as
> > > > >>> > > > > > > > > > streaming technologies
> contributors/maintainers,
> > > > could
> > > > >>> you
> > > > >>> > > > please
> > > > >>> > > > > > > join
> > > > >>> > > > > > > > > this
> > > > >>> > > > > > > > > > thread?
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > > -
> > > > >>> > > > > > > > > > Denis
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > > On Thu, Oct 10, 2019 at 7:58 AM Emmanouil
> > > > Gkatziouras <
> > > > >>> > > > > > > > > > gkatzioura@gmail.com>
> > > > >>> > > > > > > > > > wrote:
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > > > Hello everyone,
> > > > >>> > > > > > > > > > >
> > > > >>> > > > > > > > > > > I started using Ignite lately. Part of my
> work
> > > > >>> involves
> > > > >>> > > > > Pub/Sub,
> > > > >>> > > > > > > > thus I
> > > > >>> > > > > > > > > > > created a pub/sub streamer.
> > > > >>> > > > > > > > > > > Pub/Sub is pretty close to Kafka.
> > > > >>> > > > > > > > > > > Here's how the streamer works. Every node
> > creates
> > > > an
> > > > >>> > > instance
> > > > >>> > > > > of
> > > > >>> > > > > > a
> > > > >>> > > > > > > > > > Pub/Sub
> > > > >>> > > > > > > > > > > client.
> > > > >>> > > > > > > > > > > The clients subscribe to the same topic using
> > the
> > > > >>> same
> > > > >>> > > > > > subscription
> > > > >>> > > > > > > > > name.
> > > > >>> > > > > > > > > > > A message is being sent and one of the nodes
> > will
> > > > >>> receive
> > > > >>> > > the
> > > > >>> > > > > > > > message,
> > > > >>> > > > > > > > > > > process it and then forward it to Ignite
> using
> > > the
> > > > >>> > > extractor
> > > > >>> > > > > > > > provided.
> > > > >>> > > > > > > > > > >
> > > > >>> > > > > > > > > > > Any thoughts?
> > > > >>> > > > > > > > > > >
> > > > >>> > > > > > > > > > > *Emmanouil Gkatziouras*
> > > > >>> > > > > > > > > > > https://egkatzioura.com/ |
> > > > >>> > > > > > > > > > >
> > > https://www.linkedin.com/in/gkatziourasemmanouil/
> > > > >>> > > > > > > > > > > https://github.com/gkatzioura
> > > > >>> > > > > > > > > > >
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > >
> > > > >>> > > > > > >
> > > > >>> > > > > >
> > > > >>> > > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > > >>
> > > >
> > >
> >
>