You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@samza.apache.org by Chris Riccomini <cr...@apache.org> on 2015/02/04 19:15:30 UTC
[DISCUSS] SQL workflow
Hey all,
Yi, Navina, and Milinda have been working on SAMZA-390 sub-tickets related
to SQL operators. We're getting to the point where the amount of work
floating around is quite large, and some tickets build off of others.
I'm proposing that we commit this work into a samza-sql submodule on
master, and treat this module as experimental. I want to avoid branches,
and I also want to avoid revision control over JIRA. This means that there
will probably be a fair amount of commits/JIRAs on this module as we
iterate, but I think that's OK.
Does this sound good to everyone?
Cheers,
Chris
Re: [DISCUSS] SQL workflow
Posted by Yi Pan <ni...@gmail.com>.
Hi, Jacob,
Yes, for sure.
-Yi
On Wed, Feb 4, 2015 at 11:43 AM, Jakob Homan <jg...@gmail.com> wrote:
> This submodule will still be under the review-then-commit (RTC)
> regime, correct?
>
> On 4 February 2015 at 11:13, Yi Pan <ni...@gmail.com> wrote:
> > Just did the update w/ SAMZA-482.
> >
> > On Wed, Feb 4, 2015 at 10:31 AM, Chris Riccomini <cr...@apache.org>
> > wrote:
> >
> >> I think so. There was some RB downtime, but it just got fixed. Yi,
> Navina,
> >> Milinda, can you make sure your JIRAs have up to date RBs?
> >>
> >> On Wed, Feb 4, 2015 at 10:24 AM, sriram <sr...@gmail.com> wrote:
> >>
> >> > Can we have updated RBs for all the three sub tasks before we commit?
> >> This
> >> > would help us to review even after we commit.
> >> >
> >> > On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini <
> criccomini@apache.org>
> >> > wrote:
> >> >
> >> > > Hey all,
> >> > >
> >> > > Yi, Navina, and Milinda have been working on SAMZA-390 sub-tickets
> >> > related
> >> > > to SQL operators. We're getting to the point where the amount of
> work
> >> > > floating around is quite large, and some tickets build off of
> others.
> >> > >
> >> > > I'm proposing that we commit this work into a samza-sql submodule on
> >> > > master, and treat this module as experimental. I want to avoid
> >> branches,
> >> > > and I also want to avoid revision control over JIRA. This means that
> >> > there
> >> > > will probably be a fair amount of commits/JIRAs on this module as we
> >> > > iterate, but I think that's OK.
> >> > >
> >> > > Does this sound good to everyone?
> >> > >
> >> > > Cheers,
> >> > > Chris
> >> > >
> >> >
> >>
>
Re: [DISCUSS] SQL workflow
Posted by Jakob Homan <jg...@gmail.com>.
This submodule will still be under the review-then-commit (RTC)
regime, correct?
On 4 February 2015 at 11:13, Yi Pan <ni...@gmail.com> wrote:
> Just did the update w/ SAMZA-482.
>
> On Wed, Feb 4, 2015 at 10:31 AM, Chris Riccomini <cr...@apache.org>
> wrote:
>
>> I think so. There was some RB downtime, but it just got fixed. Yi, Navina,
>> Milinda, can you make sure your JIRAs have up to date RBs?
>>
>> On Wed, Feb 4, 2015 at 10:24 AM, sriram <sr...@gmail.com> wrote:
>>
>> > Can we have updated RBs for all the three sub tasks before we commit?
>> This
>> > would help us to review even after we commit.
>> >
>> > On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini <cr...@apache.org>
>> > wrote:
>> >
>> > > Hey all,
>> > >
>> > > Yi, Navina, and Milinda have been working on SAMZA-390 sub-tickets
>> > related
>> > > to SQL operators. We're getting to the point where the amount of work
>> > > floating around is quite large, and some tickets build off of others.
>> > >
>> > > I'm proposing that we commit this work into a samza-sql submodule on
>> > > master, and treat this module as experimental. I want to avoid
>> branches,
>> > > and I also want to avoid revision control over JIRA. This means that
>> > there
>> > > will probably be a fair amount of commits/JIRAs on this module as we
>> > > iterate, but I think that's OK.
>> > >
>> > > Does this sound good to everyone?
>> > >
>> > > Cheers,
>> > > Chris
>> > >
>> >
>>
Re: [DISCUSS] SQL workflow
Posted by Navina Ramesh <nr...@linkedin.com.INVALID>.
I have updates SAMZA-484 and RB is here -
https://reviews.apache.org/r/30634/diff/#
Navina
On 2/4/15, 11:13 AM, "Yi Pan" <ni...@gmail.com> wrote:
>Just did the update w/ SAMZA-482.
>
>On Wed, Feb 4, 2015 at 10:31 AM, Chris Riccomini <cr...@apache.org>
>wrote:
>
>> I think so. There was some RB downtime, but it just got fixed. Yi,
>>Navina,
>> Milinda, can you make sure your JIRAs have up to date RBs?
>>
>> On Wed, Feb 4, 2015 at 10:24 AM, sriram <sr...@gmail.com> wrote:
>>
>> > Can we have updated RBs for all the three sub tasks before we commit?
>> This
>> > would help us to review even after we commit.
>> >
>> > On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini
>><cr...@apache.org>
>> > wrote:
>> >
>> > > Hey all,
>> > >
>> > > Yi, Navina, and Milinda have been working on SAMZA-390 sub-tickets
>> > related
>> > > to SQL operators. We're getting to the point where the amount of
>>work
>> > > floating around is quite large, and some tickets build off of
>>others.
>> > >
>> > > I'm proposing that we commit this work into a samza-sql submodule on
>> > > master, and treat this module as experimental. I want to avoid
>> branches,
>> > > and I also want to avoid revision control over JIRA. This means that
>> > there
>> > > will probably be a fair amount of commits/JIRAs on this module as we
>> > > iterate, but I think that's OK.
>> > >
>> > > Does this sound good to everyone?
>> > >
>> > > Cheers,
>> > > Chris
>> > >
>> >
>>
Re: [DISCUSS] SQL workflow
Posted by Yi Pan <ni...@gmail.com>.
Just did the update w/ SAMZA-482.
On Wed, Feb 4, 2015 at 10:31 AM, Chris Riccomini <cr...@apache.org>
wrote:
> I think so. There was some RB downtime, but it just got fixed. Yi, Navina,
> Milinda, can you make sure your JIRAs have up to date RBs?
>
> On Wed, Feb 4, 2015 at 10:24 AM, sriram <sr...@gmail.com> wrote:
>
> > Can we have updated RBs for all the three sub tasks before we commit?
> This
> > would help us to review even after we commit.
> >
> > On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini <cr...@apache.org>
> > wrote:
> >
> > > Hey all,
> > >
> > > Yi, Navina, and Milinda have been working on SAMZA-390 sub-tickets
> > related
> > > to SQL operators. We're getting to the point where the amount of work
> > > floating around is quite large, and some tickets build off of others.
> > >
> > > I'm proposing that we commit this work into a samza-sql submodule on
> > > master, and treat this module as experimental. I want to avoid
> branches,
> > > and I also want to avoid revision control over JIRA. This means that
> > there
> > > will probably be a fair amount of commits/JIRAs on this module as we
> > > iterate, but I think that's OK.
> > >
> > > Does this sound good to everyone?
> > >
> > > Cheers,
> > > Chris
> > >
> >
>
Re: [DISCUSS] SQL workflow
Posted by Yi Pan <ni...@gmail.com>.
+1
On Sat, Feb 7, 2015 at 8:38 AM, Chris Riccomini <cr...@apache.org>
wrote:
> Hey all,
>
> Are we +1 on this? I think Jakob was the only one who was curious about it.
>
> Cheers,
> Chris
>
> On Thu, Feb 5, 2015 at 1:22 PM, Yi Pan <ni...@gmail.com> wrote:
>
> > Hi, Jakob,
> >
> >
> > > > Eh? Not sure what this means...
> > >
> > > I mean SAMZA-484 depends on SAMZA-482, and neither are committed. So
> > Navina
> > > is having to post Yi's patch, as well as her own, on the JIRA. It makes
> > it
> > > really hard to do code reviews because you can't tell whether Yi made
> the
> > > changes or Navina did.
> > >
> > >
> > Just to add to the point. It is also difficult to always see a long list
> of
> > changed files if the RB request is always based on the master. It is
> > possible to have RB request based on another RB request (I have tried it
> > before). But what happens if the base RB request is cancelled/discarded?
> RB
> > is not designed to track the revision changes in a dependency chain.
> >
> >
> >
> > > Cheers,
> > > Chris
> > >
> > > On Thu, Feb 5, 2015 at 12:16 PM, Jakob Homan <jg...@gmail.com>
> wrote:
> > >
> > > > > I want to avoid branches,
> > > > Just curious, any reason for this?
> > > >
> > > > > and I also want to avoid revision control over JIRA
> > > > Eh? Not sure what this means...
> > > >
> > > > Thanks,
> > > > jg
> > > >
> > > > On 4 February 2015 at 17:11, Chris Riccomini <cr...@apache.org>
> > > > wrote:
> > > > > Hey all,
> > > > >
> > > > > @Jakob, yeah I was thinking we'll follow our normal flow. RTC. I
> just
> > > > > wanted to set expectation that the code committed might be not up
> to
> > > our
> > > > > normal quality initially (missing docs, no tests, etc). Until the
> > > quality
> > > > > is raised, we should think of this module as experimental.
> > > > >
> > > > > @Milinda, awesome! Thanks. :)
> > > > >
> > > > > Cheers,
> > > > > Chris
> > > > >
> > > > > On Wed, Feb 4, 2015 at 11:57 AM, Milinda Pathirage <
> > > > mpathira@umail.iu.edu>
> > > > > wrote:
> > > > >
> > > > >> Hi Chris,
> > > > >>
> > > > >> Hope we no longer need the SQL API. I'll create a RB for Calcite
> > > > >> integration.
> > > > >>
> > > > >> Thanks
> > > > >> Milinda
> > > > >>
> > > > >> On Wed, Feb 4, 2015 at 1:31 PM, Chris Riccomini <
> > > criccomini@apache.org>
> > > > >> wrote:
> > > > >>
> > > > >> > I think so. There was some RB downtime, but it just got fixed.
> Yi,
> > > > >> Navina,
> > > > >> > Milinda, can you make sure your JIRAs have up to date RBs?
> > > > >> >
> > > > >> > On Wed, Feb 4, 2015 at 10:24 AM, sriram <sr...@gmail.com>
> > > wrote:
> > > > >> >
> > > > >> > > Can we have updated RBs for all the three sub tasks before we
> > > > commit?
> > > > >> > This
> > > > >> > > would help us to review even after we commit.
> > > > >> > >
> > > > >> > > On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini <
> > > > >> criccomini@apache.org>
> > > > >> > > wrote:
> > > > >> > >
> > > > >> > > > Hey all,
> > > > >> > > >
> > > > >> > > > Yi, Navina, and Milinda have been working on SAMZA-390
> > > sub-tickets
> > > > >> > > related
> > > > >> > > > to SQL operators. We're getting to the point where the
> amount
> > of
> > > > work
> > > > >> > > > floating around is quite large, and some tickets build off
> of
> > > > others.
> > > > >> > > >
> > > > >> > > > I'm proposing that we commit this work into a samza-sql
> > > submodule
> > > > on
> > > > >> > > > master, and treat this module as experimental. I want to
> avoid
> > > > >> > branches,
> > > > >> > > > and I also want to avoid revision control over JIRA. This
> > means
> > > > that
> > > > >> > > there
> > > > >> > > > will probably be a fair amount of commits/JIRAs on this
> module
> > > as
> > > > we
> > > > >> > > > iterate, but I think that's OK.
> > > > >> > > >
> > > > >> > > > Does this sound good to everyone?
> > > > >> > > >
> > > > >> > > > Cheers,
> > > > >> > > > Chris
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > > >>
> > > > >>
> > > > >> --
> > > > >> Milinda Pathirage
> > > > >>
> > > > >> PhD Student | Research Assistant
> > > > >> School of Informatics and Computing | Data to Insight Center
> > > > >> Indiana University
> > > > >>
> > > > >> twitter: milindalakmal
> > > > >> skype: milinda.pathirage
> > > > >> blog: http://milinda.pathirage.org
> > > > >>
> > > >
> > >
> >
>
Re: [DISCUSS] SQL workflow
Posted by Chris Riccomini <cr...@apache.org>.
Hey all,
I've reverted SAMZA-482 and SAMZA-484 on master, and moved them into a new
branch called 'samza-sql':
https://git-wip-us.apache.org/repos/asf?p=samza.git;a=tree;h=refs/heads/samza-sql;hb=refs/heads/samza-sql
Cheers,
Chris
On Thu, Feb 12, 2015 at 1:03 PM, Chris Riccomini <cr...@apache.org>
wrote:
> Hey all,
>
> Cool. I'll revert samza-sql commits, and cut a branch, then re-apply
> SAMZA-482 and SAMZA-484.
>
> Cheers,
> Chris
>
> On Thu, Feb 12, 2015 at 12:06 PM, Milinda Pathirage <mpathira@umail.iu.edu
> > wrote:
>
>> Thanks Julian.
>>
>> @Chris, I'll update the patch with Calcite version change.
>>
>> On Thu, Feb 12, 2015 at 1:56 PM, Julian Hyde <ju...@gmail.com>
>> wrote:
>>
>> > +1
>> >
>> > I've pushed a snapshot, versioned "1.1-chi-incubating-SNAPSHOT", and
>> based
>> > on
>> >
>> https://github.com/julianhyde/incubator-calcite/commit/b8a91c31859b6901976de8dcad43afd4877cad36
>> > .
>> >
>> > See
>> >
>> https://repository.apache.org/content/repositories/snapshots/org/apache/calcite/calcite/
>> >
>> >
>> > > On Feb 12, 2015, at 9:26 AM, Jakob Homan <jg...@gmail.com> wrote:
>> > >
>> > > +1, assuming we're keeping RTC on the branch (, Jakob said
>> pedantically).
>> > >
>> > > On 12 February 2015 at 08:25, Milinda Pathirage <
>> mpathira@umail.iu.edu>
>> > wrote:
>> > >> I am +1 for this approach.
>> > >>
>> > >> Milinda
>> > >>
>> > >> On Thu, Feb 12, 2015 at 11:22 AM, Chris Riccomini <
>> > criccomini@apache.org>
>> > >> wrote:
>> > >>
>> > >>> Hey Julian,
>> > >>>
>> > >>>> The rule of thumb I’ve learned integrating Calcite into Hive is
>> that
>> > only
>> > >>> a branch should depend on snapshot versions of another component
>> > >>>
>> > >>> This is very practical. I hadn't considered the Calcite dependency
>> > issue
>> > >>> when I pushed for merging into master rather than a branch.
>> > >>>
>> > >>> Based on this, it seems like the best thing to do is:
>> > >>>
>> > >>> 1) Work off of Calcite SNAPSHOT Maven publication.
>> > >>> 2) Create a samza-sql branch.
>> > >>> 3) Migrate all existing samza-sql commits from master into the
>> > samza-sql
>> > >>> branch.
>> > >>>
>> > >>> Sorry for the churn on this all. I hadn't considered the binary
>> > dependency
>> > >>> issue. Is everyone OK with this?
>> > >>>
>> > >>> Cheers,
>> > >>> Chris
>> > >>>
>> > >>> On Wed, Feb 11, 2015 at 10:51 AM, Julian Hyde <
>> julian@hydromatic.net>
>> > >>> wrote:
>> > >>>
>> > >>>> This seems more like a branch than a classifier. Calcite is
>> > developing in
>> > >>>> a branch, and would produce snapshots from that branch. The rule of
>> > thumb
>> > >>>> I’ve learned integrating Calcite into Hive is that only a branch
>> > should
>> > >>>> depend on snapshot versions of another component. (Hive broke this
>> > rule
>> > >>> and
>> > >>>> got into a sticky situation where their trunk depended on a Calcite
>> > >>>> snapshot and they couldn’t release because Calcite was stuck in
>> > >>> pre-release
>> > >>>> vote limbo.)
>> > >>>>
>> > >>>> I’m going to try to produce maven artifacts with {groupId:
>> > >>>> “org.apache.calcite”, version: "1.1.0-stream-incubating-SNAPSHOT”,
>> > >>>> artifactId: “calcite-core”, “calcite-avatica”, etc.} and push them
>> to
>> > >>>> Apache nexus. I do recommend that you use it on a branch.
>> > >>>>
>> > >>>> However, I don’t think there would be a problem getting the
>> > functionality
>> > >>>> into calcite’s master branch in time for Calcite’s next release, in
>> > >>> about 4
>> > >>>> weeks. (We commit to releasing approximately monthly.) The
>> > functionality
>> > >>>> and APIs would be marked “experimental, subject to change” but at
>> > least
>> > >>> you
>> > >>>> would be able to merge your work to master at that point. Doubtless
>> > you
>> > >>>> would need new features & bug fixes in Calcite after that, and so
>> work
>> > >>> that
>> > >>>> depended on that would have to stay in a branch until Calcite was
>> > >>> released.
>> > >>>>
>> > >>>> Julian
>> > >>>>
>> > >>>> On Feb 11, 2015, at 10:34 AM, Felix GV
>> <fvillegas@linkedin.com.INVALID
>> > >
>> > >>>> wrote:
>> > >>>>
>> > >>>>> I guess one concern with Calcite SNAPSHOT releases is that they
>> imply
>> > >>>> they'll be included in the next release, which may not be the case
>> if
>> > >>>> they're still considered very experimental.
>> > >>>>>
>> > >>>>> Alternatively, perhaps Julian can create a new version name with a
>> > >>>> classifier for streaming.
>> > >>>>>
>> > >>>>>
>> > >>>>
>> > >>>
>> >
>> http://maven.apache.org/plugins/maven-deploy-plugin/examples/deploying-with-classifiers.html
>> > >>>>>
>> > >>>>> Then he could release SNAPSHOT artifacts for both the main/master
>> > code
>> > >>>> base as well as the streaming branch, no matter how experimental it
>> > is...
>> > >>>>>
>> > >>>>> --
>> > >>>>>
>> > >>>>> Felix GV
>> > >>>>> Data Infrastructure Engineer
>> > >>>>> Distributed Data Systems
>> > >>>>> LinkedIn
>> > >>>>>
>> > >>>>> fgv@linkedin.com
>> > >>>>> linkedin.com/in/felixgv
>> > >>>>>
>> > >>>>> ________________________________________
>> > >>>>> From: Chris Riccomini [criccomini@apache.org]
>> > >>>>> Sent: Wednesday, February 11, 2015 10:23 AM
>> > >>>>> To: Chris Riccomini
>> > >>>>> Cc: dev@samza.apache.org
>> > >>>>> Subject: Re: [DISCUSS] SQL workflow
>> > >>>>>
>> > >>>>> Hey Julian,
>> > >>>>>
>> > >>>>> It looks like Calcite is at least setup to publish to Maven:
>> > >>>>>
>> > >>>>> https://repository.apache.org/#nexus-search;quick~calcite
>> > >>>>>
>> > >>>>> Julian, can you publish SNAPSHOT releases with your streaming
>> changes
>> > >>> to
>> > >>>> it?
>> > >>>>>
>> > >>>>> Thanks!
>> > >>>>> Chris
>> > >>>>>
>> > >>>>> On Wed, Feb 11, 2015 at 10:08 AM, Chris Riccomini <
>> > >>> criccomini@apache.org
>> > >>>>>
>> > >>>>> wrote:
>> > >>>>>
>> > >>>>>> Hey Milinda,
>> > >>>>>>
>> > >>>>>> I've just committed SAMZA-484, so you should be able to re-base
>> and
>> > >>> get
>> > >>>>>> all the latest code.
>> > >>>>>>
>> > >>>>>>> But we need to add Calcite as a dependency and Calcite streaming
>> > >>>>>> support is only in Julian's branch
>> > >>>>>>
>> > >>>>>> We'll have to have Julian publish a release to Apache. This can
>> be a
>> > >>>>>> SNAPSHOT to Apache's SNAPSHOT repo, or a real Calcite release. We
>> > >>> can't
>> > >>>>>> check in binaries to the repo (licensing, release complexity, and
>> > just
>> > >>>>>> generally a bad idea). We'll have to coordinate with him.
>> > >>>>>>
>> > >>>>>> Cheers,
>> > >>>>>> Chris
>> > >>>>>>
>> > >>>>>> On Wed, Feb 11, 2015 at 9:43 AM, Milinda Pathirage <
>> > >>>> mpathira@umail.iu.edu>
>> > >>>>>> wrote:
>> > >>>>>>
>> > >>>>>>> No, SAMZA-483 doesn't depend on SAMZA-484. But we need to add
>> > Calcite
>> > >>>> as a
>> > >>>>>>> dependency and Calcite streaming support is only in Julian's
>> > branch (
>> > >>>>>>> https://github.com/julianhyde/incubator-calcite/tree/chi). What
>> > can
>> > >>>> we do
>> > >>>>>>> about that?
>> > >>>>>>>
>> > >>>>>>> Milinda
>> > >>>>>>>
>> > >>>>>>> On Wed, Feb 11, 2015 at 12:30 PM, Chris Riccomini <
>> > >>>> criccomini@apache.org>
>> > >>>>>>> wrote:
>> > >>>>>>>
>> > >>>>>>>> Hey Milinda,
>> > >>>>>>>>
>> > >>>>>>>> That'd be great. Do you have any dependency on SAMZA-484? I
>> > haven't
>> > >>>> had
>> > >>>>>>> a
>> > >>>>>>>> chance to commit that one yet.
>> > >>>>>>>>
>> > >>>>>>>> Cheers,
>> > >>>>>>>> Chris
>> > >>>>>>>>
>> > >>>>>>>> On Wed, Feb 11, 2015 at 9:23 AM, Milinda Pathirage <
>> > >>>>>>> mpathira@umail.iu.edu>
>> > >>>>>>>> wrote:
>> > >>>>>>>>
>> > >>>>>>>>> /need/want
>> > >>>>>>>>>
>> > >>>>>>>>> Milinda
>> > >>>>>>>>>
>> > >>>>>>>>> On Wed, Feb 11, 2015 at 12:22 PM, Milinda Pathirage <
>> > >>>>>>>> mpathira@umail.iu.edu
>> > >>>>>>>>>>
>> > >>>>>>>>> wrote:
>> > >>>>>>>>>
>> > >>>>>>>>>> Hi Chris,
>> > >>>>>>>>>>
>> > >>>>>>>>>> Do you need me to create the SAMZA-483 patch against latest
>> > master
>> > >>>>>>> with
>> > >>>>>>>>>> SAMZA-482 patch? I think that will make it easier to review
>> the
>> > >>>>>>> patch?
>> > >>>>>>>>>>
>> > >>>>>>>>>> Thanks
>> > >>>>>>>>>> Milinda
>> > >>>>>>>>>>
>> > >>>>>>>>>> On Mon, Feb 9, 2015 at 11:39 AM, Chris Riccomini <
>> > >>>>>>>> criccomini@apache.org>
>> > >>>>>>>>>> wrote:
>> > >>>>>>>>>>
>> > >>>>>>>>>>> Cool. Looks like we've got consensus. I'll move forward with
>> > RTC
>> > >>> on
>> > >>>>>>>> some
>> > >>>>>>>>>>> of
>> > >>>>>>>>>>> the early SAMZA-390 tickets. (SAMZA-482, SAMZA-483,
>> SAMZA-484)
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> On Mon, Feb 9, 2015 at 8:35 AM, Yan Fang <
>> yanfang724@gmail.com
>> > >
>> > >>>>>>>> wrote:
>> > >>>>>>>>>>>
>> > >>>>>>>>>>>> +1 on this.
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> Fang, Yan
>> > >>>>>>>>>>>> yanfang724@gmail.com
>> > >>>>>>>>>>>> +1 (206) 849-4108
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> On Fri, Feb 6, 2015 at 4:38 PM, Chris Riccomini <
>> > >>>>>>>>> criccomini@apache.org>
>> > >>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>>> Hey all,
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> Are we +1 on this? I think Jakob was the only one who was
>> > >>>>>>> curious
>> > >>>>>>>>>>> about
>> > >>>>>>>>>>>> it.
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> Cheers,
>> > >>>>>>>>>>>>> Chris
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> On Thu, Feb 5, 2015 at 1:22 PM, Yi Pan <
>> nickpan47@gmail.com>
>> > >>>>>>>> wrote:
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> Hi, Jakob,
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>> Eh? Not sure what this means...
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> I mean SAMZA-484 depends on SAMZA-482, and neither are
>> > >>>>>>>>> committed.
>> > >>>>>>>>>>> So
>> > >>>>>>>>>>>>>> Navina
>> > >>>>>>>>>>>>>>> is having to post Yi's patch, as well as her own, on the
>> > >>>>>>> JIRA.
>> > >>>>>>>>> It
>> > >>>>>>>>>>>> makes
>> > >>>>>>>>>>>>>> it
>> > >>>>>>>>>>>>>>> really hard to do code reviews because you can't tell
>> > >>>>>>> whether
>> > >>>>>>>> Yi
>> > >>>>>>>>>>> made
>> > >>>>>>>>>>>>> the
>> > >>>>>>>>>>>>>>> changes or Navina did.
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> Just to add to the point. It is also difficult to always
>> see
>> > >>>>>>> a
>> > >>>>>>>>> long
>> > >>>>>>>>>>>> list
>> > >>>>>>>>>>>>> of
>> > >>>>>>>>>>>>>> changed files if the RB request is always based on the
>> > >>>>>>> master.
>> > >>>>>>>> It
>> > >>>>>>>>> is
>> > >>>>>>>>>>>>>> possible to have RB request based on another RB request
>> (I
>> > >>>>>>> have
>> > >>>>>>>>>>> tried
>> > >>>>>>>>>>>> it
>> > >>>>>>>>>>>>>> before). But what happens if the base RB request is
>> > >>>>>>>>>>>> cancelled/discarded?
>> > >>>>>>>>>>>>> RB
>> > >>>>>>>>>>>>>> is not designed to track the revision changes in a
>> > dependency
>> > >>>>>>>>> chain.
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> Cheers,
>> > >>>>>>>>>>>>>>> Chris
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> On Thu, Feb 5, 2015 at 12:16 PM, Jakob Homan <
>> > >>>>>>>> jghoman@gmail.com
>> > >>>>>>>>>>
>> > >>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> I want to avoid branches,
>> > >>>>>>>>>>>>>>>> Just curious, any reason for this?
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> and I also want to avoid revision control over JIRA
>> > >>>>>>>>>>>>>>>> Eh? Not sure what this means...
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>> Thanks,
>> > >>>>>>>>>>>>>>>> jg
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>> On 4 February 2015 at 17:11, Chris Riccomini <
>> > >>>>>>>>>>>> criccomini@apache.org>
>> > >>>>>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>> Hey all,
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> @Jakob, yeah I was thinking we'll follow our normal
>> > >>>>>>> flow.
>> > >>>>>>>>>>> RTC. I
>> > >>>>>>>>>>>>> just
>> > >>>>>>>>>>>>>>>>> wanted to set expectation that the code committed
>> > >>>>>>> might be
>> > >>>>>>>>>>> not up
>> > >>>>>>>>>>>>> to
>> > >>>>>>>>>>>>>>> our
>> > >>>>>>>>>>>>>>>>> normal quality initially (missing docs, no tests,
>> etc).
>> > >>>>>>>>> Until
>> > >>>>>>>>>>> the
>> > >>>>>>>>>>>>>>> quality
>> > >>>>>>>>>>>>>>>>> is raised, we should think of this module as
>> > >>>>>>> experimental.
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> @Milinda, awesome! Thanks. :)
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> Cheers,
>> > >>>>>>>>>>>>>>>>> Chris
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 11:57 AM, Milinda Pathirage <
>> > >>>>>>>>>>>>>>>> mpathira@umail.iu.edu>
>> > >>>>>>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> Hi Chris,
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> Hope we no longer need the SQL API. I'll create a RB
>> > >>>>>>> for
>> > >>>>>>>>>>> Calcite
>> > >>>>>>>>>>>>>>>>>> integration.
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> Thanks
>> > >>>>>>>>>>>>>>>>>> Milinda
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 1:31 PM, Chris Riccomini <
>> > >>>>>>>>>>>>>>> criccomini@apache.org>
>> > >>>>>>>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> I think so. There was some RB downtime, but it just
>> > >>>>>>> got
>> > >>>>>>>>>>> fixed.
>> > >>>>>>>>>>>>> Yi,
>> > >>>>>>>>>>>>>>>>>> Navina,
>> > >>>>>>>>>>>>>>>>>>> Milinda, can you make sure your JIRAs have up to
>> > >>>>>>> date
>> > >>>>>>>>> RBs?
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 10:24 AM, sriram <
>> > >>>>>>>>>>> sriram.sub@gmail.com
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>> Can we have updated RBs for all the three sub
>> > >>>>>>> tasks
>> > >>>>>>>>>>> before
>> > >>>>>>>>>>>> we
>> > >>>>>>>>>>>>>>>> commit?
>> > >>>>>>>>>>>>>>>>>>> This
>> > >>>>>>>>>>>>>>>>>>>> would help us to review even after we commit.
>> > >>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini <
>> > >>>>>>>>>>>>>>>>>> criccomini@apache.org>
>> > >>>>>>>>>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>> Hey all,
>> > >>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>> Yi, Navina, and Milinda have been working on
>> > >>>>>>>>> SAMZA-390
>> > >>>>>>>>>>>>>>> sub-tickets
>> > >>>>>>>>>>>>>>>>>>>> related
>> > >>>>>>>>>>>>>>>>>>>>> to SQL operators. We're getting to the point
>> > >>>>>>> where
>> > >>>>>>>>> the
>> > >>>>>>>>>>>>> amount
>> > >>>>>>>>>>>>>> of
>> > >>>>>>>>>>>>>>>> work
>> > >>>>>>>>>>>>>>>>>>>>> floating around is quite large, and some tickets
>> > >>>>>>>>> build
>> > >>>>>>>>>>> off
>> > >>>>>>>>>>>>> of
>> > >>>>>>>>>>>>>>>> others.
>> > >>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>> I'm proposing that we commit this work into a
>> > >>>>>>>>> samza-sql
>> > >>>>>>>>>>>>>>> submodule
>> > >>>>>>>>>>>>>>>> on
>> > >>>>>>>>>>>>>>>>>>>>> master, and treat this module as experimental. I
>> > >>>>>>>> want
>> > >>>>>>>>>>> to
>> > >>>>>>>>>>>>> avoid
>> > >>>>>>>>>>>>>>>>>>> branches,
>> > >>>>>>>>>>>>>>>>>>>>> and I also want to avoid revision control over
>> > >>>>>>>> JIRA.
>> > >>>>>>>>>>> This
>> > >>>>>>>>>>>>>> means
>> > >>>>>>>>>>>>>>>> that
>> > >>>>>>>>>>>>>>>>>>>> there
>> > >>>>>>>>>>>>>>>>>>>>> will probably be a fair amount of commits/JIRAs
>> > >>>>>>> on
>> > >>>>>>>>> this
>> > >>>>>>>>>>>>> module
>> > >>>>>>>>>>>>>>> as
>> > >>>>>>>>>>>>>>>> we
>> > >>>>>>>>>>>>>>>>>>>>> iterate, but I think that's OK.
>> > >>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>> Does this sound good to everyone?
>> > >>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>> Cheers,
>> > >>>>>>>>>>>>>>>>>>>>> Chris
>> > >>>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> --
>> > >>>>>>>>>>>>>>>>>> Milinda Pathirage
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> PhD Student | Research Assistant
>> > >>>>>>>>>>>>>>>>>> School of Informatics and Computing | Data to Insight
>> > >>>>>>>>> Center
>> > >>>>>>>>>>>>>>>>>> Indiana University
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>>> twitter: milindalakmal
>> > >>>>>>>>>>>>>>>>>> skype: milinda.pathirage
>> > >>>>>>>>>>>>>>>>>> blog: http://milinda.pathirage.org
>> > >>>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>> --
>> > >>>>>>>>>> Milinda Pathirage
>> > >>>>>>>>>>
>> > >>>>>>>>>> PhD Student | Research Assistant
>> > >>>>>>>>>> School of Informatics and Computing | Data to Insight Center
>> > >>>>>>>>>> Indiana University
>> > >>>>>>>>>>
>> > >>>>>>>>>> twitter: milindalakmal
>> > >>>>>>>>>> skype: milinda.pathirage
>> > >>>>>>>>>> blog: http://milinda.pathirage.org
>> > >>>>>>>>>>
>> > >>>>>>>>>
>> > >>>>>>>>>
>> > >>>>>>>>>
>> > >>>>>>>>> --
>> > >>>>>>>>> Milinda Pathirage
>> > >>>>>>>>>
>> > >>>>>>>>> PhD Student | Research Assistant
>> > >>>>>>>>> School of Informatics and Computing | Data to Insight Center
>> > >>>>>>>>> Indiana University
>> > >>>>>>>>>
>> > >>>>>>>>> twitter: milindalakmal
>> > >>>>>>>>> skype: milinda.pathirage
>> > >>>>>>>>> blog: http://milinda.pathirage.org
>> > >>>>>>>>>
>> > >>>>>>>>
>> > >>>>>>>
>> > >>>>>>>
>> > >>>>>>>
>> > >>>>>>> --
>> > >>>>>>> Milinda Pathirage
>> > >>>>>>>
>> > >>>>>>> PhD Student | Research Assistant
>> > >>>>>>> School of Informatics and Computing | Data to Insight Center
>> > >>>>>>> Indiana University
>> > >>>>>>>
>> > >>>>>>> twitter: milindalakmal
>> > >>>>>>> skype: milinda.pathirage
>> > >>>>>>> blog: http://milinda.pathirage.org
>> > >>>>>>>
>> > >>>>>>
>> > >>>>>>
>> > >>>>
>> > >>>>
>> > >>>
>> > >>
>> > >>
>> > >>
>> > >> --
>> > >> Milinda Pathirage
>> > >>
>> > >> PhD Student | Research Assistant
>> > >> School of Informatics and Computing | Data to Insight Center
>> > >> Indiana University
>> > >>
>> > >> twitter: milindalakmal
>> > >> skype: milinda.pathirage
>> > >> blog: http://milinda.pathirage.org
>> >
>> >
>>
>>
>> --
>> Milinda Pathirage
>>
>> PhD Student | Research Assistant
>> School of Informatics and Computing | Data to Insight Center
>> Indiana University
>>
>> twitter: milindalakmal
>> skype: milinda.pathirage
>> blog: http://milinda.pathirage.org
>>
>
>
Re: [DISCUSS] SQL workflow
Posted by Chris Riccomini <cr...@apache.org>.
Hey all,
Cool. I'll revert samza-sql commits, and cut a branch, then re-apply
SAMZA-482 and SAMZA-484.
Cheers,
Chris
On Thu, Feb 12, 2015 at 12:06 PM, Milinda Pathirage <mp...@umail.iu.edu>
wrote:
> Thanks Julian.
>
> @Chris, I'll update the patch with Calcite version change.
>
> On Thu, Feb 12, 2015 at 1:56 PM, Julian Hyde <ju...@gmail.com> wrote:
>
> > +1
> >
> > I've pushed a snapshot, versioned "1.1-chi-incubating-SNAPSHOT", and
> based
> > on
> >
> https://github.com/julianhyde/incubator-calcite/commit/b8a91c31859b6901976de8dcad43afd4877cad36
> > .
> >
> > See
> >
> https://repository.apache.org/content/repositories/snapshots/org/apache/calcite/calcite/
> >
> >
> > > On Feb 12, 2015, at 9:26 AM, Jakob Homan <jg...@gmail.com> wrote:
> > >
> > > +1, assuming we're keeping RTC on the branch (, Jakob said
> pedantically).
> > >
> > > On 12 February 2015 at 08:25, Milinda Pathirage <mpathira@umail.iu.edu
> >
> > wrote:
> > >> I am +1 for this approach.
> > >>
> > >> Milinda
> > >>
> > >> On Thu, Feb 12, 2015 at 11:22 AM, Chris Riccomini <
> > criccomini@apache.org>
> > >> wrote:
> > >>
> > >>> Hey Julian,
> > >>>
> > >>>> The rule of thumb I’ve learned integrating Calcite into Hive is that
> > only
> > >>> a branch should depend on snapshot versions of another component
> > >>>
> > >>> This is very practical. I hadn't considered the Calcite dependency
> > issue
> > >>> when I pushed for merging into master rather than a branch.
> > >>>
> > >>> Based on this, it seems like the best thing to do is:
> > >>>
> > >>> 1) Work off of Calcite SNAPSHOT Maven publication.
> > >>> 2) Create a samza-sql branch.
> > >>> 3) Migrate all existing samza-sql commits from master into the
> > samza-sql
> > >>> branch.
> > >>>
> > >>> Sorry for the churn on this all. I hadn't considered the binary
> > dependency
> > >>> issue. Is everyone OK with this?
> > >>>
> > >>> Cheers,
> > >>> Chris
> > >>>
> > >>> On Wed, Feb 11, 2015 at 10:51 AM, Julian Hyde <julian@hydromatic.net
> >
> > >>> wrote:
> > >>>
> > >>>> This seems more like a branch than a classifier. Calcite is
> > developing in
> > >>>> a branch, and would produce snapshots from that branch. The rule of
> > thumb
> > >>>> I’ve learned integrating Calcite into Hive is that only a branch
> > should
> > >>>> depend on snapshot versions of another component. (Hive broke this
> > rule
> > >>> and
> > >>>> got into a sticky situation where their trunk depended on a Calcite
> > >>>> snapshot and they couldn’t release because Calcite was stuck in
> > >>> pre-release
> > >>>> vote limbo.)
> > >>>>
> > >>>> I’m going to try to produce maven artifacts with {groupId:
> > >>>> “org.apache.calcite”, version: "1.1.0-stream-incubating-SNAPSHOT”,
> > >>>> artifactId: “calcite-core”, “calcite-avatica”, etc.} and push them
> to
> > >>>> Apache nexus. I do recommend that you use it on a branch.
> > >>>>
> > >>>> However, I don’t think there would be a problem getting the
> > functionality
> > >>>> into calcite’s master branch in time for Calcite’s next release, in
> > >>> about 4
> > >>>> weeks. (We commit to releasing approximately monthly.) The
> > functionality
> > >>>> and APIs would be marked “experimental, subject to change” but at
> > least
> > >>> you
> > >>>> would be able to merge your work to master at that point. Doubtless
> > you
> > >>>> would need new features & bug fixes in Calcite after that, and so
> work
> > >>> that
> > >>>> depended on that would have to stay in a branch until Calcite was
> > >>> released.
> > >>>>
> > >>>> Julian
> > >>>>
> > >>>> On Feb 11, 2015, at 10:34 AM, Felix GV
> <fvillegas@linkedin.com.INVALID
> > >
> > >>>> wrote:
> > >>>>
> > >>>>> I guess one concern with Calcite SNAPSHOT releases is that they
> imply
> > >>>> they'll be included in the next release, which may not be the case
> if
> > >>>> they're still considered very experimental.
> > >>>>>
> > >>>>> Alternatively, perhaps Julian can create a new version name with a
> > >>>> classifier for streaming.
> > >>>>>
> > >>>>>
> > >>>>
> > >>>
> >
> http://maven.apache.org/plugins/maven-deploy-plugin/examples/deploying-with-classifiers.html
> > >>>>>
> > >>>>> Then he could release SNAPSHOT artifacts for both the main/master
> > code
> > >>>> base as well as the streaming branch, no matter how experimental it
> > is...
> > >>>>>
> > >>>>> --
> > >>>>>
> > >>>>> Felix GV
> > >>>>> Data Infrastructure Engineer
> > >>>>> Distributed Data Systems
> > >>>>> LinkedIn
> > >>>>>
> > >>>>> fgv@linkedin.com
> > >>>>> linkedin.com/in/felixgv
> > >>>>>
> > >>>>> ________________________________________
> > >>>>> From: Chris Riccomini [criccomini@apache.org]
> > >>>>> Sent: Wednesday, February 11, 2015 10:23 AM
> > >>>>> To: Chris Riccomini
> > >>>>> Cc: dev@samza.apache.org
> > >>>>> Subject: Re: [DISCUSS] SQL workflow
> > >>>>>
> > >>>>> Hey Julian,
> > >>>>>
> > >>>>> It looks like Calcite is at least setup to publish to Maven:
> > >>>>>
> > >>>>> https://repository.apache.org/#nexus-search;quick~calcite
> > >>>>>
> > >>>>> Julian, can you publish SNAPSHOT releases with your streaming
> changes
> > >>> to
> > >>>> it?
> > >>>>>
> > >>>>> Thanks!
> > >>>>> Chris
> > >>>>>
> > >>>>> On Wed, Feb 11, 2015 at 10:08 AM, Chris Riccomini <
> > >>> criccomini@apache.org
> > >>>>>
> > >>>>> wrote:
> > >>>>>
> > >>>>>> Hey Milinda,
> > >>>>>>
> > >>>>>> I've just committed SAMZA-484, so you should be able to re-base
> and
> > >>> get
> > >>>>>> all the latest code.
> > >>>>>>
> > >>>>>>> But we need to add Calcite as a dependency and Calcite streaming
> > >>>>>> support is only in Julian's branch
> > >>>>>>
> > >>>>>> We'll have to have Julian publish a release to Apache. This can
> be a
> > >>>>>> SNAPSHOT to Apache's SNAPSHOT repo, or a real Calcite release. We
> > >>> can't
> > >>>>>> check in binaries to the repo (licensing, release complexity, and
> > just
> > >>>>>> generally a bad idea). We'll have to coordinate with him.
> > >>>>>>
> > >>>>>> Cheers,
> > >>>>>> Chris
> > >>>>>>
> > >>>>>> On Wed, Feb 11, 2015 at 9:43 AM, Milinda Pathirage <
> > >>>> mpathira@umail.iu.edu>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>>> No, SAMZA-483 doesn't depend on SAMZA-484. But we need to add
> > Calcite
> > >>>> as a
> > >>>>>>> dependency and Calcite streaming support is only in Julian's
> > branch (
> > >>>>>>> https://github.com/julianhyde/incubator-calcite/tree/chi). What
> > can
> > >>>> we do
> > >>>>>>> about that?
> > >>>>>>>
> > >>>>>>> Milinda
> > >>>>>>>
> > >>>>>>> On Wed, Feb 11, 2015 at 12:30 PM, Chris Riccomini <
> > >>>> criccomini@apache.org>
> > >>>>>>> wrote:
> > >>>>>>>
> > >>>>>>>> Hey Milinda,
> > >>>>>>>>
> > >>>>>>>> That'd be great. Do you have any dependency on SAMZA-484? I
> > haven't
> > >>>> had
> > >>>>>>> a
> > >>>>>>>> chance to commit that one yet.
> > >>>>>>>>
> > >>>>>>>> Cheers,
> > >>>>>>>> Chris
> > >>>>>>>>
> > >>>>>>>> On Wed, Feb 11, 2015 at 9:23 AM, Milinda Pathirage <
> > >>>>>>> mpathira@umail.iu.edu>
> > >>>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>>> /need/want
> > >>>>>>>>>
> > >>>>>>>>> Milinda
> > >>>>>>>>>
> > >>>>>>>>> On Wed, Feb 11, 2015 at 12:22 PM, Milinda Pathirage <
> > >>>>>>>> mpathira@umail.iu.edu
> > >>>>>>>>>>
> > >>>>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> Hi Chris,
> > >>>>>>>>>>
> > >>>>>>>>>> Do you need me to create the SAMZA-483 patch against latest
> > master
> > >>>>>>> with
> > >>>>>>>>>> SAMZA-482 patch? I think that will make it easier to review
> the
> > >>>>>>> patch?
> > >>>>>>>>>>
> > >>>>>>>>>> Thanks
> > >>>>>>>>>> Milinda
> > >>>>>>>>>>
> > >>>>>>>>>> On Mon, Feb 9, 2015 at 11:39 AM, Chris Riccomini <
> > >>>>>>>> criccomini@apache.org>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>> Cool. Looks like we've got consensus. I'll move forward with
> > RTC
> > >>> on
> > >>>>>>>> some
> > >>>>>>>>>>> of
> > >>>>>>>>>>> the early SAMZA-390 tickets. (SAMZA-482, SAMZA-483,
> SAMZA-484)
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Mon, Feb 9, 2015 at 8:35 AM, Yan Fang <
> yanfang724@gmail.com
> > >
> > >>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> +1 on this.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Fang, Yan
> > >>>>>>>>>>>> yanfang724@gmail.com
> > >>>>>>>>>>>> +1 (206) 849-4108
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Fri, Feb 6, 2015 at 4:38 PM, Chris Riccomini <
> > >>>>>>>>> criccomini@apache.org>
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> Hey all,
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Are we +1 on this? I think Jakob was the only one who was
> > >>>>>>> curious
> > >>>>>>>>>>> about
> > >>>>>>>>>>>> it.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Cheers,
> > >>>>>>>>>>>>> Chris
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On Thu, Feb 5, 2015 at 1:22 PM, Yi Pan <
> nickpan47@gmail.com>
> > >>>>>>>> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Hi, Jakob,
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Eh? Not sure what this means...
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> I mean SAMZA-484 depends on SAMZA-482, and neither are
> > >>>>>>>>> committed.
> > >>>>>>>>>>> So
> > >>>>>>>>>>>>>> Navina
> > >>>>>>>>>>>>>>> is having to post Yi's patch, as well as her own, on the
> > >>>>>>> JIRA.
> > >>>>>>>>> It
> > >>>>>>>>>>>> makes
> > >>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>> really hard to do code reviews because you can't tell
> > >>>>>>> whether
> > >>>>>>>> Yi
> > >>>>>>>>>>> made
> > >>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>> changes or Navina did.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Just to add to the point. It is also difficult to always
> see
> > >>>>>>> a
> > >>>>>>>>> long
> > >>>>>>>>>>>> list
> > >>>>>>>>>>>>> of
> > >>>>>>>>>>>>>> changed files if the RB request is always based on the
> > >>>>>>> master.
> > >>>>>>>> It
> > >>>>>>>>> is
> > >>>>>>>>>>>>>> possible to have RB request based on another RB request (I
> > >>>>>>> have
> > >>>>>>>>>>> tried
> > >>>>>>>>>>>> it
> > >>>>>>>>>>>>>> before). But what happens if the base RB request is
> > >>>>>>>>>>>> cancelled/discarded?
> > >>>>>>>>>>>>> RB
> > >>>>>>>>>>>>>> is not designed to track the revision changes in a
> > dependency
> > >>>>>>>>> chain.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Cheers,
> > >>>>>>>>>>>>>>> Chris
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> On Thu, Feb 5, 2015 at 12:16 PM, Jakob Homan <
> > >>>>>>>> jghoman@gmail.com
> > >>>>>>>>>>
> > >>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> I want to avoid branches,
> > >>>>>>>>>>>>>>>> Just curious, any reason for this?
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> and I also want to avoid revision control over JIRA
> > >>>>>>>>>>>>>>>> Eh? Not sure what this means...
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>>>>>> jg
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> On 4 February 2015 at 17:11, Chris Riccomini <
> > >>>>>>>>>>>> criccomini@apache.org>
> > >>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>> Hey all,
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> @Jakob, yeah I was thinking we'll follow our normal
> > >>>>>>> flow.
> > >>>>>>>>>>> RTC. I
> > >>>>>>>>>>>>> just
> > >>>>>>>>>>>>>>>>> wanted to set expectation that the code committed
> > >>>>>>> might be
> > >>>>>>>>>>> not up
> > >>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>> our
> > >>>>>>>>>>>>>>>>> normal quality initially (missing docs, no tests, etc).
> > >>>>>>>>> Until
> > >>>>>>>>>>> the
> > >>>>>>>>>>>>>>> quality
> > >>>>>>>>>>>>>>>>> is raised, we should think of this module as
> > >>>>>>> experimental.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> @Milinda, awesome! Thanks. :)
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Cheers,
> > >>>>>>>>>>>>>>>>> Chris
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 11:57 AM, Milinda Pathirage <
> > >>>>>>>>>>>>>>>> mpathira@umail.iu.edu>
> > >>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Hi Chris,
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Hope we no longer need the SQL API. I'll create a RB
> > >>>>>>> for
> > >>>>>>>>>>> Calcite
> > >>>>>>>>>>>>>>>>>> integration.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Thanks
> > >>>>>>>>>>>>>>>>>> Milinda
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 1:31 PM, Chris Riccomini <
> > >>>>>>>>>>>>>>> criccomini@apache.org>
> > >>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> I think so. There was some RB downtime, but it just
> > >>>>>>> got
> > >>>>>>>>>>> fixed.
> > >>>>>>>>>>>>> Yi,
> > >>>>>>>>>>>>>>>>>> Navina,
> > >>>>>>>>>>>>>>>>>>> Milinda, can you make sure your JIRAs have up to
> > >>>>>>> date
> > >>>>>>>>> RBs?
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 10:24 AM, sriram <
> > >>>>>>>>>>> sriram.sub@gmail.com
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> Can we have updated RBs for all the three sub
> > >>>>>>> tasks
> > >>>>>>>>>>> before
> > >>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>> commit?
> > >>>>>>>>>>>>>>>>>>> This
> > >>>>>>>>>>>>>>>>>>>> would help us to review even after we commit.
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini <
> > >>>>>>>>>>>>>>>>>> criccomini@apache.org>
> > >>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> Hey all,
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> Yi, Navina, and Milinda have been working on
> > >>>>>>>>> SAMZA-390
> > >>>>>>>>>>>>>>> sub-tickets
> > >>>>>>>>>>>>>>>>>>>> related
> > >>>>>>>>>>>>>>>>>>>>> to SQL operators. We're getting to the point
> > >>>>>>> where
> > >>>>>>>>> the
> > >>>>>>>>>>>>> amount
> > >>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>> work
> > >>>>>>>>>>>>>>>>>>>>> floating around is quite large, and some tickets
> > >>>>>>>>> build
> > >>>>>>>>>>> off
> > >>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>> others.
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> I'm proposing that we commit this work into a
> > >>>>>>>>> samza-sql
> > >>>>>>>>>>>>>>> submodule
> > >>>>>>>>>>>>>>>> on
> > >>>>>>>>>>>>>>>>>>>>> master, and treat this module as experimental. I
> > >>>>>>>> want
> > >>>>>>>>>>> to
> > >>>>>>>>>>>>> avoid
> > >>>>>>>>>>>>>>>>>>> branches,
> > >>>>>>>>>>>>>>>>>>>>> and I also want to avoid revision control over
> > >>>>>>>> JIRA.
> > >>>>>>>>>>> This
> > >>>>>>>>>>>>>> means
> > >>>>>>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>>>>>> there
> > >>>>>>>>>>>>>>>>>>>>> will probably be a fair amount of commits/JIRAs
> > >>>>>>> on
> > >>>>>>>>> this
> > >>>>>>>>>>>>> module
> > >>>>>>>>>>>>>>> as
> > >>>>>>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>>>>>>> iterate, but I think that's OK.
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> Does this sound good to everyone?
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> Cheers,
> > >>>>>>>>>>>>>>>>>>>>> Chris
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>> Milinda Pathirage
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> PhD Student | Research Assistant
> > >>>>>>>>>>>>>>>>>> School of Informatics and Computing | Data to Insight
> > >>>>>>>>> Center
> > >>>>>>>>>>>>>>>>>> Indiana University
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> twitter: milindalakmal
> > >>>>>>>>>>>>>>>>>> skype: milinda.pathirage
> > >>>>>>>>>>>>>>>>>> blog: http://milinda.pathirage.org
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> --
> > >>>>>>>>>> Milinda Pathirage
> > >>>>>>>>>>
> > >>>>>>>>>> PhD Student | Research Assistant
> > >>>>>>>>>> School of Informatics and Computing | Data to Insight Center
> > >>>>>>>>>> Indiana University
> > >>>>>>>>>>
> > >>>>>>>>>> twitter: milindalakmal
> > >>>>>>>>>> skype: milinda.pathirage
> > >>>>>>>>>> blog: http://milinda.pathirage.org
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> --
> > >>>>>>>>> Milinda Pathirage
> > >>>>>>>>>
> > >>>>>>>>> PhD Student | Research Assistant
> > >>>>>>>>> School of Informatics and Computing | Data to Insight Center
> > >>>>>>>>> Indiana University
> > >>>>>>>>>
> > >>>>>>>>> twitter: milindalakmal
> > >>>>>>>>> skype: milinda.pathirage
> > >>>>>>>>> blog: http://milinda.pathirage.org
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> --
> > >>>>>>> Milinda Pathirage
> > >>>>>>>
> > >>>>>>> PhD Student | Research Assistant
> > >>>>>>> School of Informatics and Computing | Data to Insight Center
> > >>>>>>> Indiana University
> > >>>>>>>
> > >>>>>>> twitter: milindalakmal
> > >>>>>>> skype: milinda.pathirage
> > >>>>>>> blog: http://milinda.pathirage.org
> > >>>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>
> > >>>>
> > >>>
> > >>
> > >>
> > >>
> > >> --
> > >> Milinda Pathirage
> > >>
> > >> PhD Student | Research Assistant
> > >> School of Informatics and Computing | Data to Insight Center
> > >> Indiana University
> > >>
> > >> twitter: milindalakmal
> > >> skype: milinda.pathirage
> > >> blog: http://milinda.pathirage.org
> >
> >
>
>
> --
> Milinda Pathirage
>
> PhD Student | Research Assistant
> School of Informatics and Computing | Data to Insight Center
> Indiana University
>
> twitter: milindalakmal
> skype: milinda.pathirage
> blog: http://milinda.pathirage.org
>
Re: [DISCUSS] SQL workflow
Posted by Milinda Pathirage <mp...@umail.iu.edu>.
Thanks Julian.
@Chris, I'll update the patch with Calcite version change.
On Thu, Feb 12, 2015 at 1:56 PM, Julian Hyde <ju...@gmail.com> wrote:
> +1
>
> I've pushed a snapshot, versioned "1.1-chi-incubating-SNAPSHOT", and based
> on
> https://github.com/julianhyde/incubator-calcite/commit/b8a91c31859b6901976de8dcad43afd4877cad36
> .
>
> See
> https://repository.apache.org/content/repositories/snapshots/org/apache/calcite/calcite/
>
>
> > On Feb 12, 2015, at 9:26 AM, Jakob Homan <jg...@gmail.com> wrote:
> >
> > +1, assuming we're keeping RTC on the branch (, Jakob said pedantically).
> >
> > On 12 February 2015 at 08:25, Milinda Pathirage <mp...@umail.iu.edu>
> wrote:
> >> I am +1 for this approach.
> >>
> >> Milinda
> >>
> >> On Thu, Feb 12, 2015 at 11:22 AM, Chris Riccomini <
> criccomini@apache.org>
> >> wrote:
> >>
> >>> Hey Julian,
> >>>
> >>>> The rule of thumb I’ve learned integrating Calcite into Hive is that
> only
> >>> a branch should depend on snapshot versions of another component
> >>>
> >>> This is very practical. I hadn't considered the Calcite dependency
> issue
> >>> when I pushed for merging into master rather than a branch.
> >>>
> >>> Based on this, it seems like the best thing to do is:
> >>>
> >>> 1) Work off of Calcite SNAPSHOT Maven publication.
> >>> 2) Create a samza-sql branch.
> >>> 3) Migrate all existing samza-sql commits from master into the
> samza-sql
> >>> branch.
> >>>
> >>> Sorry for the churn on this all. I hadn't considered the binary
> dependency
> >>> issue. Is everyone OK with this?
> >>>
> >>> Cheers,
> >>> Chris
> >>>
> >>> On Wed, Feb 11, 2015 at 10:51 AM, Julian Hyde <ju...@hydromatic.net>
> >>> wrote:
> >>>
> >>>> This seems more like a branch than a classifier. Calcite is
> developing in
> >>>> a branch, and would produce snapshots from that branch. The rule of
> thumb
> >>>> I’ve learned integrating Calcite into Hive is that only a branch
> should
> >>>> depend on snapshot versions of another component. (Hive broke this
> rule
> >>> and
> >>>> got into a sticky situation where their trunk depended on a Calcite
> >>>> snapshot and they couldn’t release because Calcite was stuck in
> >>> pre-release
> >>>> vote limbo.)
> >>>>
> >>>> I’m going to try to produce maven artifacts with {groupId:
> >>>> “org.apache.calcite”, version: "1.1.0-stream-incubating-SNAPSHOT”,
> >>>> artifactId: “calcite-core”, “calcite-avatica”, etc.} and push them to
> >>>> Apache nexus. I do recommend that you use it on a branch.
> >>>>
> >>>> However, I don’t think there would be a problem getting the
> functionality
> >>>> into calcite’s master branch in time for Calcite’s next release, in
> >>> about 4
> >>>> weeks. (We commit to releasing approximately monthly.) The
> functionality
> >>>> and APIs would be marked “experimental, subject to change” but at
> least
> >>> you
> >>>> would be able to merge your work to master at that point. Doubtless
> you
> >>>> would need new features & bug fixes in Calcite after that, and so work
> >>> that
> >>>> depended on that would have to stay in a branch until Calcite was
> >>> released.
> >>>>
> >>>> Julian
> >>>>
> >>>> On Feb 11, 2015, at 10:34 AM, Felix GV <fvillegas@linkedin.com.INVALID
> >
> >>>> wrote:
> >>>>
> >>>>> I guess one concern with Calcite SNAPSHOT releases is that they imply
> >>>> they'll be included in the next release, which may not be the case if
> >>>> they're still considered very experimental.
> >>>>>
> >>>>> Alternatively, perhaps Julian can create a new version name with a
> >>>> classifier for streaming.
> >>>>>
> >>>>>
> >>>>
> >>>
> http://maven.apache.org/plugins/maven-deploy-plugin/examples/deploying-with-classifiers.html
> >>>>>
> >>>>> Then he could release SNAPSHOT artifacts for both the main/master
> code
> >>>> base as well as the streaming branch, no matter how experimental it
> is...
> >>>>>
> >>>>> --
> >>>>>
> >>>>> Felix GV
> >>>>> Data Infrastructure Engineer
> >>>>> Distributed Data Systems
> >>>>> LinkedIn
> >>>>>
> >>>>> fgv@linkedin.com
> >>>>> linkedin.com/in/felixgv
> >>>>>
> >>>>> ________________________________________
> >>>>> From: Chris Riccomini [criccomini@apache.org]
> >>>>> Sent: Wednesday, February 11, 2015 10:23 AM
> >>>>> To: Chris Riccomini
> >>>>> Cc: dev@samza.apache.org
> >>>>> Subject: Re: [DISCUSS] SQL workflow
> >>>>>
> >>>>> Hey Julian,
> >>>>>
> >>>>> It looks like Calcite is at least setup to publish to Maven:
> >>>>>
> >>>>> https://repository.apache.org/#nexus-search;quick~calcite
> >>>>>
> >>>>> Julian, can you publish SNAPSHOT releases with your streaming changes
> >>> to
> >>>> it?
> >>>>>
> >>>>> Thanks!
> >>>>> Chris
> >>>>>
> >>>>> On Wed, Feb 11, 2015 at 10:08 AM, Chris Riccomini <
> >>> criccomini@apache.org
> >>>>>
> >>>>> wrote:
> >>>>>
> >>>>>> Hey Milinda,
> >>>>>>
> >>>>>> I've just committed SAMZA-484, so you should be able to re-base and
> >>> get
> >>>>>> all the latest code.
> >>>>>>
> >>>>>>> But we need to add Calcite as a dependency and Calcite streaming
> >>>>>> support is only in Julian's branch
> >>>>>>
> >>>>>> We'll have to have Julian publish a release to Apache. This can be a
> >>>>>> SNAPSHOT to Apache's SNAPSHOT repo, or a real Calcite release. We
> >>> can't
> >>>>>> check in binaries to the repo (licensing, release complexity, and
> just
> >>>>>> generally a bad idea). We'll have to coordinate with him.
> >>>>>>
> >>>>>> Cheers,
> >>>>>> Chris
> >>>>>>
> >>>>>> On Wed, Feb 11, 2015 at 9:43 AM, Milinda Pathirage <
> >>>> mpathira@umail.iu.edu>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> No, SAMZA-483 doesn't depend on SAMZA-484. But we need to add
> Calcite
> >>>> as a
> >>>>>>> dependency and Calcite streaming support is only in Julian's
> branch (
> >>>>>>> https://github.com/julianhyde/incubator-calcite/tree/chi). What
> can
> >>>> we do
> >>>>>>> about that?
> >>>>>>>
> >>>>>>> Milinda
> >>>>>>>
> >>>>>>> On Wed, Feb 11, 2015 at 12:30 PM, Chris Riccomini <
> >>>> criccomini@apache.org>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Hey Milinda,
> >>>>>>>>
> >>>>>>>> That'd be great. Do you have any dependency on SAMZA-484? I
> haven't
> >>>> had
> >>>>>>> a
> >>>>>>>> chance to commit that one yet.
> >>>>>>>>
> >>>>>>>> Cheers,
> >>>>>>>> Chris
> >>>>>>>>
> >>>>>>>> On Wed, Feb 11, 2015 at 9:23 AM, Milinda Pathirage <
> >>>>>>> mpathira@umail.iu.edu>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> /need/want
> >>>>>>>>>
> >>>>>>>>> Milinda
> >>>>>>>>>
> >>>>>>>>> On Wed, Feb 11, 2015 at 12:22 PM, Milinda Pathirage <
> >>>>>>>> mpathira@umail.iu.edu
> >>>>>>>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Hi Chris,
> >>>>>>>>>>
> >>>>>>>>>> Do you need me to create the SAMZA-483 patch against latest
> master
> >>>>>>> with
> >>>>>>>>>> SAMZA-482 patch? I think that will make it easier to review the
> >>>>>>> patch?
> >>>>>>>>>>
> >>>>>>>>>> Thanks
> >>>>>>>>>> Milinda
> >>>>>>>>>>
> >>>>>>>>>> On Mon, Feb 9, 2015 at 11:39 AM, Chris Riccomini <
> >>>>>>>> criccomini@apache.org>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Cool. Looks like we've got consensus. I'll move forward with
> RTC
> >>> on
> >>>>>>>> some
> >>>>>>>>>>> of
> >>>>>>>>>>> the early SAMZA-390 tickets. (SAMZA-482, SAMZA-483, SAMZA-484)
> >>>>>>>>>>>
> >>>>>>>>>>> On Mon, Feb 9, 2015 at 8:35 AM, Yan Fang <yanfang724@gmail.com
> >
> >>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> +1 on this.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Fang, Yan
> >>>>>>>>>>>> yanfang724@gmail.com
> >>>>>>>>>>>> +1 (206) 849-4108
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Fri, Feb 6, 2015 at 4:38 PM, Chris Riccomini <
> >>>>>>>>> criccomini@apache.org>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Hey all,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Are we +1 on this? I think Jakob was the only one who was
> >>>>>>> curious
> >>>>>>>>>>> about
> >>>>>>>>>>>> it.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Cheers,
> >>>>>>>>>>>>> Chris
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Thu, Feb 5, 2015 at 1:22 PM, Yi Pan <ni...@gmail.com>
> >>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hi, Jakob,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Eh? Not sure what this means...
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I mean SAMZA-484 depends on SAMZA-482, and neither are
> >>>>>>>>> committed.
> >>>>>>>>>>> So
> >>>>>>>>>>>>>> Navina
> >>>>>>>>>>>>>>> is having to post Yi's patch, as well as her own, on the
> >>>>>>> JIRA.
> >>>>>>>>> It
> >>>>>>>>>>>> makes
> >>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>> really hard to do code reviews because you can't tell
> >>>>>>> whether
> >>>>>>>> Yi
> >>>>>>>>>>> made
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>> changes or Navina did.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Just to add to the point. It is also difficult to always see
> >>>>>>> a
> >>>>>>>>> long
> >>>>>>>>>>>> list
> >>>>>>>>>>>>> of
> >>>>>>>>>>>>>> changed files if the RB request is always based on the
> >>>>>>> master.
> >>>>>>>> It
> >>>>>>>>> is
> >>>>>>>>>>>>>> possible to have RB request based on another RB request (I
> >>>>>>> have
> >>>>>>>>>>> tried
> >>>>>>>>>>>> it
> >>>>>>>>>>>>>> before). But what happens if the base RB request is
> >>>>>>>>>>>> cancelled/discarded?
> >>>>>>>>>>>>> RB
> >>>>>>>>>>>>>> is not designed to track the revision changes in a
> dependency
> >>>>>>>>> chain.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Cheers,
> >>>>>>>>>>>>>>> Chris
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Thu, Feb 5, 2015 at 12:16 PM, Jakob Homan <
> >>>>>>>> jghoman@gmail.com
> >>>>>>>>>>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I want to avoid branches,
> >>>>>>>>>>>>>>>> Just curious, any reason for this?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> and I also want to avoid revision control over JIRA
> >>>>>>>>>>>>>>>> Eh? Not sure what this means...
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>>>> jg
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On 4 February 2015 at 17:11, Chris Riccomini <
> >>>>>>>>>>>> criccomini@apache.org>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>> Hey all,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> @Jakob, yeah I was thinking we'll follow our normal
> >>>>>>> flow.
> >>>>>>>>>>> RTC. I
> >>>>>>>>>>>>> just
> >>>>>>>>>>>>>>>>> wanted to set expectation that the code committed
> >>>>>>> might be
> >>>>>>>>>>> not up
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>>> our
> >>>>>>>>>>>>>>>>> normal quality initially (missing docs, no tests, etc).
> >>>>>>>>> Until
> >>>>>>>>>>> the
> >>>>>>>>>>>>>>> quality
> >>>>>>>>>>>>>>>>> is raised, we should think of this module as
> >>>>>>> experimental.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> @Milinda, awesome! Thanks. :)
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Cheers,
> >>>>>>>>>>>>>>>>> Chris
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 11:57 AM, Milinda Pathirage <
> >>>>>>>>>>>>>>>> mpathira@umail.iu.edu>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Hi Chris,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Hope we no longer need the SQL API. I'll create a RB
> >>>>>>> for
> >>>>>>>>>>> Calcite
> >>>>>>>>>>>>>>>>>> integration.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Thanks
> >>>>>>>>>>>>>>>>>> Milinda
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 1:31 PM, Chris Riccomini <
> >>>>>>>>>>>>>>> criccomini@apache.org>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I think so. There was some RB downtime, but it just
> >>>>>>> got
> >>>>>>>>>>> fixed.
> >>>>>>>>>>>>> Yi,
> >>>>>>>>>>>>>>>>>> Navina,
> >>>>>>>>>>>>>>>>>>> Milinda, can you make sure your JIRAs have up to
> >>>>>>> date
> >>>>>>>>> RBs?
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 10:24 AM, sriram <
> >>>>>>>>>>> sriram.sub@gmail.com
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Can we have updated RBs for all the three sub
> >>>>>>> tasks
> >>>>>>>>>>> before
> >>>>>>>>>>>> we
> >>>>>>>>>>>>>>>> commit?
> >>>>>>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>>>>>> would help us to review even after we commit.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini <
> >>>>>>>>>>>>>>>>>> criccomini@apache.org>
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Hey all,
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Yi, Navina, and Milinda have been working on
> >>>>>>>>> SAMZA-390
> >>>>>>>>>>>>>>> sub-tickets
> >>>>>>>>>>>>>>>>>>>> related
> >>>>>>>>>>>>>>>>>>>>> to SQL operators. We're getting to the point
> >>>>>>> where
> >>>>>>>>> the
> >>>>>>>>>>>>> amount
> >>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>>> floating around is quite large, and some tickets
> >>>>>>>>> build
> >>>>>>>>>>> off
> >>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>> others.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I'm proposing that we commit this work into a
> >>>>>>>>> samza-sql
> >>>>>>>>>>>>>>> submodule
> >>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>>> master, and treat this module as experimental. I
> >>>>>>>> want
> >>>>>>>>>>> to
> >>>>>>>>>>>>> avoid
> >>>>>>>>>>>>>>>>>>> branches,
> >>>>>>>>>>>>>>>>>>>>> and I also want to avoid revision control over
> >>>>>>>> JIRA.
> >>>>>>>>>>> This
> >>>>>>>>>>>>>> means
> >>>>>>>>>>>>>>>> that
> >>>>>>>>>>>>>>>>>>>> there
> >>>>>>>>>>>>>>>>>>>>> will probably be a fair amount of commits/JIRAs
> >>>>>>> on
> >>>>>>>>> this
> >>>>>>>>>>>>> module
> >>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>> iterate, but I think that's OK.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Does this sound good to everyone?
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Cheers,
> >>>>>>>>>>>>>>>>>>>>> Chris
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>> Milinda Pathirage
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> PhD Student | Research Assistant
> >>>>>>>>>>>>>>>>>> School of Informatics and Computing | Data to Insight
> >>>>>>>>> Center
> >>>>>>>>>>>>>>>>>> Indiana University
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> twitter: milindalakmal
> >>>>>>>>>>>>>>>>>> skype: milinda.pathirage
> >>>>>>>>>>>>>>>>>> blog: http://milinda.pathirage.org
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> Milinda Pathirage
> >>>>>>>>>>
> >>>>>>>>>> PhD Student | Research Assistant
> >>>>>>>>>> School of Informatics and Computing | Data to Insight Center
> >>>>>>>>>> Indiana University
> >>>>>>>>>>
> >>>>>>>>>> twitter: milindalakmal
> >>>>>>>>>> skype: milinda.pathirage
> >>>>>>>>>> blog: http://milinda.pathirage.org
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> Milinda Pathirage
> >>>>>>>>>
> >>>>>>>>> PhD Student | Research Assistant
> >>>>>>>>> School of Informatics and Computing | Data to Insight Center
> >>>>>>>>> Indiana University
> >>>>>>>>>
> >>>>>>>>> twitter: milindalakmal
> >>>>>>>>> skype: milinda.pathirage
> >>>>>>>>> blog: http://milinda.pathirage.org
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> --
> >>>>>>> Milinda Pathirage
> >>>>>>>
> >>>>>>> PhD Student | Research Assistant
> >>>>>>> School of Informatics and Computing | Data to Insight Center
> >>>>>>> Indiana University
> >>>>>>>
> >>>>>>> twitter: milindalakmal
> >>>>>>> skype: milinda.pathirage
> >>>>>>> blog: http://milinda.pathirage.org
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>
> >>>>
> >>>
> >>
> >>
> >>
> >> --
> >> Milinda Pathirage
> >>
> >> PhD Student | Research Assistant
> >> School of Informatics and Computing | Data to Insight Center
> >> Indiana University
> >>
> >> twitter: milindalakmal
> >> skype: milinda.pathirage
> >> blog: http://milinda.pathirage.org
>
>
--
Milinda Pathirage
PhD Student | Research Assistant
School of Informatics and Computing | Data to Insight Center
Indiana University
twitter: milindalakmal
skype: milinda.pathirage
blog: http://milinda.pathirage.org
Re: [DISCUSS] SQL workflow
Posted by Julian Hyde <ju...@gmail.com>.
+1
I've pushed a snapshot, versioned "1.1-chi-incubating-SNAPSHOT", and based on https://github.com/julianhyde/incubator-calcite/commit/b8a91c31859b6901976de8dcad43afd4877cad36.
See https://repository.apache.org/content/repositories/snapshots/org/apache/calcite/calcite/
> On Feb 12, 2015, at 9:26 AM, Jakob Homan <jg...@gmail.com> wrote:
>
> +1, assuming we're keeping RTC on the branch (, Jakob said pedantically).
>
> On 12 February 2015 at 08:25, Milinda Pathirage <mp...@umail.iu.edu> wrote:
>> I am +1 for this approach.
>>
>> Milinda
>>
>> On Thu, Feb 12, 2015 at 11:22 AM, Chris Riccomini <cr...@apache.org>
>> wrote:
>>
>>> Hey Julian,
>>>
>>>> The rule of thumb I’ve learned integrating Calcite into Hive is that only
>>> a branch should depend on snapshot versions of another component
>>>
>>> This is very practical. I hadn't considered the Calcite dependency issue
>>> when I pushed for merging into master rather than a branch.
>>>
>>> Based on this, it seems like the best thing to do is:
>>>
>>> 1) Work off of Calcite SNAPSHOT Maven publication.
>>> 2) Create a samza-sql branch.
>>> 3) Migrate all existing samza-sql commits from master into the samza-sql
>>> branch.
>>>
>>> Sorry for the churn on this all. I hadn't considered the binary dependency
>>> issue. Is everyone OK with this?
>>>
>>> Cheers,
>>> Chris
>>>
>>> On Wed, Feb 11, 2015 at 10:51 AM, Julian Hyde <ju...@hydromatic.net>
>>> wrote:
>>>
>>>> This seems more like a branch than a classifier. Calcite is developing in
>>>> a branch, and would produce snapshots from that branch. The rule of thumb
>>>> I’ve learned integrating Calcite into Hive is that only a branch should
>>>> depend on snapshot versions of another component. (Hive broke this rule
>>> and
>>>> got into a sticky situation where their trunk depended on a Calcite
>>>> snapshot and they couldn’t release because Calcite was stuck in
>>> pre-release
>>>> vote limbo.)
>>>>
>>>> I’m going to try to produce maven artifacts with {groupId:
>>>> “org.apache.calcite”, version: "1.1.0-stream-incubating-SNAPSHOT”,
>>>> artifactId: “calcite-core”, “calcite-avatica”, etc.} and push them to
>>>> Apache nexus. I do recommend that you use it on a branch.
>>>>
>>>> However, I don’t think there would be a problem getting the functionality
>>>> into calcite’s master branch in time for Calcite’s next release, in
>>> about 4
>>>> weeks. (We commit to releasing approximately monthly.) The functionality
>>>> and APIs would be marked “experimental, subject to change” but at least
>>> you
>>>> would be able to merge your work to master at that point. Doubtless you
>>>> would need new features & bug fixes in Calcite after that, and so work
>>> that
>>>> depended on that would have to stay in a branch until Calcite was
>>> released.
>>>>
>>>> Julian
>>>>
>>>> On Feb 11, 2015, at 10:34 AM, Felix GV <fv...@linkedin.com.INVALID>
>>>> wrote:
>>>>
>>>>> I guess one concern with Calcite SNAPSHOT releases is that they imply
>>>> they'll be included in the next release, which may not be the case if
>>>> they're still considered very experimental.
>>>>>
>>>>> Alternatively, perhaps Julian can create a new version name with a
>>>> classifier for streaming.
>>>>>
>>>>>
>>>>
>>> http://maven.apache.org/plugins/maven-deploy-plugin/examples/deploying-with-classifiers.html
>>>>>
>>>>> Then he could release SNAPSHOT artifacts for both the main/master code
>>>> base as well as the streaming branch, no matter how experimental it is...
>>>>>
>>>>> --
>>>>>
>>>>> Felix GV
>>>>> Data Infrastructure Engineer
>>>>> Distributed Data Systems
>>>>> LinkedIn
>>>>>
>>>>> fgv@linkedin.com
>>>>> linkedin.com/in/felixgv
>>>>>
>>>>> ________________________________________
>>>>> From: Chris Riccomini [criccomini@apache.org]
>>>>> Sent: Wednesday, February 11, 2015 10:23 AM
>>>>> To: Chris Riccomini
>>>>> Cc: dev@samza.apache.org
>>>>> Subject: Re: [DISCUSS] SQL workflow
>>>>>
>>>>> Hey Julian,
>>>>>
>>>>> It looks like Calcite is at least setup to publish to Maven:
>>>>>
>>>>> https://repository.apache.org/#nexus-search;quick~calcite
>>>>>
>>>>> Julian, can you publish SNAPSHOT releases with your streaming changes
>>> to
>>>> it?
>>>>>
>>>>> Thanks!
>>>>> Chris
>>>>>
>>>>> On Wed, Feb 11, 2015 at 10:08 AM, Chris Riccomini <
>>> criccomini@apache.org
>>>>>
>>>>> wrote:
>>>>>
>>>>>> Hey Milinda,
>>>>>>
>>>>>> I've just committed SAMZA-484, so you should be able to re-base and
>>> get
>>>>>> all the latest code.
>>>>>>
>>>>>>> But we need to add Calcite as a dependency and Calcite streaming
>>>>>> support is only in Julian's branch
>>>>>>
>>>>>> We'll have to have Julian publish a release to Apache. This can be a
>>>>>> SNAPSHOT to Apache's SNAPSHOT repo, or a real Calcite release. We
>>> can't
>>>>>> check in binaries to the repo (licensing, release complexity, and just
>>>>>> generally a bad idea). We'll have to coordinate with him.
>>>>>>
>>>>>> Cheers,
>>>>>> Chris
>>>>>>
>>>>>> On Wed, Feb 11, 2015 at 9:43 AM, Milinda Pathirage <
>>>> mpathira@umail.iu.edu>
>>>>>> wrote:
>>>>>>
>>>>>>> No, SAMZA-483 doesn't depend on SAMZA-484. But we need to add Calcite
>>>> as a
>>>>>>> dependency and Calcite streaming support is only in Julian's branch (
>>>>>>> https://github.com/julianhyde/incubator-calcite/tree/chi). What can
>>>> we do
>>>>>>> about that?
>>>>>>>
>>>>>>> Milinda
>>>>>>>
>>>>>>> On Wed, Feb 11, 2015 at 12:30 PM, Chris Riccomini <
>>>> criccomini@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hey Milinda,
>>>>>>>>
>>>>>>>> That'd be great. Do you have any dependency on SAMZA-484? I haven't
>>>> had
>>>>>>> a
>>>>>>>> chance to commit that one yet.
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> Chris
>>>>>>>>
>>>>>>>> On Wed, Feb 11, 2015 at 9:23 AM, Milinda Pathirage <
>>>>>>> mpathira@umail.iu.edu>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> /need/want
>>>>>>>>>
>>>>>>>>> Milinda
>>>>>>>>>
>>>>>>>>> On Wed, Feb 11, 2015 at 12:22 PM, Milinda Pathirage <
>>>>>>>> mpathira@umail.iu.edu
>>>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Chris,
>>>>>>>>>>
>>>>>>>>>> Do you need me to create the SAMZA-483 patch against latest master
>>>>>>> with
>>>>>>>>>> SAMZA-482 patch? I think that will make it easier to review the
>>>>>>> patch?
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>> Milinda
>>>>>>>>>>
>>>>>>>>>> On Mon, Feb 9, 2015 at 11:39 AM, Chris Riccomini <
>>>>>>>> criccomini@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Cool. Looks like we've got consensus. I'll move forward with RTC
>>> on
>>>>>>>> some
>>>>>>>>>>> of
>>>>>>>>>>> the early SAMZA-390 tickets. (SAMZA-482, SAMZA-483, SAMZA-484)
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Feb 9, 2015 at 8:35 AM, Yan Fang <ya...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> +1 on this.
>>>>>>>>>>>>
>>>>>>>>>>>> Fang, Yan
>>>>>>>>>>>> yanfang724@gmail.com
>>>>>>>>>>>> +1 (206) 849-4108
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Feb 6, 2015 at 4:38 PM, Chris Riccomini <
>>>>>>>>> criccomini@apache.org>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hey all,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Are we +1 on this? I think Jakob was the only one who was
>>>>>>> curious
>>>>>>>>>>> about
>>>>>>>>>>>> it.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>> Chris
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Feb 5, 2015 at 1:22 PM, Yi Pan <ni...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi, Jakob,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Eh? Not sure what this means...
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I mean SAMZA-484 depends on SAMZA-482, and neither are
>>>>>>>>> committed.
>>>>>>>>>>> So
>>>>>>>>>>>>>> Navina
>>>>>>>>>>>>>>> is having to post Yi's patch, as well as her own, on the
>>>>>>> JIRA.
>>>>>>>>> It
>>>>>>>>>>>> makes
>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>> really hard to do code reviews because you can't tell
>>>>>>> whether
>>>>>>>> Yi
>>>>>>>>>>> made
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> changes or Navina did.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Just to add to the point. It is also difficult to always see
>>>>>>> a
>>>>>>>>> long
>>>>>>>>>>>> list
>>>>>>>>>>>>> of
>>>>>>>>>>>>>> changed files if the RB request is always based on the
>>>>>>> master.
>>>>>>>> It
>>>>>>>>> is
>>>>>>>>>>>>>> possible to have RB request based on another RB request (I
>>>>>>> have
>>>>>>>>>>> tried
>>>>>>>>>>>> it
>>>>>>>>>>>>>> before). But what happens if the base RB request is
>>>>>>>>>>>> cancelled/discarded?
>>>>>>>>>>>>> RB
>>>>>>>>>>>>>> is not designed to track the revision changes in a dependency
>>>>>>>>> chain.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>> Chris
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Thu, Feb 5, 2015 at 12:16 PM, Jakob Homan <
>>>>>>>> jghoman@gmail.com
>>>>>>>>>>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I want to avoid branches,
>>>>>>>>>>>>>>>> Just curious, any reason for this?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> and I also want to avoid revision control over JIRA
>>>>>>>>>>>>>>>> Eh? Not sure what this means...
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> jg
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 4 February 2015 at 17:11, Chris Riccomini <
>>>>>>>>>>>> criccomini@apache.org>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> Hey all,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> @Jakob, yeah I was thinking we'll follow our normal
>>>>>>> flow.
>>>>>>>>>>> RTC. I
>>>>>>>>>>>>> just
>>>>>>>>>>>>>>>>> wanted to set expectation that the code committed
>>>>>>> might be
>>>>>>>>>>> not up
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> our
>>>>>>>>>>>>>>>>> normal quality initially (missing docs, no tests, etc).
>>>>>>>>> Until
>>>>>>>>>>> the
>>>>>>>>>>>>>>> quality
>>>>>>>>>>>>>>>>> is raised, we should think of this module as
>>>>>>> experimental.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> @Milinda, awesome! Thanks. :)
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>>> Chris
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 11:57 AM, Milinda Pathirage <
>>>>>>>>>>>>>>>> mpathira@umail.iu.edu>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi Chris,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hope we no longer need the SQL API. I'll create a RB
>>>>>>> for
>>>>>>>>>>> Calcite
>>>>>>>>>>>>>>>>>> integration.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>> Milinda
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 1:31 PM, Chris Riccomini <
>>>>>>>>>>>>>>> criccomini@apache.org>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I think so. There was some RB downtime, but it just
>>>>>>> got
>>>>>>>>>>> fixed.
>>>>>>>>>>>>> Yi,
>>>>>>>>>>>>>>>>>> Navina,
>>>>>>>>>>>>>>>>>>> Milinda, can you make sure your JIRAs have up to
>>>>>>> date
>>>>>>>>> RBs?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 10:24 AM, sriram <
>>>>>>>>>>> sriram.sub@gmail.com
>>>>>>>>>>>>>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Can we have updated RBs for all the three sub
>>>>>>> tasks
>>>>>>>>>>> before
>>>>>>>>>>>> we
>>>>>>>>>>>>>>>> commit?
>>>>>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>>>>> would help us to review even after we commit.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini <
>>>>>>>>>>>>>>>>>> criccomini@apache.org>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Hey all,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Yi, Navina, and Milinda have been working on
>>>>>>>>> SAMZA-390
>>>>>>>>>>>>>>> sub-tickets
>>>>>>>>>>>>>>>>>>>> related
>>>>>>>>>>>>>>>>>>>>> to SQL operators. We're getting to the point
>>>>>>> where
>>>>>>>>> the
>>>>>>>>>>>>> amount
>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>>> floating around is quite large, and some tickets
>>>>>>>>> build
>>>>>>>>>>> off
>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>> others.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I'm proposing that we commit this work into a
>>>>>>>>> samza-sql
>>>>>>>>>>>>>>> submodule
>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>>> master, and treat this module as experimental. I
>>>>>>>> want
>>>>>>>>>>> to
>>>>>>>>>>>>> avoid
>>>>>>>>>>>>>>>>>>> branches,
>>>>>>>>>>>>>>>>>>>>> and I also want to avoid revision control over
>>>>>>>> JIRA.
>>>>>>>>>>> This
>>>>>>>>>>>>>> means
>>>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>>>>>> will probably be a fair amount of commits/JIRAs
>>>>>>> on
>>>>>>>>> this
>>>>>>>>>>>>> module
>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>> iterate, but I think that's OK.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Does this sound good to everyone?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>>>>>>> Chris
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>> Milinda Pathirage
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> PhD Student | Research Assistant
>>>>>>>>>>>>>>>>>> School of Informatics and Computing | Data to Insight
>>>>>>>>> Center
>>>>>>>>>>>>>>>>>> Indiana University
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> twitter: milindalakmal
>>>>>>>>>>>>>>>>>> skype: milinda.pathirage
>>>>>>>>>>>>>>>>>> blog: http://milinda.pathirage.org
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Milinda Pathirage
>>>>>>>>>>
>>>>>>>>>> PhD Student | Research Assistant
>>>>>>>>>> School of Informatics and Computing | Data to Insight Center
>>>>>>>>>> Indiana University
>>>>>>>>>>
>>>>>>>>>> twitter: milindalakmal
>>>>>>>>>> skype: milinda.pathirage
>>>>>>>>>> blog: http://milinda.pathirage.org
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Milinda Pathirage
>>>>>>>>>
>>>>>>>>> PhD Student | Research Assistant
>>>>>>>>> School of Informatics and Computing | Data to Insight Center
>>>>>>>>> Indiana University
>>>>>>>>>
>>>>>>>>> twitter: milindalakmal
>>>>>>>>> skype: milinda.pathirage
>>>>>>>>> blog: http://milinda.pathirage.org
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Milinda Pathirage
>>>>>>>
>>>>>>> PhD Student | Research Assistant
>>>>>>> School of Informatics and Computing | Data to Insight Center
>>>>>>> Indiana University
>>>>>>>
>>>>>>> twitter: milindalakmal
>>>>>>> skype: milinda.pathirage
>>>>>>> blog: http://milinda.pathirage.org
>>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>>
>>
>>
>>
>> --
>> Milinda Pathirage
>>
>> PhD Student | Research Assistant
>> School of Informatics and Computing | Data to Insight Center
>> Indiana University
>>
>> twitter: milindalakmal
>> skype: milinda.pathirage
>> blog: http://milinda.pathirage.org
Re: [DISCUSS] SQL workflow
Posted by Chris Riccomini <cr...@apache.org>.
@Jakob, totally agree. I think the semantics of the SQL module won't
change. RTC, experimental, more lax on tests/docs. When we commit into
master, we can enforce much stronger test/doc requirements. That's what I'm
thinking.
On Thu, Feb 12, 2015 at 9:26 AM, Jakob Homan <jg...@gmail.com> wrote:
> +1, assuming we're keeping RTC on the branch (, Jakob said pedantically).
>
> On 12 February 2015 at 08:25, Milinda Pathirage <mp...@umail.iu.edu>
> wrote:
> > I am +1 for this approach.
> >
> > Milinda
> >
> > On Thu, Feb 12, 2015 at 11:22 AM, Chris Riccomini <criccomini@apache.org
> >
> > wrote:
> >
> >> Hey Julian,
> >>
> >> > The rule of thumb I’ve learned integrating Calcite into Hive is that
> only
> >> a branch should depend on snapshot versions of another component
> >>
> >> This is very practical. I hadn't considered the Calcite dependency issue
> >> when I pushed for merging into master rather than a branch.
> >>
> >> Based on this, it seems like the best thing to do is:
> >>
> >> 1) Work off of Calcite SNAPSHOT Maven publication.
> >> 2) Create a samza-sql branch.
> >> 3) Migrate all existing samza-sql commits from master into the samza-sql
> >> branch.
> >>
> >> Sorry for the churn on this all. I hadn't considered the binary
> dependency
> >> issue. Is everyone OK with this?
> >>
> >> Cheers,
> >> Chris
> >>
> >> On Wed, Feb 11, 2015 at 10:51 AM, Julian Hyde <ju...@hydromatic.net>
> >> wrote:
> >>
> >> > This seems more like a branch than a classifier. Calcite is
> developing in
> >> > a branch, and would produce snapshots from that branch. The rule of
> thumb
> >> > I’ve learned integrating Calcite into Hive is that only a branch
> should
> >> > depend on snapshot versions of another component. (Hive broke this
> rule
> >> and
> >> > got into a sticky situation where their trunk depended on a Calcite
> >> > snapshot and they couldn’t release because Calcite was stuck in
> >> pre-release
> >> > vote limbo.)
> >> >
> >> > I’m going to try to produce maven artifacts with {groupId:
> >> > “org.apache.calcite”, version: "1.1.0-stream-incubating-SNAPSHOT”,
> >> > artifactId: “calcite-core”, “calcite-avatica”, etc.} and push them to
> >> > Apache nexus. I do recommend that you use it on a branch.
> >> >
> >> > However, I don’t think there would be a problem getting the
> functionality
> >> > into calcite’s master branch in time for Calcite’s next release, in
> >> about 4
> >> > weeks. (We commit to releasing approximately monthly.) The
> functionality
> >> > and APIs would be marked “experimental, subject to change” but at
> least
> >> you
> >> > would be able to merge your work to master at that point. Doubtless
> you
> >> > would need new features & bug fixes in Calcite after that, and so work
> >> that
> >> > depended on that would have to stay in a branch until Calcite was
> >> released.
> >> >
> >> > Julian
> >> >
> >> > On Feb 11, 2015, at 10:34 AM, Felix GV <fvillegas@linkedin.com.INVALID
> >
> >> > wrote:
> >> >
> >> > > I guess one concern with Calcite SNAPSHOT releases is that they
> imply
> >> > they'll be included in the next release, which may not be the case if
> >> > they're still considered very experimental.
> >> > >
> >> > > Alternatively, perhaps Julian can create a new version name with a
> >> > classifier for streaming.
> >> > >
> >> > >
> >> >
> >>
> http://maven.apache.org/plugins/maven-deploy-plugin/examples/deploying-with-classifiers.html
> >> > >
> >> > > Then he could release SNAPSHOT artifacts for both the main/master
> code
> >> > base as well as the streaming branch, no matter how experimental it
> is...
> >> > >
> >> > > --
> >> > >
> >> > > Felix GV
> >> > > Data Infrastructure Engineer
> >> > > Distributed Data Systems
> >> > > LinkedIn
> >> > >
> >> > > fgv@linkedin.com
> >> > > linkedin.com/in/felixgv
> >> > >
> >> > > ________________________________________
> >> > > From: Chris Riccomini [criccomini@apache.org]
> >> > > Sent: Wednesday, February 11, 2015 10:23 AM
> >> > > To: Chris Riccomini
> >> > > Cc: dev@samza.apache.org
> >> > > Subject: Re: [DISCUSS] SQL workflow
> >> > >
> >> > > Hey Julian,
> >> > >
> >> > > It looks like Calcite is at least setup to publish to Maven:
> >> > >
> >> > > https://repository.apache.org/#nexus-search;quick~calcite
> >> > >
> >> > > Julian, can you publish SNAPSHOT releases with your streaming
> changes
> >> to
> >> > it?
> >> > >
> >> > > Thanks!
> >> > > Chris
> >> > >
> >> > > On Wed, Feb 11, 2015 at 10:08 AM, Chris Riccomini <
> >> criccomini@apache.org
> >> > >
> >> > > wrote:
> >> > >
> >> > >> Hey Milinda,
> >> > >>
> >> > >> I've just committed SAMZA-484, so you should be able to re-base and
> >> get
> >> > >> all the latest code.
> >> > >>
> >> > >>> But we need to add Calcite as a dependency and Calcite streaming
> >> > >> support is only in Julian's branch
> >> > >>
> >> > >> We'll have to have Julian publish a release to Apache. This can be
> a
> >> > >> SNAPSHOT to Apache's SNAPSHOT repo, or a real Calcite release. We
> >> can't
> >> > >> check in binaries to the repo (licensing, release complexity, and
> just
> >> > >> generally a bad idea). We'll have to coordinate with him.
> >> > >>
> >> > >> Cheers,
> >> > >> Chris
> >> > >>
> >> > >> On Wed, Feb 11, 2015 at 9:43 AM, Milinda Pathirage <
> >> > mpathira@umail.iu.edu>
> >> > >> wrote:
> >> > >>
> >> > >>> No, SAMZA-483 doesn't depend on SAMZA-484. But we need to add
> Calcite
> >> > as a
> >> > >>> dependency and Calcite streaming support is only in Julian's
> branch (
> >> > >>> https://github.com/julianhyde/incubator-calcite/tree/chi). What
> can
> >> > we do
> >> > >>> about that?
> >> > >>>
> >> > >>> Milinda
> >> > >>>
> >> > >>> On Wed, Feb 11, 2015 at 12:30 PM, Chris Riccomini <
> >> > criccomini@apache.org>
> >> > >>> wrote:
> >> > >>>
> >> > >>>> Hey Milinda,
> >> > >>>>
> >> > >>>> That'd be great. Do you have any dependency on SAMZA-484? I
> haven't
> >> > had
> >> > >>> a
> >> > >>>> chance to commit that one yet.
> >> > >>>>
> >> > >>>> Cheers,
> >> > >>>> Chris
> >> > >>>>
> >> > >>>> On Wed, Feb 11, 2015 at 9:23 AM, Milinda Pathirage <
> >> > >>> mpathira@umail.iu.edu>
> >> > >>>> wrote:
> >> > >>>>
> >> > >>>>> /need/want
> >> > >>>>>
> >> > >>>>> Milinda
> >> > >>>>>
> >> > >>>>> On Wed, Feb 11, 2015 at 12:22 PM, Milinda Pathirage <
> >> > >>>> mpathira@umail.iu.edu
> >> > >>>>>>
> >> > >>>>> wrote:
> >> > >>>>>
> >> > >>>>>> Hi Chris,
> >> > >>>>>>
> >> > >>>>>> Do you need me to create the SAMZA-483 patch against latest
> master
> >> > >>> with
> >> > >>>>>> SAMZA-482 patch? I think that will make it easier to review the
> >> > >>> patch?
> >> > >>>>>>
> >> > >>>>>> Thanks
> >> > >>>>>> Milinda
> >> > >>>>>>
> >> > >>>>>> On Mon, Feb 9, 2015 at 11:39 AM, Chris Riccomini <
> >> > >>>> criccomini@apache.org>
> >> > >>>>>> wrote:
> >> > >>>>>>
> >> > >>>>>>> Cool. Looks like we've got consensus. I'll move forward with
> RTC
> >> on
> >> > >>>> some
> >> > >>>>>>> of
> >> > >>>>>>> the early SAMZA-390 tickets. (SAMZA-482, SAMZA-483, SAMZA-484)
> >> > >>>>>>>
> >> > >>>>>>> On Mon, Feb 9, 2015 at 8:35 AM, Yan Fang <
> yanfang724@gmail.com>
> >> > >>>> wrote:
> >> > >>>>>>>
> >> > >>>>>>>> +1 on this.
> >> > >>>>>>>>
> >> > >>>>>>>> Fang, Yan
> >> > >>>>>>>> yanfang724@gmail.com
> >> > >>>>>>>> +1 (206) 849-4108
> >> > >>>>>>>>
> >> > >>>>>>>> On Fri, Feb 6, 2015 at 4:38 PM, Chris Riccomini <
> >> > >>>>> criccomini@apache.org>
> >> > >>>>>>>> wrote:
> >> > >>>>>>>>
> >> > >>>>>>>>> Hey all,
> >> > >>>>>>>>>
> >> > >>>>>>>>> Are we +1 on this? I think Jakob was the only one who was
> >> > >>> curious
> >> > >>>>>>> about
> >> > >>>>>>>> it.
> >> > >>>>>>>>>
> >> > >>>>>>>>> Cheers,
> >> > >>>>>>>>> Chris
> >> > >>>>>>>>>
> >> > >>>>>>>>> On Thu, Feb 5, 2015 at 1:22 PM, Yi Pan <nickpan47@gmail.com
> >
> >> > >>>> wrote:
> >> > >>>>>>>>>
> >> > >>>>>>>>>> Hi, Jakob,
> >> > >>>>>>>>>>
> >> > >>>>>>>>>>
> >> > >>>>>>>>>>>> Eh? Not sure what this means...
> >> > >>>>>>>>>>>
> >> > >>>>>>>>>>> I mean SAMZA-484 depends on SAMZA-482, and neither are
> >> > >>>>> committed.
> >> > >>>>>>> So
> >> > >>>>>>>>>> Navina
> >> > >>>>>>>>>>> is having to post Yi's patch, as well as her own, on the
> >> > >>> JIRA.
> >> > >>>>> It
> >> > >>>>>>>> makes
> >> > >>>>>>>>>> it
> >> > >>>>>>>>>>> really hard to do code reviews because you can't tell
> >> > >>> whether
> >> > >>>> Yi
> >> > >>>>>>> made
> >> > >>>>>>>>> the
> >> > >>>>>>>>>>> changes or Navina did.
> >> > >>>>>>>>>>>
> >> > >>>>>>>>>>>
> >> > >>>>>>>>>> Just to add to the point. It is also difficult to always
> see
> >> > >>> a
> >> > >>>>> long
> >> > >>>>>>>> list
> >> > >>>>>>>>> of
> >> > >>>>>>>>>> changed files if the RB request is always based on the
> >> > >>> master.
> >> > >>>> It
> >> > >>>>> is
> >> > >>>>>>>>>> possible to have RB request based on another RB request (I
> >> > >>> have
> >> > >>>>>>> tried
> >> > >>>>>>>> it
> >> > >>>>>>>>>> before). But what happens if the base RB request is
> >> > >>>>>>>> cancelled/discarded?
> >> > >>>>>>>>> RB
> >> > >>>>>>>>>> is not designed to track the revision changes in a
> dependency
> >> > >>>>> chain.
> >> > >>>>>>>>>>
> >> > >>>>>>>>>>
> >> > >>>>>>>>>>
> >> > >>>>>>>>>>> Cheers,
> >> > >>>>>>>>>>> Chris
> >> > >>>>>>>>>>>
> >> > >>>>>>>>>>> On Thu, Feb 5, 2015 at 12:16 PM, Jakob Homan <
> >> > >>>> jghoman@gmail.com
> >> > >>>>>>
> >> > >>>>>>>>> wrote:
> >> > >>>>>>>>>>>
> >> > >>>>>>>>>>>>> I want to avoid branches,
> >> > >>>>>>>>>>>> Just curious, any reason for this?
> >> > >>>>>>>>>>>>
> >> > >>>>>>>>>>>>> and I also want to avoid revision control over JIRA
> >> > >>>>>>>>>>>> Eh? Not sure what this means...
> >> > >>>>>>>>>>>>
> >> > >>>>>>>>>>>> Thanks,
> >> > >>>>>>>>>>>> jg
> >> > >>>>>>>>>>>>
> >> > >>>>>>>>>>>> On 4 February 2015 at 17:11, Chris Riccomini <
> >> > >>>>>>>> criccomini@apache.org>
> >> > >>>>>>>>>>>> wrote:
> >> > >>>>>>>>>>>>> Hey all,
> >> > >>>>>>>>>>>>>
> >> > >>>>>>>>>>>>> @Jakob, yeah I was thinking we'll follow our normal
> >> > >>> flow.
> >> > >>>>>>> RTC. I
> >> > >>>>>>>>> just
> >> > >>>>>>>>>>>>> wanted to set expectation that the code committed
> >> > >>> might be
> >> > >>>>>>> not up
> >> > >>>>>>>>> to
> >> > >>>>>>>>>>> our
> >> > >>>>>>>>>>>>> normal quality initially (missing docs, no tests, etc).
> >> > >>>>> Until
> >> > >>>>>>> the
> >> > >>>>>>>>>>> quality
> >> > >>>>>>>>>>>>> is raised, we should think of this module as
> >> > >>> experimental.
> >> > >>>>>>>>>>>>>
> >> > >>>>>>>>>>>>> @Milinda, awesome! Thanks. :)
> >> > >>>>>>>>>>>>>
> >> > >>>>>>>>>>>>> Cheers,
> >> > >>>>>>>>>>>>> Chris
> >> > >>>>>>>>>>>>>
> >> > >>>>>>>>>>>>> On Wed, Feb 4, 2015 at 11:57 AM, Milinda Pathirage <
> >> > >>>>>>>>>>>> mpathira@umail.iu.edu>
> >> > >>>>>>>>>>>>> wrote:
> >> > >>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>> Hi Chris,
> >> > >>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>> Hope we no longer need the SQL API. I'll create a RB
> >> > >>> for
> >> > >>>>>>> Calcite
> >> > >>>>>>>>>>>>>> integration.
> >> > >>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>> Thanks
> >> > >>>>>>>>>>>>>> Milinda
> >> > >>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 1:31 PM, Chris Riccomini <
> >> > >>>>>>>>>>> criccomini@apache.org>
> >> > >>>>>>>>>>>>>> wrote:
> >> > >>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>>> I think so. There was some RB downtime, but it just
> >> > >>> got
> >> > >>>>>>> fixed.
> >> > >>>>>>>>> Yi,
> >> > >>>>>>>>>>>>>> Navina,
> >> > >>>>>>>>>>>>>>> Milinda, can you make sure your JIRAs have up to
> >> > >>> date
> >> > >>>>> RBs?
> >> > >>>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 10:24 AM, sriram <
> >> > >>>>>>> sriram.sub@gmail.com
> >> > >>>>>>>>>
> >> > >>>>>>>>>>> wrote:
> >> > >>>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>>>> Can we have updated RBs for all the three sub
> >> > >>> tasks
> >> > >>>>>>> before
> >> > >>>>>>>> we
> >> > >>>>>>>>>>>> commit?
> >> > >>>>>>>>>>>>>>> This
> >> > >>>>>>>>>>>>>>>> would help us to review even after we commit.
> >> > >>>>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini <
> >> > >>>>>>>>>>>>>> criccomini@apache.org>
> >> > >>>>>>>>>>>>>>>> wrote:
> >> > >>>>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>>>>> Hey all,
> >> > >>>>>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>>>>> Yi, Navina, and Milinda have been working on
> >> > >>>>> SAMZA-390
> >> > >>>>>>>>>>> sub-tickets
> >> > >>>>>>>>>>>>>>>> related
> >> > >>>>>>>>>>>>>>>>> to SQL operators. We're getting to the point
> >> > >>> where
> >> > >>>>> the
> >> > >>>>>>>>> amount
> >> > >>>>>>>>>> of
> >> > >>>>>>>>>>>> work
> >> > >>>>>>>>>>>>>>>>> floating around is quite large, and some tickets
> >> > >>>>> build
> >> > >>>>>>> off
> >> > >>>>>>>>> of
> >> > >>>>>>>>>>>> others.
> >> > >>>>>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>>>>> I'm proposing that we commit this work into a
> >> > >>>>> samza-sql
> >> > >>>>>>>>>>> submodule
> >> > >>>>>>>>>>>> on
> >> > >>>>>>>>>>>>>>>>> master, and treat this module as experimental. I
> >> > >>>> want
> >> > >>>>>>> to
> >> > >>>>>>>>> avoid
> >> > >>>>>>>>>>>>>>> branches,
> >> > >>>>>>>>>>>>>>>>> and I also want to avoid revision control over
> >> > >>>> JIRA.
> >> > >>>>>>> This
> >> > >>>>>>>>>> means
> >> > >>>>>>>>>>>> that
> >> > >>>>>>>>>>>>>>>> there
> >> > >>>>>>>>>>>>>>>>> will probably be a fair amount of commits/JIRAs
> >> > >>> on
> >> > >>>>> this
> >> > >>>>>>>>> module
> >> > >>>>>>>>>>> as
> >> > >>>>>>>>>>>> we
> >> > >>>>>>>>>>>>>>>>> iterate, but I think that's OK.
> >> > >>>>>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>>>>> Does this sound good to everyone?
> >> > >>>>>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>>>>> Cheers,
> >> > >>>>>>>>>>>>>>>>> Chris
> >> > >>>>>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>> --
> >> > >>>>>>>>>>>>>> Milinda Pathirage
> >> > >>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>> PhD Student | Research Assistant
> >> > >>>>>>>>>>>>>> School of Informatics and Computing | Data to Insight
> >> > >>>>> Center
> >> > >>>>>>>>>>>>>> Indiana University
> >> > >>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>>> twitter: milindalakmal
> >> > >>>>>>>>>>>>>> skype: milinda.pathirage
> >> > >>>>>>>>>>>>>> blog: http://milinda.pathirage.org
> >> > >>>>>>>>>>>>>>
> >> > >>>>>>>>>>>>
> >> > >>>>>>>>>>>
> >> > >>>>>>>>>>
> >> > >>>>>>>>>
> >> > >>>>>>>>
> >> > >>>>>>>
> >> > >>>>>>
> >> > >>>>>>
> >> > >>>>>>
> >> > >>>>>> --
> >> > >>>>>> Milinda Pathirage
> >> > >>>>>>
> >> > >>>>>> PhD Student | Research Assistant
> >> > >>>>>> School of Informatics and Computing | Data to Insight Center
> >> > >>>>>> Indiana University
> >> > >>>>>>
> >> > >>>>>> twitter: milindalakmal
> >> > >>>>>> skype: milinda.pathirage
> >> > >>>>>> blog: http://milinda.pathirage.org
> >> > >>>>>>
> >> > >>>>>
> >> > >>>>>
> >> > >>>>>
> >> > >>>>> --
> >> > >>>>> Milinda Pathirage
> >> > >>>>>
> >> > >>>>> PhD Student | Research Assistant
> >> > >>>>> School of Informatics and Computing | Data to Insight Center
> >> > >>>>> Indiana University
> >> > >>>>>
> >> > >>>>> twitter: milindalakmal
> >> > >>>>> skype: milinda.pathirage
> >> > >>>>> blog: http://milinda.pathirage.org
> >> > >>>>>
> >> > >>>>
> >> > >>>
> >> > >>>
> >> > >>>
> >> > >>> --
> >> > >>> Milinda Pathirage
> >> > >>>
> >> > >>> PhD Student | Research Assistant
> >> > >>> School of Informatics and Computing | Data to Insight Center
> >> > >>> Indiana University
> >> > >>>
> >> > >>> twitter: milindalakmal
> >> > >>> skype: milinda.pathirage
> >> > >>> blog: http://milinda.pathirage.org
> >> > >>>
> >> > >>
> >> > >>
> >> >
> >> >
> >>
> >
> >
> >
> > --
> > Milinda Pathirage
> >
> > PhD Student | Research Assistant
> > School of Informatics and Computing | Data to Insight Center
> > Indiana University
> >
> > twitter: milindalakmal
> > skype: milinda.pathirage
> > blog: http://milinda.pathirage.org
>
Re: [DISCUSS] SQL workflow
Posted by Jakob Homan <jg...@gmail.com>.
+1, assuming we're keeping RTC on the branch (, Jakob said pedantically).
On 12 February 2015 at 08:25, Milinda Pathirage <mp...@umail.iu.edu> wrote:
> I am +1 for this approach.
>
> Milinda
>
> On Thu, Feb 12, 2015 at 11:22 AM, Chris Riccomini <cr...@apache.org>
> wrote:
>
>> Hey Julian,
>>
>> > The rule of thumb I’ve learned integrating Calcite into Hive is that only
>> a branch should depend on snapshot versions of another component
>>
>> This is very practical. I hadn't considered the Calcite dependency issue
>> when I pushed for merging into master rather than a branch.
>>
>> Based on this, it seems like the best thing to do is:
>>
>> 1) Work off of Calcite SNAPSHOT Maven publication.
>> 2) Create a samza-sql branch.
>> 3) Migrate all existing samza-sql commits from master into the samza-sql
>> branch.
>>
>> Sorry for the churn on this all. I hadn't considered the binary dependency
>> issue. Is everyone OK with this?
>>
>> Cheers,
>> Chris
>>
>> On Wed, Feb 11, 2015 at 10:51 AM, Julian Hyde <ju...@hydromatic.net>
>> wrote:
>>
>> > This seems more like a branch than a classifier. Calcite is developing in
>> > a branch, and would produce snapshots from that branch. The rule of thumb
>> > I’ve learned integrating Calcite into Hive is that only a branch should
>> > depend on snapshot versions of another component. (Hive broke this rule
>> and
>> > got into a sticky situation where their trunk depended on a Calcite
>> > snapshot and they couldn’t release because Calcite was stuck in
>> pre-release
>> > vote limbo.)
>> >
>> > I’m going to try to produce maven artifacts with {groupId:
>> > “org.apache.calcite”, version: "1.1.0-stream-incubating-SNAPSHOT”,
>> > artifactId: “calcite-core”, “calcite-avatica”, etc.} and push them to
>> > Apache nexus. I do recommend that you use it on a branch.
>> >
>> > However, I don’t think there would be a problem getting the functionality
>> > into calcite’s master branch in time for Calcite’s next release, in
>> about 4
>> > weeks. (We commit to releasing approximately monthly.) The functionality
>> > and APIs would be marked “experimental, subject to change” but at least
>> you
>> > would be able to merge your work to master at that point. Doubtless you
>> > would need new features & bug fixes in Calcite after that, and so work
>> that
>> > depended on that would have to stay in a branch until Calcite was
>> released.
>> >
>> > Julian
>> >
>> > On Feb 11, 2015, at 10:34 AM, Felix GV <fv...@linkedin.com.INVALID>
>> > wrote:
>> >
>> > > I guess one concern with Calcite SNAPSHOT releases is that they imply
>> > they'll be included in the next release, which may not be the case if
>> > they're still considered very experimental.
>> > >
>> > > Alternatively, perhaps Julian can create a new version name with a
>> > classifier for streaming.
>> > >
>> > >
>> >
>> http://maven.apache.org/plugins/maven-deploy-plugin/examples/deploying-with-classifiers.html
>> > >
>> > > Then he could release SNAPSHOT artifacts for both the main/master code
>> > base as well as the streaming branch, no matter how experimental it is...
>> > >
>> > > --
>> > >
>> > > Felix GV
>> > > Data Infrastructure Engineer
>> > > Distributed Data Systems
>> > > LinkedIn
>> > >
>> > > fgv@linkedin.com
>> > > linkedin.com/in/felixgv
>> > >
>> > > ________________________________________
>> > > From: Chris Riccomini [criccomini@apache.org]
>> > > Sent: Wednesday, February 11, 2015 10:23 AM
>> > > To: Chris Riccomini
>> > > Cc: dev@samza.apache.org
>> > > Subject: Re: [DISCUSS] SQL workflow
>> > >
>> > > Hey Julian,
>> > >
>> > > It looks like Calcite is at least setup to publish to Maven:
>> > >
>> > > https://repository.apache.org/#nexus-search;quick~calcite
>> > >
>> > > Julian, can you publish SNAPSHOT releases with your streaming changes
>> to
>> > it?
>> > >
>> > > Thanks!
>> > > Chris
>> > >
>> > > On Wed, Feb 11, 2015 at 10:08 AM, Chris Riccomini <
>> criccomini@apache.org
>> > >
>> > > wrote:
>> > >
>> > >> Hey Milinda,
>> > >>
>> > >> I've just committed SAMZA-484, so you should be able to re-base and
>> get
>> > >> all the latest code.
>> > >>
>> > >>> But we need to add Calcite as a dependency and Calcite streaming
>> > >> support is only in Julian's branch
>> > >>
>> > >> We'll have to have Julian publish a release to Apache. This can be a
>> > >> SNAPSHOT to Apache's SNAPSHOT repo, or a real Calcite release. We
>> can't
>> > >> check in binaries to the repo (licensing, release complexity, and just
>> > >> generally a bad idea). We'll have to coordinate with him.
>> > >>
>> > >> Cheers,
>> > >> Chris
>> > >>
>> > >> On Wed, Feb 11, 2015 at 9:43 AM, Milinda Pathirage <
>> > mpathira@umail.iu.edu>
>> > >> wrote:
>> > >>
>> > >>> No, SAMZA-483 doesn't depend on SAMZA-484. But we need to add Calcite
>> > as a
>> > >>> dependency and Calcite streaming support is only in Julian's branch (
>> > >>> https://github.com/julianhyde/incubator-calcite/tree/chi). What can
>> > we do
>> > >>> about that?
>> > >>>
>> > >>> Milinda
>> > >>>
>> > >>> On Wed, Feb 11, 2015 at 12:30 PM, Chris Riccomini <
>> > criccomini@apache.org>
>> > >>> wrote:
>> > >>>
>> > >>>> Hey Milinda,
>> > >>>>
>> > >>>> That'd be great. Do you have any dependency on SAMZA-484? I haven't
>> > had
>> > >>> a
>> > >>>> chance to commit that one yet.
>> > >>>>
>> > >>>> Cheers,
>> > >>>> Chris
>> > >>>>
>> > >>>> On Wed, Feb 11, 2015 at 9:23 AM, Milinda Pathirage <
>> > >>> mpathira@umail.iu.edu>
>> > >>>> wrote:
>> > >>>>
>> > >>>>> /need/want
>> > >>>>>
>> > >>>>> Milinda
>> > >>>>>
>> > >>>>> On Wed, Feb 11, 2015 at 12:22 PM, Milinda Pathirage <
>> > >>>> mpathira@umail.iu.edu
>> > >>>>>>
>> > >>>>> wrote:
>> > >>>>>
>> > >>>>>> Hi Chris,
>> > >>>>>>
>> > >>>>>> Do you need me to create the SAMZA-483 patch against latest master
>> > >>> with
>> > >>>>>> SAMZA-482 patch? I think that will make it easier to review the
>> > >>> patch?
>> > >>>>>>
>> > >>>>>> Thanks
>> > >>>>>> Milinda
>> > >>>>>>
>> > >>>>>> On Mon, Feb 9, 2015 at 11:39 AM, Chris Riccomini <
>> > >>>> criccomini@apache.org>
>> > >>>>>> wrote:
>> > >>>>>>
>> > >>>>>>> Cool. Looks like we've got consensus. I'll move forward with RTC
>> on
>> > >>>> some
>> > >>>>>>> of
>> > >>>>>>> the early SAMZA-390 tickets. (SAMZA-482, SAMZA-483, SAMZA-484)
>> > >>>>>>>
>> > >>>>>>> On Mon, Feb 9, 2015 at 8:35 AM, Yan Fang <ya...@gmail.com>
>> > >>>> wrote:
>> > >>>>>>>
>> > >>>>>>>> +1 on this.
>> > >>>>>>>>
>> > >>>>>>>> Fang, Yan
>> > >>>>>>>> yanfang724@gmail.com
>> > >>>>>>>> +1 (206) 849-4108
>> > >>>>>>>>
>> > >>>>>>>> On Fri, Feb 6, 2015 at 4:38 PM, Chris Riccomini <
>> > >>>>> criccomini@apache.org>
>> > >>>>>>>> wrote:
>> > >>>>>>>>
>> > >>>>>>>>> Hey all,
>> > >>>>>>>>>
>> > >>>>>>>>> Are we +1 on this? I think Jakob was the only one who was
>> > >>> curious
>> > >>>>>>> about
>> > >>>>>>>> it.
>> > >>>>>>>>>
>> > >>>>>>>>> Cheers,
>> > >>>>>>>>> Chris
>> > >>>>>>>>>
>> > >>>>>>>>> On Thu, Feb 5, 2015 at 1:22 PM, Yi Pan <ni...@gmail.com>
>> > >>>> wrote:
>> > >>>>>>>>>
>> > >>>>>>>>>> Hi, Jakob,
>> > >>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>>>> Eh? Not sure what this means...
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> I mean SAMZA-484 depends on SAMZA-482, and neither are
>> > >>>>> committed.
>> > >>>>>>> So
>> > >>>>>>>>>> Navina
>> > >>>>>>>>>>> is having to post Yi's patch, as well as her own, on the
>> > >>> JIRA.
>> > >>>>> It
>> > >>>>>>>> makes
>> > >>>>>>>>>> it
>> > >>>>>>>>>>> really hard to do code reviews because you can't tell
>> > >>> whether
>> > >>>> Yi
>> > >>>>>>> made
>> > >>>>>>>>> the
>> > >>>>>>>>>>> changes or Navina did.
>> > >>>>>>>>>>>
>> > >>>>>>>>>>>
>> > >>>>>>>>>> Just to add to the point. It is also difficult to always see
>> > >>> a
>> > >>>>> long
>> > >>>>>>>> list
>> > >>>>>>>>> of
>> > >>>>>>>>>> changed files if the RB request is always based on the
>> > >>> master.
>> > >>>> It
>> > >>>>> is
>> > >>>>>>>>>> possible to have RB request based on another RB request (I
>> > >>> have
>> > >>>>>>> tried
>> > >>>>>>>> it
>> > >>>>>>>>>> before). But what happens if the base RB request is
>> > >>>>>>>> cancelled/discarded?
>> > >>>>>>>>> RB
>> > >>>>>>>>>> is not designed to track the revision changes in a dependency
>> > >>>>> chain.
>> > >>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>>> Cheers,
>> > >>>>>>>>>>> Chris
>> > >>>>>>>>>>>
>> > >>>>>>>>>>> On Thu, Feb 5, 2015 at 12:16 PM, Jakob Homan <
>> > >>>> jghoman@gmail.com
>> > >>>>>>
>> > >>>>>>>>> wrote:
>> > >>>>>>>>>>>
>> > >>>>>>>>>>>>> I want to avoid branches,
>> > >>>>>>>>>>>> Just curious, any reason for this?
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>>> and I also want to avoid revision control over JIRA
>> > >>>>>>>>>>>> Eh? Not sure what this means...
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> Thanks,
>> > >>>>>>>>>>>> jg
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>> On 4 February 2015 at 17:11, Chris Riccomini <
>> > >>>>>>>> criccomini@apache.org>
>> > >>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>> Hey all,
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> @Jakob, yeah I was thinking we'll follow our normal
>> > >>> flow.
>> > >>>>>>> RTC. I
>> > >>>>>>>>> just
>> > >>>>>>>>>>>>> wanted to set expectation that the code committed
>> > >>> might be
>> > >>>>>>> not up
>> > >>>>>>>>> to
>> > >>>>>>>>>>> our
>> > >>>>>>>>>>>>> normal quality initially (missing docs, no tests, etc).
>> > >>>>> Until
>> > >>>>>>> the
>> > >>>>>>>>>>> quality
>> > >>>>>>>>>>>>> is raised, we should think of this module as
>> > >>> experimental.
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> @Milinda, awesome! Thanks. :)
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> Cheers,
>> > >>>>>>>>>>>>> Chris
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>> On Wed, Feb 4, 2015 at 11:57 AM, Milinda Pathirage <
>> > >>>>>>>>>>>> mpathira@umail.iu.edu>
>> > >>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> Hi Chris,
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> Hope we no longer need the SQL API. I'll create a RB
>> > >>> for
>> > >>>>>>> Calcite
>> > >>>>>>>>>>>>>> integration.
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> Thanks
>> > >>>>>>>>>>>>>> Milinda
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 1:31 PM, Chris Riccomini <
>> > >>>>>>>>>>> criccomini@apache.org>
>> > >>>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> I think so. There was some RB downtime, but it just
>> > >>> got
>> > >>>>>>> fixed.
>> > >>>>>>>>> Yi,
>> > >>>>>>>>>>>>>> Navina,
>> > >>>>>>>>>>>>>>> Milinda, can you make sure your JIRAs have up to
>> > >>> date
>> > >>>>> RBs?
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 10:24 AM, sriram <
>> > >>>>>>> sriram.sub@gmail.com
>> > >>>>>>>>>
>> > >>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>> Can we have updated RBs for all the three sub
>> > >>> tasks
>> > >>>>>>> before
>> > >>>>>>>> we
>> > >>>>>>>>>>>> commit?
>> > >>>>>>>>>>>>>>> This
>> > >>>>>>>>>>>>>>>> would help us to review even after we commit.
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini <
>> > >>>>>>>>>>>>>> criccomini@apache.org>
>> > >>>>>>>>>>>>>>>> wrote:
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> Hey all,
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> Yi, Navina, and Milinda have been working on
>> > >>>>> SAMZA-390
>> > >>>>>>>>>>> sub-tickets
>> > >>>>>>>>>>>>>>>> related
>> > >>>>>>>>>>>>>>>>> to SQL operators. We're getting to the point
>> > >>> where
>> > >>>>> the
>> > >>>>>>>>> amount
>> > >>>>>>>>>> of
>> > >>>>>>>>>>>> work
>> > >>>>>>>>>>>>>>>>> floating around is quite large, and some tickets
>> > >>>>> build
>> > >>>>>>> off
>> > >>>>>>>>> of
>> > >>>>>>>>>>>> others.
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> I'm proposing that we commit this work into a
>> > >>>>> samza-sql
>> > >>>>>>>>>>> submodule
>> > >>>>>>>>>>>> on
>> > >>>>>>>>>>>>>>>>> master, and treat this module as experimental. I
>> > >>>> want
>> > >>>>>>> to
>> > >>>>>>>>> avoid
>> > >>>>>>>>>>>>>>> branches,
>> > >>>>>>>>>>>>>>>>> and I also want to avoid revision control over
>> > >>>> JIRA.
>> > >>>>>>> This
>> > >>>>>>>>>> means
>> > >>>>>>>>>>>> that
>> > >>>>>>>>>>>>>>>> there
>> > >>>>>>>>>>>>>>>>> will probably be a fair amount of commits/JIRAs
>> > >>> on
>> > >>>>> this
>> > >>>>>>>>> module
>> > >>>>>>>>>>> as
>> > >>>>>>>>>>>> we
>> > >>>>>>>>>>>>>>>>> iterate, but I think that's OK.
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> Does this sound good to everyone?
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>> Cheers,
>> > >>>>>>>>>>>>>>>>> Chris
>> > >>>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> --
>> > >>>>>>>>>>>>>> Milinda Pathirage
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> PhD Student | Research Assistant
>> > >>>>>>>>>>>>>> School of Informatics and Computing | Data to Insight
>> > >>>>> Center
>> > >>>>>>>>>>>>>> Indiana University
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>>> twitter: milindalakmal
>> > >>>>>>>>>>>>>> skype: milinda.pathirage
>> > >>>>>>>>>>>>>> blog: http://milinda.pathirage.org
>> > >>>>>>>>>>>>>>
>> > >>>>>>>>>>>>
>> > >>>>>>>>>>>
>> > >>>>>>>>>>
>> > >>>>>>>>>
>> > >>>>>>>>
>> > >>>>>>>
>> > >>>>>>
>> > >>>>>>
>> > >>>>>>
>> > >>>>>> --
>> > >>>>>> Milinda Pathirage
>> > >>>>>>
>> > >>>>>> PhD Student | Research Assistant
>> > >>>>>> School of Informatics and Computing | Data to Insight Center
>> > >>>>>> Indiana University
>> > >>>>>>
>> > >>>>>> twitter: milindalakmal
>> > >>>>>> skype: milinda.pathirage
>> > >>>>>> blog: http://milinda.pathirage.org
>> > >>>>>>
>> > >>>>>
>> > >>>>>
>> > >>>>>
>> > >>>>> --
>> > >>>>> Milinda Pathirage
>> > >>>>>
>> > >>>>> PhD Student | Research Assistant
>> > >>>>> School of Informatics and Computing | Data to Insight Center
>> > >>>>> Indiana University
>> > >>>>>
>> > >>>>> twitter: milindalakmal
>> > >>>>> skype: milinda.pathirage
>> > >>>>> blog: http://milinda.pathirage.org
>> > >>>>>
>> > >>>>
>> > >>>
>> > >>>
>> > >>>
>> > >>> --
>> > >>> Milinda Pathirage
>> > >>>
>> > >>> PhD Student | Research Assistant
>> > >>> School of Informatics and Computing | Data to Insight Center
>> > >>> Indiana University
>> > >>>
>> > >>> twitter: milindalakmal
>> > >>> skype: milinda.pathirage
>> > >>> blog: http://milinda.pathirage.org
>> > >>>
>> > >>
>> > >>
>> >
>> >
>>
>
>
>
> --
> Milinda Pathirage
>
> PhD Student | Research Assistant
> School of Informatics and Computing | Data to Insight Center
> Indiana University
>
> twitter: milindalakmal
> skype: milinda.pathirage
> blog: http://milinda.pathirage.org
Re: [DISCUSS] SQL workflow
Posted by Milinda Pathirage <mp...@umail.iu.edu>.
I am +1 for this approach.
Milinda
On Thu, Feb 12, 2015 at 11:22 AM, Chris Riccomini <cr...@apache.org>
wrote:
> Hey Julian,
>
> > The rule of thumb I’ve learned integrating Calcite into Hive is that only
> a branch should depend on snapshot versions of another component
>
> This is very practical. I hadn't considered the Calcite dependency issue
> when I pushed for merging into master rather than a branch.
>
> Based on this, it seems like the best thing to do is:
>
> 1) Work off of Calcite SNAPSHOT Maven publication.
> 2) Create a samza-sql branch.
> 3) Migrate all existing samza-sql commits from master into the samza-sql
> branch.
>
> Sorry for the churn on this all. I hadn't considered the binary dependency
> issue. Is everyone OK with this?
>
> Cheers,
> Chris
>
> On Wed, Feb 11, 2015 at 10:51 AM, Julian Hyde <ju...@hydromatic.net>
> wrote:
>
> > This seems more like a branch than a classifier. Calcite is developing in
> > a branch, and would produce snapshots from that branch. The rule of thumb
> > I’ve learned integrating Calcite into Hive is that only a branch should
> > depend on snapshot versions of another component. (Hive broke this rule
> and
> > got into a sticky situation where their trunk depended on a Calcite
> > snapshot and they couldn’t release because Calcite was stuck in
> pre-release
> > vote limbo.)
> >
> > I’m going to try to produce maven artifacts with {groupId:
> > “org.apache.calcite”, version: "1.1.0-stream-incubating-SNAPSHOT”,
> > artifactId: “calcite-core”, “calcite-avatica”, etc.} and push them to
> > Apache nexus. I do recommend that you use it on a branch.
> >
> > However, I don’t think there would be a problem getting the functionality
> > into calcite’s master branch in time for Calcite’s next release, in
> about 4
> > weeks. (We commit to releasing approximately monthly.) The functionality
> > and APIs would be marked “experimental, subject to change” but at least
> you
> > would be able to merge your work to master at that point. Doubtless you
> > would need new features & bug fixes in Calcite after that, and so work
> that
> > depended on that would have to stay in a branch until Calcite was
> released.
> >
> > Julian
> >
> > On Feb 11, 2015, at 10:34 AM, Felix GV <fv...@linkedin.com.INVALID>
> > wrote:
> >
> > > I guess one concern with Calcite SNAPSHOT releases is that they imply
> > they'll be included in the next release, which may not be the case if
> > they're still considered very experimental.
> > >
> > > Alternatively, perhaps Julian can create a new version name with a
> > classifier for streaming.
> > >
> > >
> >
> http://maven.apache.org/plugins/maven-deploy-plugin/examples/deploying-with-classifiers.html
> > >
> > > Then he could release SNAPSHOT artifacts for both the main/master code
> > base as well as the streaming branch, no matter how experimental it is...
> > >
> > > --
> > >
> > > Felix GV
> > > Data Infrastructure Engineer
> > > Distributed Data Systems
> > > LinkedIn
> > >
> > > fgv@linkedin.com
> > > linkedin.com/in/felixgv
> > >
> > > ________________________________________
> > > From: Chris Riccomini [criccomini@apache.org]
> > > Sent: Wednesday, February 11, 2015 10:23 AM
> > > To: Chris Riccomini
> > > Cc: dev@samza.apache.org
> > > Subject: Re: [DISCUSS] SQL workflow
> > >
> > > Hey Julian,
> > >
> > > It looks like Calcite is at least setup to publish to Maven:
> > >
> > > https://repository.apache.org/#nexus-search;quick~calcite
> > >
> > > Julian, can you publish SNAPSHOT releases with your streaming changes
> to
> > it?
> > >
> > > Thanks!
> > > Chris
> > >
> > > On Wed, Feb 11, 2015 at 10:08 AM, Chris Riccomini <
> criccomini@apache.org
> > >
> > > wrote:
> > >
> > >> Hey Milinda,
> > >>
> > >> I've just committed SAMZA-484, so you should be able to re-base and
> get
> > >> all the latest code.
> > >>
> > >>> But we need to add Calcite as a dependency and Calcite streaming
> > >> support is only in Julian's branch
> > >>
> > >> We'll have to have Julian publish a release to Apache. This can be a
> > >> SNAPSHOT to Apache's SNAPSHOT repo, or a real Calcite release. We
> can't
> > >> check in binaries to the repo (licensing, release complexity, and just
> > >> generally a bad idea). We'll have to coordinate with him.
> > >>
> > >> Cheers,
> > >> Chris
> > >>
> > >> On Wed, Feb 11, 2015 at 9:43 AM, Milinda Pathirage <
> > mpathira@umail.iu.edu>
> > >> wrote:
> > >>
> > >>> No, SAMZA-483 doesn't depend on SAMZA-484. But we need to add Calcite
> > as a
> > >>> dependency and Calcite streaming support is only in Julian's branch (
> > >>> https://github.com/julianhyde/incubator-calcite/tree/chi). What can
> > we do
> > >>> about that?
> > >>>
> > >>> Milinda
> > >>>
> > >>> On Wed, Feb 11, 2015 at 12:30 PM, Chris Riccomini <
> > criccomini@apache.org>
> > >>> wrote:
> > >>>
> > >>>> Hey Milinda,
> > >>>>
> > >>>> That'd be great. Do you have any dependency on SAMZA-484? I haven't
> > had
> > >>> a
> > >>>> chance to commit that one yet.
> > >>>>
> > >>>> Cheers,
> > >>>> Chris
> > >>>>
> > >>>> On Wed, Feb 11, 2015 at 9:23 AM, Milinda Pathirage <
> > >>> mpathira@umail.iu.edu>
> > >>>> wrote:
> > >>>>
> > >>>>> /need/want
> > >>>>>
> > >>>>> Milinda
> > >>>>>
> > >>>>> On Wed, Feb 11, 2015 at 12:22 PM, Milinda Pathirage <
> > >>>> mpathira@umail.iu.edu
> > >>>>>>
> > >>>>> wrote:
> > >>>>>
> > >>>>>> Hi Chris,
> > >>>>>>
> > >>>>>> Do you need me to create the SAMZA-483 patch against latest master
> > >>> with
> > >>>>>> SAMZA-482 patch? I think that will make it easier to review the
> > >>> patch?
> > >>>>>>
> > >>>>>> Thanks
> > >>>>>> Milinda
> > >>>>>>
> > >>>>>> On Mon, Feb 9, 2015 at 11:39 AM, Chris Riccomini <
> > >>>> criccomini@apache.org>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>>> Cool. Looks like we've got consensus. I'll move forward with RTC
> on
> > >>>> some
> > >>>>>>> of
> > >>>>>>> the early SAMZA-390 tickets. (SAMZA-482, SAMZA-483, SAMZA-484)
> > >>>>>>>
> > >>>>>>> On Mon, Feb 9, 2015 at 8:35 AM, Yan Fang <ya...@gmail.com>
> > >>>> wrote:
> > >>>>>>>
> > >>>>>>>> +1 on this.
> > >>>>>>>>
> > >>>>>>>> Fang, Yan
> > >>>>>>>> yanfang724@gmail.com
> > >>>>>>>> +1 (206) 849-4108
> > >>>>>>>>
> > >>>>>>>> On Fri, Feb 6, 2015 at 4:38 PM, Chris Riccomini <
> > >>>>> criccomini@apache.org>
> > >>>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>>> Hey all,
> > >>>>>>>>>
> > >>>>>>>>> Are we +1 on this? I think Jakob was the only one who was
> > >>> curious
> > >>>>>>> about
> > >>>>>>>> it.
> > >>>>>>>>>
> > >>>>>>>>> Cheers,
> > >>>>>>>>> Chris
> > >>>>>>>>>
> > >>>>>>>>> On Thu, Feb 5, 2015 at 1:22 PM, Yi Pan <ni...@gmail.com>
> > >>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> Hi, Jakob,
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>>> Eh? Not sure what this means...
> > >>>>>>>>>>>
> > >>>>>>>>>>> I mean SAMZA-484 depends on SAMZA-482, and neither are
> > >>>>> committed.
> > >>>>>>> So
> > >>>>>>>>>> Navina
> > >>>>>>>>>>> is having to post Yi's patch, as well as her own, on the
> > >>> JIRA.
> > >>>>> It
> > >>>>>>>> makes
> > >>>>>>>>>> it
> > >>>>>>>>>>> really hard to do code reviews because you can't tell
> > >>> whether
> > >>>> Yi
> > >>>>>>> made
> > >>>>>>>>> the
> > >>>>>>>>>>> changes or Navina did.
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>> Just to add to the point. It is also difficult to always see
> > >>> a
> > >>>>> long
> > >>>>>>>> list
> > >>>>>>>>> of
> > >>>>>>>>>> changed files if the RB request is always based on the
> > >>> master.
> > >>>> It
> > >>>>> is
> > >>>>>>>>>> possible to have RB request based on another RB request (I
> > >>> have
> > >>>>>>> tried
> > >>>>>>>> it
> > >>>>>>>>>> before). But what happens if the base RB request is
> > >>>>>>>> cancelled/discarded?
> > >>>>>>>>> RB
> > >>>>>>>>>> is not designed to track the revision changes in a dependency
> > >>>>> chain.
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>> Cheers,
> > >>>>>>>>>>> Chris
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Thu, Feb 5, 2015 at 12:16 PM, Jakob Homan <
> > >>>> jghoman@gmail.com
> > >>>>>>
> > >>>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>>> I want to avoid branches,
> > >>>>>>>>>>>> Just curious, any reason for this?
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> and I also want to avoid revision control over JIRA
> > >>>>>>>>>>>> Eh? Not sure what this means...
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Thanks,
> > >>>>>>>>>>>> jg
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On 4 February 2015 at 17:11, Chris Riccomini <
> > >>>>>>>> criccomini@apache.org>
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>> Hey all,
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> @Jakob, yeah I was thinking we'll follow our normal
> > >>> flow.
> > >>>>>>> RTC. I
> > >>>>>>>>> just
> > >>>>>>>>>>>>> wanted to set expectation that the code committed
> > >>> might be
> > >>>>>>> not up
> > >>>>>>>>> to
> > >>>>>>>>>>> our
> > >>>>>>>>>>>>> normal quality initially (missing docs, no tests, etc).
> > >>>>> Until
> > >>>>>>> the
> > >>>>>>>>>>> quality
> > >>>>>>>>>>>>> is raised, we should think of this module as
> > >>> experimental.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> @Milinda, awesome! Thanks. :)
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Cheers,
> > >>>>>>>>>>>>> Chris
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On Wed, Feb 4, 2015 at 11:57 AM, Milinda Pathirage <
> > >>>>>>>>>>>> mpathira@umail.iu.edu>
> > >>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Hi Chris,
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Hope we no longer need the SQL API. I'll create a RB
> > >>> for
> > >>>>>>> Calcite
> > >>>>>>>>>>>>>> integration.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Thanks
> > >>>>>>>>>>>>>> Milinda
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 1:31 PM, Chris Riccomini <
> > >>>>>>>>>>> criccomini@apache.org>
> > >>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> I think so. There was some RB downtime, but it just
> > >>> got
> > >>>>>>> fixed.
> > >>>>>>>>> Yi,
> > >>>>>>>>>>>>>> Navina,
> > >>>>>>>>>>>>>>> Milinda, can you make sure your JIRAs have up to
> > >>> date
> > >>>>> RBs?
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 10:24 AM, sriram <
> > >>>>>>> sriram.sub@gmail.com
> > >>>>>>>>>
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Can we have updated RBs for all the three sub
> > >>> tasks
> > >>>>>>> before
> > >>>>>>>> we
> > >>>>>>>>>>>> commit?
> > >>>>>>>>>>>>>>> This
> > >>>>>>>>>>>>>>>> would help us to review even after we commit.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini <
> > >>>>>>>>>>>>>> criccomini@apache.org>
> > >>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Hey all,
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Yi, Navina, and Milinda have been working on
> > >>>>> SAMZA-390
> > >>>>>>>>>>> sub-tickets
> > >>>>>>>>>>>>>>>> related
> > >>>>>>>>>>>>>>>>> to SQL operators. We're getting to the point
> > >>> where
> > >>>>> the
> > >>>>>>>>> amount
> > >>>>>>>>>> of
> > >>>>>>>>>>>> work
> > >>>>>>>>>>>>>>>>> floating around is quite large, and some tickets
> > >>>>> build
> > >>>>>>> off
> > >>>>>>>>> of
> > >>>>>>>>>>>> others.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> I'm proposing that we commit this work into a
> > >>>>> samza-sql
> > >>>>>>>>>>> submodule
> > >>>>>>>>>>>> on
> > >>>>>>>>>>>>>>>>> master, and treat this module as experimental. I
> > >>>> want
> > >>>>>>> to
> > >>>>>>>>> avoid
> > >>>>>>>>>>>>>>> branches,
> > >>>>>>>>>>>>>>>>> and I also want to avoid revision control over
> > >>>> JIRA.
> > >>>>>>> This
> > >>>>>>>>>> means
> > >>>>>>>>>>>> that
> > >>>>>>>>>>>>>>>> there
> > >>>>>>>>>>>>>>>>> will probably be a fair amount of commits/JIRAs
> > >>> on
> > >>>>> this
> > >>>>>>>>> module
> > >>>>>>>>>>> as
> > >>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>>> iterate, but I think that's OK.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Does this sound good to everyone?
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Cheers,
> > >>>>>>>>>>>>>>>>> Chris
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>> Milinda Pathirage
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> PhD Student | Research Assistant
> > >>>>>>>>>>>>>> School of Informatics and Computing | Data to Insight
> > >>>>> Center
> > >>>>>>>>>>>>>> Indiana University
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> twitter: milindalakmal
> > >>>>>>>>>>>>>> skype: milinda.pathirage
> > >>>>>>>>>>>>>> blog: http://milinda.pathirage.org
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> --
> > >>>>>> Milinda Pathirage
> > >>>>>>
> > >>>>>> PhD Student | Research Assistant
> > >>>>>> School of Informatics and Computing | Data to Insight Center
> > >>>>>> Indiana University
> > >>>>>>
> > >>>>>> twitter: milindalakmal
> > >>>>>> skype: milinda.pathirage
> > >>>>>> blog: http://milinda.pathirage.org
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> --
> > >>>>> Milinda Pathirage
> > >>>>>
> > >>>>> PhD Student | Research Assistant
> > >>>>> School of Informatics and Computing | Data to Insight Center
> > >>>>> Indiana University
> > >>>>>
> > >>>>> twitter: milindalakmal
> > >>>>> skype: milinda.pathirage
> > >>>>> blog: http://milinda.pathirage.org
> > >>>>>
> > >>>>
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>> Milinda Pathirage
> > >>>
> > >>> PhD Student | Research Assistant
> > >>> School of Informatics and Computing | Data to Insight Center
> > >>> Indiana University
> > >>>
> > >>> twitter: milindalakmal
> > >>> skype: milinda.pathirage
> > >>> blog: http://milinda.pathirage.org
> > >>>
> > >>
> > >>
> >
> >
>
--
Milinda Pathirage
PhD Student | Research Assistant
School of Informatics and Computing | Data to Insight Center
Indiana University
twitter: milindalakmal
skype: milinda.pathirage
blog: http://milinda.pathirage.org
Re: [DISCUSS] SQL workflow
Posted by Chris Riccomini <cr...@apache.org>.
Hey Julian,
> The rule of thumb I’ve learned integrating Calcite into Hive is that only
a branch should depend on snapshot versions of another component
This is very practical. I hadn't considered the Calcite dependency issue
when I pushed for merging into master rather than a branch.
Based on this, it seems like the best thing to do is:
1) Work off of Calcite SNAPSHOT Maven publication.
2) Create a samza-sql branch.
3) Migrate all existing samza-sql commits from master into the samza-sql
branch.
Sorry for the churn on this all. I hadn't considered the binary dependency
issue. Is everyone OK with this?
Cheers,
Chris
On Wed, Feb 11, 2015 at 10:51 AM, Julian Hyde <ju...@hydromatic.net> wrote:
> This seems more like a branch than a classifier. Calcite is developing in
> a branch, and would produce snapshots from that branch. The rule of thumb
> I’ve learned integrating Calcite into Hive is that only a branch should
> depend on snapshot versions of another component. (Hive broke this rule and
> got into a sticky situation where their trunk depended on a Calcite
> snapshot and they couldn’t release because Calcite was stuck in pre-release
> vote limbo.)
>
> I’m going to try to produce maven artifacts with {groupId:
> “org.apache.calcite”, version: "1.1.0-stream-incubating-SNAPSHOT”,
> artifactId: “calcite-core”, “calcite-avatica”, etc.} and push them to
> Apache nexus. I do recommend that you use it on a branch.
>
> However, I don’t think there would be a problem getting the functionality
> into calcite’s master branch in time for Calcite’s next release, in about 4
> weeks. (We commit to releasing approximately monthly.) The functionality
> and APIs would be marked “experimental, subject to change” but at least you
> would be able to merge your work to master at that point. Doubtless you
> would need new features & bug fixes in Calcite after that, and so work that
> depended on that would have to stay in a branch until Calcite was released.
>
> Julian
>
> On Feb 11, 2015, at 10:34 AM, Felix GV <fv...@linkedin.com.INVALID>
> wrote:
>
> > I guess one concern with Calcite SNAPSHOT releases is that they imply
> they'll be included in the next release, which may not be the case if
> they're still considered very experimental.
> >
> > Alternatively, perhaps Julian can create a new version name with a
> classifier for streaming.
> >
> >
> http://maven.apache.org/plugins/maven-deploy-plugin/examples/deploying-with-classifiers.html
> >
> > Then he could release SNAPSHOT artifacts for both the main/master code
> base as well as the streaming branch, no matter how experimental it is...
> >
> > --
> >
> > Felix GV
> > Data Infrastructure Engineer
> > Distributed Data Systems
> > LinkedIn
> >
> > fgv@linkedin.com
> > linkedin.com/in/felixgv
> >
> > ________________________________________
> > From: Chris Riccomini [criccomini@apache.org]
> > Sent: Wednesday, February 11, 2015 10:23 AM
> > To: Chris Riccomini
> > Cc: dev@samza.apache.org
> > Subject: Re: [DISCUSS] SQL workflow
> >
> > Hey Julian,
> >
> > It looks like Calcite is at least setup to publish to Maven:
> >
> > https://repository.apache.org/#nexus-search;quick~calcite
> >
> > Julian, can you publish SNAPSHOT releases with your streaming changes to
> it?
> >
> > Thanks!
> > Chris
> >
> > On Wed, Feb 11, 2015 at 10:08 AM, Chris Riccomini <criccomini@apache.org
> >
> > wrote:
> >
> >> Hey Milinda,
> >>
> >> I've just committed SAMZA-484, so you should be able to re-base and get
> >> all the latest code.
> >>
> >>> But we need to add Calcite as a dependency and Calcite streaming
> >> support is only in Julian's branch
> >>
> >> We'll have to have Julian publish a release to Apache. This can be a
> >> SNAPSHOT to Apache's SNAPSHOT repo, or a real Calcite release. We can't
> >> check in binaries to the repo (licensing, release complexity, and just
> >> generally a bad idea). We'll have to coordinate with him.
> >>
> >> Cheers,
> >> Chris
> >>
> >> On Wed, Feb 11, 2015 at 9:43 AM, Milinda Pathirage <
> mpathira@umail.iu.edu>
> >> wrote:
> >>
> >>> No, SAMZA-483 doesn't depend on SAMZA-484. But we need to add Calcite
> as a
> >>> dependency and Calcite streaming support is only in Julian's branch (
> >>> https://github.com/julianhyde/incubator-calcite/tree/chi). What can
> we do
> >>> about that?
> >>>
> >>> Milinda
> >>>
> >>> On Wed, Feb 11, 2015 at 12:30 PM, Chris Riccomini <
> criccomini@apache.org>
> >>> wrote:
> >>>
> >>>> Hey Milinda,
> >>>>
> >>>> That'd be great. Do you have any dependency on SAMZA-484? I haven't
> had
> >>> a
> >>>> chance to commit that one yet.
> >>>>
> >>>> Cheers,
> >>>> Chris
> >>>>
> >>>> On Wed, Feb 11, 2015 at 9:23 AM, Milinda Pathirage <
> >>> mpathira@umail.iu.edu>
> >>>> wrote:
> >>>>
> >>>>> /need/want
> >>>>>
> >>>>> Milinda
> >>>>>
> >>>>> On Wed, Feb 11, 2015 at 12:22 PM, Milinda Pathirage <
> >>>> mpathira@umail.iu.edu
> >>>>>>
> >>>>> wrote:
> >>>>>
> >>>>>> Hi Chris,
> >>>>>>
> >>>>>> Do you need me to create the SAMZA-483 patch against latest master
> >>> with
> >>>>>> SAMZA-482 patch? I think that will make it easier to review the
> >>> patch?
> >>>>>>
> >>>>>> Thanks
> >>>>>> Milinda
> >>>>>>
> >>>>>> On Mon, Feb 9, 2015 at 11:39 AM, Chris Riccomini <
> >>>> criccomini@apache.org>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Cool. Looks like we've got consensus. I'll move forward with RTC on
> >>>> some
> >>>>>>> of
> >>>>>>> the early SAMZA-390 tickets. (SAMZA-482, SAMZA-483, SAMZA-484)
> >>>>>>>
> >>>>>>> On Mon, Feb 9, 2015 at 8:35 AM, Yan Fang <ya...@gmail.com>
> >>>> wrote:
> >>>>>>>
> >>>>>>>> +1 on this.
> >>>>>>>>
> >>>>>>>> Fang, Yan
> >>>>>>>> yanfang724@gmail.com
> >>>>>>>> +1 (206) 849-4108
> >>>>>>>>
> >>>>>>>> On Fri, Feb 6, 2015 at 4:38 PM, Chris Riccomini <
> >>>>> criccomini@apache.org>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Hey all,
> >>>>>>>>>
> >>>>>>>>> Are we +1 on this? I think Jakob was the only one who was
> >>> curious
> >>>>>>> about
> >>>>>>>> it.
> >>>>>>>>>
> >>>>>>>>> Cheers,
> >>>>>>>>> Chris
> >>>>>>>>>
> >>>>>>>>> On Thu, Feb 5, 2015 at 1:22 PM, Yi Pan <ni...@gmail.com>
> >>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Hi, Jakob,
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>> Eh? Not sure what this means...
> >>>>>>>>>>>
> >>>>>>>>>>> I mean SAMZA-484 depends on SAMZA-482, and neither are
> >>>>> committed.
> >>>>>>> So
> >>>>>>>>>> Navina
> >>>>>>>>>>> is having to post Yi's patch, as well as her own, on the
> >>> JIRA.
> >>>>> It
> >>>>>>>> makes
> >>>>>>>>>> it
> >>>>>>>>>>> really hard to do code reviews because you can't tell
> >>> whether
> >>>> Yi
> >>>>>>> made
> >>>>>>>>> the
> >>>>>>>>>>> changes or Navina did.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>> Just to add to the point. It is also difficult to always see
> >>> a
> >>>>> long
> >>>>>>>> list
> >>>>>>>>> of
> >>>>>>>>>> changed files if the RB request is always based on the
> >>> master.
> >>>> It
> >>>>> is
> >>>>>>>>>> possible to have RB request based on another RB request (I
> >>> have
> >>>>>>> tried
> >>>>>>>> it
> >>>>>>>>>> before). But what happens if the base RB request is
> >>>>>>>> cancelled/discarded?
> >>>>>>>>> RB
> >>>>>>>>>> is not designed to track the revision changes in a dependency
> >>>>> chain.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> Cheers,
> >>>>>>>>>>> Chris
> >>>>>>>>>>>
> >>>>>>>>>>> On Thu, Feb 5, 2015 at 12:16 PM, Jakob Homan <
> >>>> jghoman@gmail.com
> >>>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>>> I want to avoid branches,
> >>>>>>>>>>>> Just curious, any reason for this?
> >>>>>>>>>>>>
> >>>>>>>>>>>>> and I also want to avoid revision control over JIRA
> >>>>>>>>>>>> Eh? Not sure what this means...
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks,
> >>>>>>>>>>>> jg
> >>>>>>>>>>>>
> >>>>>>>>>>>> On 4 February 2015 at 17:11, Chris Riccomini <
> >>>>>>>> criccomini@apache.org>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>> Hey all,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> @Jakob, yeah I was thinking we'll follow our normal
> >>> flow.
> >>>>>>> RTC. I
> >>>>>>>>> just
> >>>>>>>>>>>>> wanted to set expectation that the code committed
> >>> might be
> >>>>>>> not up
> >>>>>>>>> to
> >>>>>>>>>>> our
> >>>>>>>>>>>>> normal quality initially (missing docs, no tests, etc).
> >>>>> Until
> >>>>>>> the
> >>>>>>>>>>> quality
> >>>>>>>>>>>>> is raised, we should think of this module as
> >>> experimental.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> @Milinda, awesome! Thanks. :)
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Cheers,
> >>>>>>>>>>>>> Chris
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Wed, Feb 4, 2015 at 11:57 AM, Milinda Pathirage <
> >>>>>>>>>>>> mpathira@umail.iu.edu>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hi Chris,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Hope we no longer need the SQL API. I'll create a RB
> >>> for
> >>>>>>> Calcite
> >>>>>>>>>>>>>> integration.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks
> >>>>>>>>>>>>>> Milinda
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 1:31 PM, Chris Riccomini <
> >>>>>>>>>>> criccomini@apache.org>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I think so. There was some RB downtime, but it just
> >>> got
> >>>>>>> fixed.
> >>>>>>>>> Yi,
> >>>>>>>>>>>>>> Navina,
> >>>>>>>>>>>>>>> Milinda, can you make sure your JIRAs have up to
> >>> date
> >>>>> RBs?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 10:24 AM, sriram <
> >>>>>>> sriram.sub@gmail.com
> >>>>>>>>>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Can we have updated RBs for all the three sub
> >>> tasks
> >>>>>>> before
> >>>>>>>> we
> >>>>>>>>>>>> commit?
> >>>>>>>>>>>>>>> This
> >>>>>>>>>>>>>>>> would help us to review even after we commit.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini <
> >>>>>>>>>>>>>> criccomini@apache.org>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Hey all,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Yi, Navina, and Milinda have been working on
> >>>>> SAMZA-390
> >>>>>>>>>>> sub-tickets
> >>>>>>>>>>>>>>>> related
> >>>>>>>>>>>>>>>>> to SQL operators. We're getting to the point
> >>> where
> >>>>> the
> >>>>>>>>> amount
> >>>>>>>>>> of
> >>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>> floating around is quite large, and some tickets
> >>>>> build
> >>>>>>> off
> >>>>>>>>> of
> >>>>>>>>>>>> others.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I'm proposing that we commit this work into a
> >>>>> samza-sql
> >>>>>>>>>>> submodule
> >>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>> master, and treat this module as experimental. I
> >>>> want
> >>>>>>> to
> >>>>>>>>> avoid
> >>>>>>>>>>>>>>> branches,
> >>>>>>>>>>>>>>>>> and I also want to avoid revision control over
> >>>> JIRA.
> >>>>>>> This
> >>>>>>>>>> means
> >>>>>>>>>>>> that
> >>>>>>>>>>>>>>>> there
> >>>>>>>>>>>>>>>>> will probably be a fair amount of commits/JIRAs
> >>> on
> >>>>> this
> >>>>>>>>> module
> >>>>>>>>>>> as
> >>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>> iterate, but I think that's OK.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Does this sound good to everyone?
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Cheers,
> >>>>>>>>>>>>>>>>> Chris
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> --
> >>>>>>>>>>>>>> Milinda Pathirage
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> PhD Student | Research Assistant
> >>>>>>>>>>>>>> School of Informatics and Computing | Data to Insight
> >>>>> Center
> >>>>>>>>>>>>>> Indiana University
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> twitter: milindalakmal
> >>>>>>>>>>>>>> skype: milinda.pathirage
> >>>>>>>>>>>>>> blog: http://milinda.pathirage.org
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Milinda Pathirage
> >>>>>>
> >>>>>> PhD Student | Research Assistant
> >>>>>> School of Informatics and Computing | Data to Insight Center
> >>>>>> Indiana University
> >>>>>>
> >>>>>> twitter: milindalakmal
> >>>>>> skype: milinda.pathirage
> >>>>>> blog: http://milinda.pathirage.org
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Milinda Pathirage
> >>>>>
> >>>>> PhD Student | Research Assistant
> >>>>> School of Informatics and Computing | Data to Insight Center
> >>>>> Indiana University
> >>>>>
> >>>>> twitter: milindalakmal
> >>>>> skype: milinda.pathirage
> >>>>> blog: http://milinda.pathirage.org
> >>>>>
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Milinda Pathirage
> >>>
> >>> PhD Student | Research Assistant
> >>> School of Informatics and Computing | Data to Insight Center
> >>> Indiana University
> >>>
> >>> twitter: milindalakmal
> >>> skype: milinda.pathirage
> >>> blog: http://milinda.pathirage.org
> >>>
> >>
> >>
>
>
Re: [DISCUSS] SQL workflow
Posted by Julian Hyde <ju...@hydromatic.net>.
This seems more like a branch than a classifier. Calcite is developing in a branch, and would produce snapshots from that branch. The rule of thumb I’ve learned integrating Calcite into Hive is that only a branch should depend on snapshot versions of another component. (Hive broke this rule and got into a sticky situation where their trunk depended on a Calcite snapshot and they couldn’t release because Calcite was stuck in pre-release vote limbo.)
I’m going to try to produce maven artifacts with {groupId: “org.apache.calcite”, version: "1.1.0-stream-incubating-SNAPSHOT”, artifactId: “calcite-core”, “calcite-avatica”, etc.} and push them to Apache nexus. I do recommend that you use it on a branch.
However, I don’t think there would be a problem getting the functionality into calcite’s master branch in time for Calcite’s next release, in about 4 weeks. (We commit to releasing approximately monthly.) The functionality and APIs would be marked “experimental, subject to change” but at least you would be able to merge your work to master at that point. Doubtless you would need new features & bug fixes in Calcite after that, and so work that depended on that would have to stay in a branch until Calcite was released.
Julian
On Feb 11, 2015, at 10:34 AM, Felix GV <fv...@linkedin.com.INVALID> wrote:
> I guess one concern with Calcite SNAPSHOT releases is that they imply they'll be included in the next release, which may not be the case if they're still considered very experimental.
>
> Alternatively, perhaps Julian can create a new version name with a classifier for streaming.
>
> http://maven.apache.org/plugins/maven-deploy-plugin/examples/deploying-with-classifiers.html
>
> Then he could release SNAPSHOT artifacts for both the main/master code base as well as the streaming branch, no matter how experimental it is...
>
> --
>
> Felix GV
> Data Infrastructure Engineer
> Distributed Data Systems
> LinkedIn
>
> fgv@linkedin.com
> linkedin.com/in/felixgv
>
> ________________________________________
> From: Chris Riccomini [criccomini@apache.org]
> Sent: Wednesday, February 11, 2015 10:23 AM
> To: Chris Riccomini
> Cc: dev@samza.apache.org
> Subject: Re: [DISCUSS] SQL workflow
>
> Hey Julian,
>
> It looks like Calcite is at least setup to publish to Maven:
>
> https://repository.apache.org/#nexus-search;quick~calcite
>
> Julian, can you publish SNAPSHOT releases with your streaming changes to it?
>
> Thanks!
> Chris
>
> On Wed, Feb 11, 2015 at 10:08 AM, Chris Riccomini <cr...@apache.org>
> wrote:
>
>> Hey Milinda,
>>
>> I've just committed SAMZA-484, so you should be able to re-base and get
>> all the latest code.
>>
>>> But we need to add Calcite as a dependency and Calcite streaming
>> support is only in Julian's branch
>>
>> We'll have to have Julian publish a release to Apache. This can be a
>> SNAPSHOT to Apache's SNAPSHOT repo, or a real Calcite release. We can't
>> check in binaries to the repo (licensing, release complexity, and just
>> generally a bad idea). We'll have to coordinate with him.
>>
>> Cheers,
>> Chris
>>
>> On Wed, Feb 11, 2015 at 9:43 AM, Milinda Pathirage <mp...@umail.iu.edu>
>> wrote:
>>
>>> No, SAMZA-483 doesn't depend on SAMZA-484. But we need to add Calcite as a
>>> dependency and Calcite streaming support is only in Julian's branch (
>>> https://github.com/julianhyde/incubator-calcite/tree/chi). What can we do
>>> about that?
>>>
>>> Milinda
>>>
>>> On Wed, Feb 11, 2015 at 12:30 PM, Chris Riccomini <cr...@apache.org>
>>> wrote:
>>>
>>>> Hey Milinda,
>>>>
>>>> That'd be great. Do you have any dependency on SAMZA-484? I haven't had
>>> a
>>>> chance to commit that one yet.
>>>>
>>>> Cheers,
>>>> Chris
>>>>
>>>> On Wed, Feb 11, 2015 at 9:23 AM, Milinda Pathirage <
>>> mpathira@umail.iu.edu>
>>>> wrote:
>>>>
>>>>> /need/want
>>>>>
>>>>> Milinda
>>>>>
>>>>> On Wed, Feb 11, 2015 at 12:22 PM, Milinda Pathirage <
>>>> mpathira@umail.iu.edu
>>>>>>
>>>>> wrote:
>>>>>
>>>>>> Hi Chris,
>>>>>>
>>>>>> Do you need me to create the SAMZA-483 patch against latest master
>>> with
>>>>>> SAMZA-482 patch? I think that will make it easier to review the
>>> patch?
>>>>>>
>>>>>> Thanks
>>>>>> Milinda
>>>>>>
>>>>>> On Mon, Feb 9, 2015 at 11:39 AM, Chris Riccomini <
>>>> criccomini@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Cool. Looks like we've got consensus. I'll move forward with RTC on
>>>> some
>>>>>>> of
>>>>>>> the early SAMZA-390 tickets. (SAMZA-482, SAMZA-483, SAMZA-484)
>>>>>>>
>>>>>>> On Mon, Feb 9, 2015 at 8:35 AM, Yan Fang <ya...@gmail.com>
>>>> wrote:
>>>>>>>
>>>>>>>> +1 on this.
>>>>>>>>
>>>>>>>> Fang, Yan
>>>>>>>> yanfang724@gmail.com
>>>>>>>> +1 (206) 849-4108
>>>>>>>>
>>>>>>>> On Fri, Feb 6, 2015 at 4:38 PM, Chris Riccomini <
>>>>> criccomini@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hey all,
>>>>>>>>>
>>>>>>>>> Are we +1 on this? I think Jakob was the only one who was
>>> curious
>>>>>>> about
>>>>>>>> it.
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>> Chris
>>>>>>>>>
>>>>>>>>> On Thu, Feb 5, 2015 at 1:22 PM, Yi Pan <ni...@gmail.com>
>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi, Jakob,
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>> Eh? Not sure what this means...
>>>>>>>>>>>
>>>>>>>>>>> I mean SAMZA-484 depends on SAMZA-482, and neither are
>>>>> committed.
>>>>>>> So
>>>>>>>>>> Navina
>>>>>>>>>>> is having to post Yi's patch, as well as her own, on the
>>> JIRA.
>>>>> It
>>>>>>>> makes
>>>>>>>>>> it
>>>>>>>>>>> really hard to do code reviews because you can't tell
>>> whether
>>>> Yi
>>>>>>> made
>>>>>>>>> the
>>>>>>>>>>> changes or Navina did.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> Just to add to the point. It is also difficult to always see
>>> a
>>>>> long
>>>>>>>> list
>>>>>>>>> of
>>>>>>>>>> changed files if the RB request is always based on the
>>> master.
>>>> It
>>>>> is
>>>>>>>>>> possible to have RB request based on another RB request (I
>>> have
>>>>>>> tried
>>>>>>>> it
>>>>>>>>>> before). But what happens if the base RB request is
>>>>>>>> cancelled/discarded?
>>>>>>>>> RB
>>>>>>>>>> is not designed to track the revision changes in a dependency
>>>>> chain.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Cheers,
>>>>>>>>>>> Chris
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Feb 5, 2015 at 12:16 PM, Jakob Homan <
>>>> jghoman@gmail.com
>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>>> I want to avoid branches,
>>>>>>>>>>>> Just curious, any reason for this?
>>>>>>>>>>>>
>>>>>>>>>>>>> and I also want to avoid revision control over JIRA
>>>>>>>>>>>> Eh? Not sure what this means...
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> jg
>>>>>>>>>>>>
>>>>>>>>>>>> On 4 February 2015 at 17:11, Chris Riccomini <
>>>>>>>> criccomini@apache.org>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> Hey all,
>>>>>>>>>>>>>
>>>>>>>>>>>>> @Jakob, yeah I was thinking we'll follow our normal
>>> flow.
>>>>>>> RTC. I
>>>>>>>>> just
>>>>>>>>>>>>> wanted to set expectation that the code committed
>>> might be
>>>>>>> not up
>>>>>>>>> to
>>>>>>>>>>> our
>>>>>>>>>>>>> normal quality initially (missing docs, no tests, etc).
>>>>> Until
>>>>>>> the
>>>>>>>>>>> quality
>>>>>>>>>>>>> is raised, we should think of this module as
>>> experimental.
>>>>>>>>>>>>>
>>>>>>>>>>>>> @Milinda, awesome! Thanks. :)
>>>>>>>>>>>>>
>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>> Chris
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 11:57 AM, Milinda Pathirage <
>>>>>>>>>>>> mpathira@umail.iu.edu>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Chris,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hope we no longer need the SQL API. I'll create a RB
>>> for
>>>>>>> Calcite
>>>>>>>>>>>>>> integration.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>> Milinda
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 1:31 PM, Chris Riccomini <
>>>>>>>>>>> criccomini@apache.org>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I think so. There was some RB downtime, but it just
>>> got
>>>>>>> fixed.
>>>>>>>>> Yi,
>>>>>>>>>>>>>> Navina,
>>>>>>>>>>>>>>> Milinda, can you make sure your JIRAs have up to
>>> date
>>>>> RBs?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 10:24 AM, sriram <
>>>>>>> sriram.sub@gmail.com
>>>>>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Can we have updated RBs for all the three sub
>>> tasks
>>>>>>> before
>>>>>>>> we
>>>>>>>>>>>> commit?
>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>> would help us to review even after we commit.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini <
>>>>>>>>>>>>>> criccomini@apache.org>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hey all,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Yi, Navina, and Milinda have been working on
>>>>> SAMZA-390
>>>>>>>>>>> sub-tickets
>>>>>>>>>>>>>>>> related
>>>>>>>>>>>>>>>>> to SQL operators. We're getting to the point
>>> where
>>>>> the
>>>>>>>>> amount
>>>>>>>>>> of
>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>> floating around is quite large, and some tickets
>>>>> build
>>>>>>> off
>>>>>>>>> of
>>>>>>>>>>>> others.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I'm proposing that we commit this work into a
>>>>> samza-sql
>>>>>>>>>>> submodule
>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>> master, and treat this module as experimental. I
>>>> want
>>>>>>> to
>>>>>>>>> avoid
>>>>>>>>>>>>>>> branches,
>>>>>>>>>>>>>>>>> and I also want to avoid revision control over
>>>> JIRA.
>>>>>>> This
>>>>>>>>>> means
>>>>>>>>>>>> that
>>>>>>>>>>>>>>>> there
>>>>>>>>>>>>>>>>> will probably be a fair amount of commits/JIRAs
>>> on
>>>>> this
>>>>>>>>> module
>>>>>>>>>>> as
>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>> iterate, but I think that's OK.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Does this sound good to everyone?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>>>>> Chris
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Milinda Pathirage
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> PhD Student | Research Assistant
>>>>>>>>>>>>>> School of Informatics and Computing | Data to Insight
>>>>> Center
>>>>>>>>>>>>>> Indiana University
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> twitter: milindalakmal
>>>>>>>>>>>>>> skype: milinda.pathirage
>>>>>>>>>>>>>> blog: http://milinda.pathirage.org
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Milinda Pathirage
>>>>>>
>>>>>> PhD Student | Research Assistant
>>>>>> School of Informatics and Computing | Data to Insight Center
>>>>>> Indiana University
>>>>>>
>>>>>> twitter: milindalakmal
>>>>>> skype: milinda.pathirage
>>>>>> blog: http://milinda.pathirage.org
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Milinda Pathirage
>>>>>
>>>>> PhD Student | Research Assistant
>>>>> School of Informatics and Computing | Data to Insight Center
>>>>> Indiana University
>>>>>
>>>>> twitter: milindalakmal
>>>>> skype: milinda.pathirage
>>>>> blog: http://milinda.pathirage.org
>>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Milinda Pathirage
>>>
>>> PhD Student | Research Assistant
>>> School of Informatics and Computing | Data to Insight Center
>>> Indiana University
>>>
>>> twitter: milindalakmal
>>> skype: milinda.pathirage
>>> blog: http://milinda.pathirage.org
>>>
>>
>>
RE: [DISCUSS] SQL workflow
Posted by Felix GV <fv...@linkedin.com.INVALID>.
I guess one concern with Calcite SNAPSHOT releases is that they imply they'll be included in the next release, which may not be the case if they're still considered very experimental.
Alternatively, perhaps Julian can create a new version name with a classifier for streaming.
http://maven.apache.org/plugins/maven-deploy-plugin/examples/deploying-with-classifiers.html
Then he could release SNAPSHOT artifacts for both the main/master code base as well as the streaming branch, no matter how experimental it is...
--
Felix GV
Data Infrastructure Engineer
Distributed Data Systems
LinkedIn
fgv@linkedin.com
linkedin.com/in/felixgv
________________________________________
From: Chris Riccomini [criccomini@apache.org]
Sent: Wednesday, February 11, 2015 10:23 AM
To: Chris Riccomini
Cc: dev@samza.apache.org
Subject: Re: [DISCUSS] SQL workflow
Hey Julian,
It looks like Calcite is at least setup to publish to Maven:
https://repository.apache.org/#nexus-search;quick~calcite
Julian, can you publish SNAPSHOT releases with your streaming changes to it?
Thanks!
Chris
On Wed, Feb 11, 2015 at 10:08 AM, Chris Riccomini <cr...@apache.org>
wrote:
> Hey Milinda,
>
> I've just committed SAMZA-484, so you should be able to re-base and get
> all the latest code.
>
> > But we need to add Calcite as a dependency and Calcite streaming
> support is only in Julian's branch
>
> We'll have to have Julian publish a release to Apache. This can be a
> SNAPSHOT to Apache's SNAPSHOT repo, or a real Calcite release. We can't
> check in binaries to the repo (licensing, release complexity, and just
> generally a bad idea). We'll have to coordinate with him.
>
> Cheers,
> Chris
>
> On Wed, Feb 11, 2015 at 9:43 AM, Milinda Pathirage <mp...@umail.iu.edu>
> wrote:
>
>> No, SAMZA-483 doesn't depend on SAMZA-484. But we need to add Calcite as a
>> dependency and Calcite streaming support is only in Julian's branch (
>> https://github.com/julianhyde/incubator-calcite/tree/chi). What can we do
>> about that?
>>
>> Milinda
>>
>> On Wed, Feb 11, 2015 at 12:30 PM, Chris Riccomini <cr...@apache.org>
>> wrote:
>>
>> > Hey Milinda,
>> >
>> > That'd be great. Do you have any dependency on SAMZA-484? I haven't had
>> a
>> > chance to commit that one yet.
>> >
>> > Cheers,
>> > Chris
>> >
>> > On Wed, Feb 11, 2015 at 9:23 AM, Milinda Pathirage <
>> mpathira@umail.iu.edu>
>> > wrote:
>> >
>> > > /need/want
>> > >
>> > > Milinda
>> > >
>> > > On Wed, Feb 11, 2015 at 12:22 PM, Milinda Pathirage <
>> > mpathira@umail.iu.edu
>> > > >
>> > > wrote:
>> > >
>> > > > Hi Chris,
>> > > >
>> > > > Do you need me to create the SAMZA-483 patch against latest master
>> with
>> > > > SAMZA-482 patch? I think that will make it easier to review the
>> patch?
>> > > >
>> > > > Thanks
>> > > > Milinda
>> > > >
>> > > > On Mon, Feb 9, 2015 at 11:39 AM, Chris Riccomini <
>> > criccomini@apache.org>
>> > > > wrote:
>> > > >
>> > > >> Cool. Looks like we've got consensus. I'll move forward with RTC on
>> > some
>> > > >> of
>> > > >> the early SAMZA-390 tickets. (SAMZA-482, SAMZA-483, SAMZA-484)
>> > > >>
>> > > >> On Mon, Feb 9, 2015 at 8:35 AM, Yan Fang <ya...@gmail.com>
>> > wrote:
>> > > >>
>> > > >> > +1 on this.
>> > > >> >
>> > > >> > Fang, Yan
>> > > >> > yanfang724@gmail.com
>> > > >> > +1 (206) 849-4108
>> > > >> >
>> > > >> > On Fri, Feb 6, 2015 at 4:38 PM, Chris Riccomini <
>> > > criccomini@apache.org>
>> > > >> > wrote:
>> > > >> >
>> > > >> > > Hey all,
>> > > >> > >
>> > > >> > > Are we +1 on this? I think Jakob was the only one who was
>> curious
>> > > >> about
>> > > >> > it.
>> > > >> > >
>> > > >> > > Cheers,
>> > > >> > > Chris
>> > > >> > >
>> > > >> > > On Thu, Feb 5, 2015 at 1:22 PM, Yi Pan <ni...@gmail.com>
>> > wrote:
>> > > >> > >
>> > > >> > > > Hi, Jakob,
>> > > >> > > >
>> > > >> > > >
>> > > >> > > > > > Eh? Not sure what this means...
>> > > >> > > > >
>> > > >> > > > > I mean SAMZA-484 depends on SAMZA-482, and neither are
>> > > committed.
>> > > >> So
>> > > >> > > > Navina
>> > > >> > > > > is having to post Yi's patch, as well as her own, on the
>> JIRA.
>> > > It
>> > > >> > makes
>> > > >> > > > it
>> > > >> > > > > really hard to do code reviews because you can't tell
>> whether
>> > Yi
>> > > >> made
>> > > >> > > the
>> > > >> > > > > changes or Navina did.
>> > > >> > > > >
>> > > >> > > > >
>> > > >> > > > Just to add to the point. It is also difficult to always see
>> a
>> > > long
>> > > >> > list
>> > > >> > > of
>> > > >> > > > changed files if the RB request is always based on the
>> master.
>> > It
>> > > is
>> > > >> > > > possible to have RB request based on another RB request (I
>> have
>> > > >> tried
>> > > >> > it
>> > > >> > > > before). But what happens if the base RB request is
>> > > >> > cancelled/discarded?
>> > > >> > > RB
>> > > >> > > > is not designed to track the revision changes in a dependency
>> > > chain.
>> > > >> > > >
>> > > >> > > >
>> > > >> > > >
>> > > >> > > > > Cheers,
>> > > >> > > > > Chris
>> > > >> > > > >
>> > > >> > > > > On Thu, Feb 5, 2015 at 12:16 PM, Jakob Homan <
>> > jghoman@gmail.com
>> > > >
>> > > >> > > wrote:
>> > > >> > > > >
>> > > >> > > > > > > I want to avoid branches,
>> > > >> > > > > > Just curious, any reason for this?
>> > > >> > > > > >
>> > > >> > > > > > > and I also want to avoid revision control over JIRA
>> > > >> > > > > > Eh? Not sure what this means...
>> > > >> > > > > >
>> > > >> > > > > > Thanks,
>> > > >> > > > > > jg
>> > > >> > > > > >
>> > > >> > > > > > On 4 February 2015 at 17:11, Chris Riccomini <
>> > > >> > criccomini@apache.org>
>> > > >> > > > > > wrote:
>> > > >> > > > > > > Hey all,
>> > > >> > > > > > >
>> > > >> > > > > > > @Jakob, yeah I was thinking we'll follow our normal
>> flow.
>> > > >> RTC. I
>> > > >> > > just
>> > > >> > > > > > > wanted to set expectation that the code committed
>> might be
>> > > >> not up
>> > > >> > > to
>> > > >> > > > > our
>> > > >> > > > > > > normal quality initially (missing docs, no tests, etc).
>> > > Until
>> > > >> the
>> > > >> > > > > quality
>> > > >> > > > > > > is raised, we should think of this module as
>> experimental.
>> > > >> > > > > > >
>> > > >> > > > > > > @Milinda, awesome! Thanks. :)
>> > > >> > > > > > >
>> > > >> > > > > > > Cheers,
>> > > >> > > > > > > Chris
>> > > >> > > > > > >
>> > > >> > > > > > > On Wed, Feb 4, 2015 at 11:57 AM, Milinda Pathirage <
>> > > >> > > > > > mpathira@umail.iu.edu>
>> > > >> > > > > > > wrote:
>> > > >> > > > > > >
>> > > >> > > > > > >> Hi Chris,
>> > > >> > > > > > >>
>> > > >> > > > > > >> Hope we no longer need the SQL API. I'll create a RB
>> for
>> > > >> Calcite
>> > > >> > > > > > >> integration.
>> > > >> > > > > > >>
>> > > >> > > > > > >> Thanks
>> > > >> > > > > > >> Milinda
>> > > >> > > > > > >>
>> > > >> > > > > > >> On Wed, Feb 4, 2015 at 1:31 PM, Chris Riccomini <
>> > > >> > > > > criccomini@apache.org>
>> > > >> > > > > > >> wrote:
>> > > >> > > > > > >>
>> > > >> > > > > > >> > I think so. There was some RB downtime, but it just
>> got
>> > > >> fixed.
>> > > >> > > Yi,
>> > > >> > > > > > >> Navina,
>> > > >> > > > > > >> > Milinda, can you make sure your JIRAs have up to
>> date
>> > > RBs?
>> > > >> > > > > > >> >
>> > > >> > > > > > >> > On Wed, Feb 4, 2015 at 10:24 AM, sriram <
>> > > >> sriram.sub@gmail.com
>> > > >> > >
>> > > >> > > > > wrote:
>> > > >> > > > > > >> >
>> > > >> > > > > > >> > > Can we have updated RBs for all the three sub
>> tasks
>> > > >> before
>> > > >> > we
>> > > >> > > > > > commit?
>> > > >> > > > > > >> > This
>> > > >> > > > > > >> > > would help us to review even after we commit.
>> > > >> > > > > > >> > >
>> > > >> > > > > > >> > > On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini <
>> > > >> > > > > > >> criccomini@apache.org>
>> > > >> > > > > > >> > > wrote:
>> > > >> > > > > > >> > >
>> > > >> > > > > > >> > > > Hey all,
>> > > >> > > > > > >> > > >
>> > > >> > > > > > >> > > > Yi, Navina, and Milinda have been working on
>> > > SAMZA-390
>> > > >> > > > > sub-tickets
>> > > >> > > > > > >> > > related
>> > > >> > > > > > >> > > > to SQL operators. We're getting to the point
>> where
>> > > the
>> > > >> > > amount
>> > > >> > > > of
>> > > >> > > > > > work
>> > > >> > > > > > >> > > > floating around is quite large, and some tickets
>> > > build
>> > > >> off
>> > > >> > > of
>> > > >> > > > > > others.
>> > > >> > > > > > >> > > >
>> > > >> > > > > > >> > > > I'm proposing that we commit this work into a
>> > > samza-sql
>> > > >> > > > > submodule
>> > > >> > > > > > on
>> > > >> > > > > > >> > > > master, and treat this module as experimental. I
>> > want
>> > > >> to
>> > > >> > > avoid
>> > > >> > > > > > >> > branches,
>> > > >> > > > > > >> > > > and I also want to avoid revision control over
>> > JIRA.
>> > > >> This
>> > > >> > > > means
>> > > >> > > > > > that
>> > > >> > > > > > >> > > there
>> > > >> > > > > > >> > > > will probably be a fair amount of commits/JIRAs
>> on
>> > > this
>> > > >> > > module
>> > > >> > > > > as
>> > > >> > > > > > we
>> > > >> > > > > > >> > > > iterate, but I think that's OK.
>> > > >> > > > > > >> > > >
>> > > >> > > > > > >> > > > Does this sound good to everyone?
>> > > >> > > > > > >> > > >
>> > > >> > > > > > >> > > > Cheers,
>> > > >> > > > > > >> > > > Chris
>> > > >> > > > > > >> > > >
>> > > >> > > > > > >> > >
>> > > >> > > > > > >> >
>> > > >> > > > > > >>
>> > > >> > > > > > >>
>> > > >> > > > > > >>
>> > > >> > > > > > >> --
>> > > >> > > > > > >> Milinda Pathirage
>> > > >> > > > > > >>
>> > > >> > > > > > >> PhD Student | Research Assistant
>> > > >> > > > > > >> School of Informatics and Computing | Data to Insight
>> > > Center
>> > > >> > > > > > >> Indiana University
>> > > >> > > > > > >>
>> > > >> > > > > > >> twitter: milindalakmal
>> > > >> > > > > > >> skype: milinda.pathirage
>> > > >> > > > > > >> blog: http://milinda.pathirage.org
>> > > >> > > > > > >>
>> > > >> > > > > >
>> > > >> > > > >
>> > > >> > > >
>> > > >> > >
>> > > >> >
>> > > >>
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > Milinda Pathirage
>> > > >
>> > > > PhD Student | Research Assistant
>> > > > School of Informatics and Computing | Data to Insight Center
>> > > > Indiana University
>> > > >
>> > > > twitter: milindalakmal
>> > > > skype: milinda.pathirage
>> > > > blog: http://milinda.pathirage.org
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > Milinda Pathirage
>> > >
>> > > PhD Student | Research Assistant
>> > > School of Informatics and Computing | Data to Insight Center
>> > > Indiana University
>> > >
>> > > twitter: milindalakmal
>> > > skype: milinda.pathirage
>> > > blog: http://milinda.pathirage.org
>> > >
>> >
>>
>>
>>
>> --
>> Milinda Pathirage
>>
>> PhD Student | Research Assistant
>> School of Informatics and Computing | Data to Insight Center
>> Indiana University
>>
>> twitter: milindalakmal
>> skype: milinda.pathirage
>> blog: http://milinda.pathirage.org
>>
>
>
Re: [DISCUSS] SQL workflow
Posted by Chris Riccomini <cr...@apache.org>.
Hey Julian,
It looks like Calcite is at least setup to publish to Maven:
https://repository.apache.org/#nexus-search;quick~calcite
Julian, can you publish SNAPSHOT releases with your streaming changes to it?
Thanks!
Chris
On Wed, Feb 11, 2015 at 10:08 AM, Chris Riccomini <cr...@apache.org>
wrote:
> Hey Milinda,
>
> I've just committed SAMZA-484, so you should be able to re-base and get
> all the latest code.
>
> > But we need to add Calcite as a dependency and Calcite streaming
> support is only in Julian's branch
>
> We'll have to have Julian publish a release to Apache. This can be a
> SNAPSHOT to Apache's SNAPSHOT repo, or a real Calcite release. We can't
> check in binaries to the repo (licensing, release complexity, and just
> generally a bad idea). We'll have to coordinate with him.
>
> Cheers,
> Chris
>
> On Wed, Feb 11, 2015 at 9:43 AM, Milinda Pathirage <mp...@umail.iu.edu>
> wrote:
>
>> No, SAMZA-483 doesn't depend on SAMZA-484. But we need to add Calcite as a
>> dependency and Calcite streaming support is only in Julian's branch (
>> https://github.com/julianhyde/incubator-calcite/tree/chi). What can we do
>> about that?
>>
>> Milinda
>>
>> On Wed, Feb 11, 2015 at 12:30 PM, Chris Riccomini <cr...@apache.org>
>> wrote:
>>
>> > Hey Milinda,
>> >
>> > That'd be great. Do you have any dependency on SAMZA-484? I haven't had
>> a
>> > chance to commit that one yet.
>> >
>> > Cheers,
>> > Chris
>> >
>> > On Wed, Feb 11, 2015 at 9:23 AM, Milinda Pathirage <
>> mpathira@umail.iu.edu>
>> > wrote:
>> >
>> > > /need/want
>> > >
>> > > Milinda
>> > >
>> > > On Wed, Feb 11, 2015 at 12:22 PM, Milinda Pathirage <
>> > mpathira@umail.iu.edu
>> > > >
>> > > wrote:
>> > >
>> > > > Hi Chris,
>> > > >
>> > > > Do you need me to create the SAMZA-483 patch against latest master
>> with
>> > > > SAMZA-482 patch? I think that will make it easier to review the
>> patch?
>> > > >
>> > > > Thanks
>> > > > Milinda
>> > > >
>> > > > On Mon, Feb 9, 2015 at 11:39 AM, Chris Riccomini <
>> > criccomini@apache.org>
>> > > > wrote:
>> > > >
>> > > >> Cool. Looks like we've got consensus. I'll move forward with RTC on
>> > some
>> > > >> of
>> > > >> the early SAMZA-390 tickets. (SAMZA-482, SAMZA-483, SAMZA-484)
>> > > >>
>> > > >> On Mon, Feb 9, 2015 at 8:35 AM, Yan Fang <ya...@gmail.com>
>> > wrote:
>> > > >>
>> > > >> > +1 on this.
>> > > >> >
>> > > >> > Fang, Yan
>> > > >> > yanfang724@gmail.com
>> > > >> > +1 (206) 849-4108
>> > > >> >
>> > > >> > On Fri, Feb 6, 2015 at 4:38 PM, Chris Riccomini <
>> > > criccomini@apache.org>
>> > > >> > wrote:
>> > > >> >
>> > > >> > > Hey all,
>> > > >> > >
>> > > >> > > Are we +1 on this? I think Jakob was the only one who was
>> curious
>> > > >> about
>> > > >> > it.
>> > > >> > >
>> > > >> > > Cheers,
>> > > >> > > Chris
>> > > >> > >
>> > > >> > > On Thu, Feb 5, 2015 at 1:22 PM, Yi Pan <ni...@gmail.com>
>> > wrote:
>> > > >> > >
>> > > >> > > > Hi, Jakob,
>> > > >> > > >
>> > > >> > > >
>> > > >> > > > > > Eh? Not sure what this means...
>> > > >> > > > >
>> > > >> > > > > I mean SAMZA-484 depends on SAMZA-482, and neither are
>> > > committed.
>> > > >> So
>> > > >> > > > Navina
>> > > >> > > > > is having to post Yi's patch, as well as her own, on the
>> JIRA.
>> > > It
>> > > >> > makes
>> > > >> > > > it
>> > > >> > > > > really hard to do code reviews because you can't tell
>> whether
>> > Yi
>> > > >> made
>> > > >> > > the
>> > > >> > > > > changes or Navina did.
>> > > >> > > > >
>> > > >> > > > >
>> > > >> > > > Just to add to the point. It is also difficult to always see
>> a
>> > > long
>> > > >> > list
>> > > >> > > of
>> > > >> > > > changed files if the RB request is always based on the
>> master.
>> > It
>> > > is
>> > > >> > > > possible to have RB request based on another RB request (I
>> have
>> > > >> tried
>> > > >> > it
>> > > >> > > > before). But what happens if the base RB request is
>> > > >> > cancelled/discarded?
>> > > >> > > RB
>> > > >> > > > is not designed to track the revision changes in a dependency
>> > > chain.
>> > > >> > > >
>> > > >> > > >
>> > > >> > > >
>> > > >> > > > > Cheers,
>> > > >> > > > > Chris
>> > > >> > > > >
>> > > >> > > > > On Thu, Feb 5, 2015 at 12:16 PM, Jakob Homan <
>> > jghoman@gmail.com
>> > > >
>> > > >> > > wrote:
>> > > >> > > > >
>> > > >> > > > > > > I want to avoid branches,
>> > > >> > > > > > Just curious, any reason for this?
>> > > >> > > > > >
>> > > >> > > > > > > and I also want to avoid revision control over JIRA
>> > > >> > > > > > Eh? Not sure what this means...
>> > > >> > > > > >
>> > > >> > > > > > Thanks,
>> > > >> > > > > > jg
>> > > >> > > > > >
>> > > >> > > > > > On 4 February 2015 at 17:11, Chris Riccomini <
>> > > >> > criccomini@apache.org>
>> > > >> > > > > > wrote:
>> > > >> > > > > > > Hey all,
>> > > >> > > > > > >
>> > > >> > > > > > > @Jakob, yeah I was thinking we'll follow our normal
>> flow.
>> > > >> RTC. I
>> > > >> > > just
>> > > >> > > > > > > wanted to set expectation that the code committed
>> might be
>> > > >> not up
>> > > >> > > to
>> > > >> > > > > our
>> > > >> > > > > > > normal quality initially (missing docs, no tests, etc).
>> > > Until
>> > > >> the
>> > > >> > > > > quality
>> > > >> > > > > > > is raised, we should think of this module as
>> experimental.
>> > > >> > > > > > >
>> > > >> > > > > > > @Milinda, awesome! Thanks. :)
>> > > >> > > > > > >
>> > > >> > > > > > > Cheers,
>> > > >> > > > > > > Chris
>> > > >> > > > > > >
>> > > >> > > > > > > On Wed, Feb 4, 2015 at 11:57 AM, Milinda Pathirage <
>> > > >> > > > > > mpathira@umail.iu.edu>
>> > > >> > > > > > > wrote:
>> > > >> > > > > > >
>> > > >> > > > > > >> Hi Chris,
>> > > >> > > > > > >>
>> > > >> > > > > > >> Hope we no longer need the SQL API. I'll create a RB
>> for
>> > > >> Calcite
>> > > >> > > > > > >> integration.
>> > > >> > > > > > >>
>> > > >> > > > > > >> Thanks
>> > > >> > > > > > >> Milinda
>> > > >> > > > > > >>
>> > > >> > > > > > >> On Wed, Feb 4, 2015 at 1:31 PM, Chris Riccomini <
>> > > >> > > > > criccomini@apache.org>
>> > > >> > > > > > >> wrote:
>> > > >> > > > > > >>
>> > > >> > > > > > >> > I think so. There was some RB downtime, but it just
>> got
>> > > >> fixed.
>> > > >> > > Yi,
>> > > >> > > > > > >> Navina,
>> > > >> > > > > > >> > Milinda, can you make sure your JIRAs have up to
>> date
>> > > RBs?
>> > > >> > > > > > >> >
>> > > >> > > > > > >> > On Wed, Feb 4, 2015 at 10:24 AM, sriram <
>> > > >> sriram.sub@gmail.com
>> > > >> > >
>> > > >> > > > > wrote:
>> > > >> > > > > > >> >
>> > > >> > > > > > >> > > Can we have updated RBs for all the three sub
>> tasks
>> > > >> before
>> > > >> > we
>> > > >> > > > > > commit?
>> > > >> > > > > > >> > This
>> > > >> > > > > > >> > > would help us to review even after we commit.
>> > > >> > > > > > >> > >
>> > > >> > > > > > >> > > On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini <
>> > > >> > > > > > >> criccomini@apache.org>
>> > > >> > > > > > >> > > wrote:
>> > > >> > > > > > >> > >
>> > > >> > > > > > >> > > > Hey all,
>> > > >> > > > > > >> > > >
>> > > >> > > > > > >> > > > Yi, Navina, and Milinda have been working on
>> > > SAMZA-390
>> > > >> > > > > sub-tickets
>> > > >> > > > > > >> > > related
>> > > >> > > > > > >> > > > to SQL operators. We're getting to the point
>> where
>> > > the
>> > > >> > > amount
>> > > >> > > > of
>> > > >> > > > > > work
>> > > >> > > > > > >> > > > floating around is quite large, and some tickets
>> > > build
>> > > >> off
>> > > >> > > of
>> > > >> > > > > > others.
>> > > >> > > > > > >> > > >
>> > > >> > > > > > >> > > > I'm proposing that we commit this work into a
>> > > samza-sql
>> > > >> > > > > submodule
>> > > >> > > > > > on
>> > > >> > > > > > >> > > > master, and treat this module as experimental. I
>> > want
>> > > >> to
>> > > >> > > avoid
>> > > >> > > > > > >> > branches,
>> > > >> > > > > > >> > > > and I also want to avoid revision control over
>> > JIRA.
>> > > >> This
>> > > >> > > > means
>> > > >> > > > > > that
>> > > >> > > > > > >> > > there
>> > > >> > > > > > >> > > > will probably be a fair amount of commits/JIRAs
>> on
>> > > this
>> > > >> > > module
>> > > >> > > > > as
>> > > >> > > > > > we
>> > > >> > > > > > >> > > > iterate, but I think that's OK.
>> > > >> > > > > > >> > > >
>> > > >> > > > > > >> > > > Does this sound good to everyone?
>> > > >> > > > > > >> > > >
>> > > >> > > > > > >> > > > Cheers,
>> > > >> > > > > > >> > > > Chris
>> > > >> > > > > > >> > > >
>> > > >> > > > > > >> > >
>> > > >> > > > > > >> >
>> > > >> > > > > > >>
>> > > >> > > > > > >>
>> > > >> > > > > > >>
>> > > >> > > > > > >> --
>> > > >> > > > > > >> Milinda Pathirage
>> > > >> > > > > > >>
>> > > >> > > > > > >> PhD Student | Research Assistant
>> > > >> > > > > > >> School of Informatics and Computing | Data to Insight
>> > > Center
>> > > >> > > > > > >> Indiana University
>> > > >> > > > > > >>
>> > > >> > > > > > >> twitter: milindalakmal
>> > > >> > > > > > >> skype: milinda.pathirage
>> > > >> > > > > > >> blog: http://milinda.pathirage.org
>> > > >> > > > > > >>
>> > > >> > > > > >
>> > > >> > > > >
>> > > >> > > >
>> > > >> > >
>> > > >> >
>> > > >>
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > Milinda Pathirage
>> > > >
>> > > > PhD Student | Research Assistant
>> > > > School of Informatics and Computing | Data to Insight Center
>> > > > Indiana University
>> > > >
>> > > > twitter: milindalakmal
>> > > > skype: milinda.pathirage
>> > > > blog: http://milinda.pathirage.org
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > Milinda Pathirage
>> > >
>> > > PhD Student | Research Assistant
>> > > School of Informatics and Computing | Data to Insight Center
>> > > Indiana University
>> > >
>> > > twitter: milindalakmal
>> > > skype: milinda.pathirage
>> > > blog: http://milinda.pathirage.org
>> > >
>> >
>>
>>
>>
>> --
>> Milinda Pathirage
>>
>> PhD Student | Research Assistant
>> School of Informatics and Computing | Data to Insight Center
>> Indiana University
>>
>> twitter: milindalakmal
>> skype: milinda.pathirage
>> blog: http://milinda.pathirage.org
>>
>
>
Re: [DISCUSS] SQL workflow
Posted by Chris Riccomini <cr...@apache.org>.
Hey Milinda,
I've just committed SAMZA-484, so you should be able to re-base and get all
the latest code.
> But we need to add Calcite as a dependency and Calcite streaming support
is only in Julian's branch
We'll have to have Julian publish a release to Apache. This can be a
SNAPSHOT to Apache's SNAPSHOT repo, or a real Calcite release. We can't
check in binaries to the repo (licensing, release complexity, and just
generally a bad idea). We'll have to coordinate with him.
Cheers,
Chris
On Wed, Feb 11, 2015 at 9:43 AM, Milinda Pathirage <mp...@umail.iu.edu>
wrote:
> No, SAMZA-483 doesn't depend on SAMZA-484. But we need to add Calcite as a
> dependency and Calcite streaming support is only in Julian's branch (
> https://github.com/julianhyde/incubator-calcite/tree/chi). What can we do
> about that?
>
> Milinda
>
> On Wed, Feb 11, 2015 at 12:30 PM, Chris Riccomini <cr...@apache.org>
> wrote:
>
> > Hey Milinda,
> >
> > That'd be great. Do you have any dependency on SAMZA-484? I haven't had a
> > chance to commit that one yet.
> >
> > Cheers,
> > Chris
> >
> > On Wed, Feb 11, 2015 at 9:23 AM, Milinda Pathirage <
> mpathira@umail.iu.edu>
> > wrote:
> >
> > > /need/want
> > >
> > > Milinda
> > >
> > > On Wed, Feb 11, 2015 at 12:22 PM, Milinda Pathirage <
> > mpathira@umail.iu.edu
> > > >
> > > wrote:
> > >
> > > > Hi Chris,
> > > >
> > > > Do you need me to create the SAMZA-483 patch against latest master
> with
> > > > SAMZA-482 patch? I think that will make it easier to review the
> patch?
> > > >
> > > > Thanks
> > > > Milinda
> > > >
> > > > On Mon, Feb 9, 2015 at 11:39 AM, Chris Riccomini <
> > criccomini@apache.org>
> > > > wrote:
> > > >
> > > >> Cool. Looks like we've got consensus. I'll move forward with RTC on
> > some
> > > >> of
> > > >> the early SAMZA-390 tickets. (SAMZA-482, SAMZA-483, SAMZA-484)
> > > >>
> > > >> On Mon, Feb 9, 2015 at 8:35 AM, Yan Fang <ya...@gmail.com>
> > wrote:
> > > >>
> > > >> > +1 on this.
> > > >> >
> > > >> > Fang, Yan
> > > >> > yanfang724@gmail.com
> > > >> > +1 (206) 849-4108
> > > >> >
> > > >> > On Fri, Feb 6, 2015 at 4:38 PM, Chris Riccomini <
> > > criccomini@apache.org>
> > > >> > wrote:
> > > >> >
> > > >> > > Hey all,
> > > >> > >
> > > >> > > Are we +1 on this? I think Jakob was the only one who was
> curious
> > > >> about
> > > >> > it.
> > > >> > >
> > > >> > > Cheers,
> > > >> > > Chris
> > > >> > >
> > > >> > > On Thu, Feb 5, 2015 at 1:22 PM, Yi Pan <ni...@gmail.com>
> > wrote:
> > > >> > >
> > > >> > > > Hi, Jakob,
> > > >> > > >
> > > >> > > >
> > > >> > > > > > Eh? Not sure what this means...
> > > >> > > > >
> > > >> > > > > I mean SAMZA-484 depends on SAMZA-482, and neither are
> > > committed.
> > > >> So
> > > >> > > > Navina
> > > >> > > > > is having to post Yi's patch, as well as her own, on the
> JIRA.
> > > It
> > > >> > makes
> > > >> > > > it
> > > >> > > > > really hard to do code reviews because you can't tell
> whether
> > Yi
> > > >> made
> > > >> > > the
> > > >> > > > > changes or Navina did.
> > > >> > > > >
> > > >> > > > >
> > > >> > > > Just to add to the point. It is also difficult to always see a
> > > long
> > > >> > list
> > > >> > > of
> > > >> > > > changed files if the RB request is always based on the master.
> > It
> > > is
> > > >> > > > possible to have RB request based on another RB request (I
> have
> > > >> tried
> > > >> > it
> > > >> > > > before). But what happens if the base RB request is
> > > >> > cancelled/discarded?
> > > >> > > RB
> > > >> > > > is not designed to track the revision changes in a dependency
> > > chain.
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > > > Cheers,
> > > >> > > > > Chris
> > > >> > > > >
> > > >> > > > > On Thu, Feb 5, 2015 at 12:16 PM, Jakob Homan <
> > jghoman@gmail.com
> > > >
> > > >> > > wrote:
> > > >> > > > >
> > > >> > > > > > > I want to avoid branches,
> > > >> > > > > > Just curious, any reason for this?
> > > >> > > > > >
> > > >> > > > > > > and I also want to avoid revision control over JIRA
> > > >> > > > > > Eh? Not sure what this means...
> > > >> > > > > >
> > > >> > > > > > Thanks,
> > > >> > > > > > jg
> > > >> > > > > >
> > > >> > > > > > On 4 February 2015 at 17:11, Chris Riccomini <
> > > >> > criccomini@apache.org>
> > > >> > > > > > wrote:
> > > >> > > > > > > Hey all,
> > > >> > > > > > >
> > > >> > > > > > > @Jakob, yeah I was thinking we'll follow our normal
> flow.
> > > >> RTC. I
> > > >> > > just
> > > >> > > > > > > wanted to set expectation that the code committed might
> be
> > > >> not up
> > > >> > > to
> > > >> > > > > our
> > > >> > > > > > > normal quality initially (missing docs, no tests, etc).
> > > Until
> > > >> the
> > > >> > > > > quality
> > > >> > > > > > > is raised, we should think of this module as
> experimental.
> > > >> > > > > > >
> > > >> > > > > > > @Milinda, awesome! Thanks. :)
> > > >> > > > > > >
> > > >> > > > > > > Cheers,
> > > >> > > > > > > Chris
> > > >> > > > > > >
> > > >> > > > > > > On Wed, Feb 4, 2015 at 11:57 AM, Milinda Pathirage <
> > > >> > > > > > mpathira@umail.iu.edu>
> > > >> > > > > > > wrote:
> > > >> > > > > > >
> > > >> > > > > > >> Hi Chris,
> > > >> > > > > > >>
> > > >> > > > > > >> Hope we no longer need the SQL API. I'll create a RB
> for
> > > >> Calcite
> > > >> > > > > > >> integration.
> > > >> > > > > > >>
> > > >> > > > > > >> Thanks
> > > >> > > > > > >> Milinda
> > > >> > > > > > >>
> > > >> > > > > > >> On Wed, Feb 4, 2015 at 1:31 PM, Chris Riccomini <
> > > >> > > > > criccomini@apache.org>
> > > >> > > > > > >> wrote:
> > > >> > > > > > >>
> > > >> > > > > > >> > I think so. There was some RB downtime, but it just
> got
> > > >> fixed.
> > > >> > > Yi,
> > > >> > > > > > >> Navina,
> > > >> > > > > > >> > Milinda, can you make sure your JIRAs have up to date
> > > RBs?
> > > >> > > > > > >> >
> > > >> > > > > > >> > On Wed, Feb 4, 2015 at 10:24 AM, sriram <
> > > >> sriram.sub@gmail.com
> > > >> > >
> > > >> > > > > wrote:
> > > >> > > > > > >> >
> > > >> > > > > > >> > > Can we have updated RBs for all the three sub tasks
> > > >> before
> > > >> > we
> > > >> > > > > > commit?
> > > >> > > > > > >> > This
> > > >> > > > > > >> > > would help us to review even after we commit.
> > > >> > > > > > >> > >
> > > >> > > > > > >> > > On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini <
> > > >> > > > > > >> criccomini@apache.org>
> > > >> > > > > > >> > > wrote:
> > > >> > > > > > >> > >
> > > >> > > > > > >> > > > Hey all,
> > > >> > > > > > >> > > >
> > > >> > > > > > >> > > > Yi, Navina, and Milinda have been working on
> > > SAMZA-390
> > > >> > > > > sub-tickets
> > > >> > > > > > >> > > related
> > > >> > > > > > >> > > > to SQL operators. We're getting to the point
> where
> > > the
> > > >> > > amount
> > > >> > > > of
> > > >> > > > > > work
> > > >> > > > > > >> > > > floating around is quite large, and some tickets
> > > build
> > > >> off
> > > >> > > of
> > > >> > > > > > others.
> > > >> > > > > > >> > > >
> > > >> > > > > > >> > > > I'm proposing that we commit this work into a
> > > samza-sql
> > > >> > > > > submodule
> > > >> > > > > > on
> > > >> > > > > > >> > > > master, and treat this module as experimental. I
> > want
> > > >> to
> > > >> > > avoid
> > > >> > > > > > >> > branches,
> > > >> > > > > > >> > > > and I also want to avoid revision control over
> > JIRA.
> > > >> This
> > > >> > > > means
> > > >> > > > > > that
> > > >> > > > > > >> > > there
> > > >> > > > > > >> > > > will probably be a fair amount of commits/JIRAs
> on
> > > this
> > > >> > > module
> > > >> > > > > as
> > > >> > > > > > we
> > > >> > > > > > >> > > > iterate, but I think that's OK.
> > > >> > > > > > >> > > >
> > > >> > > > > > >> > > > Does this sound good to everyone?
> > > >> > > > > > >> > > >
> > > >> > > > > > >> > > > Cheers,
> > > >> > > > > > >> > > > Chris
> > > >> > > > > > >> > > >
> > > >> > > > > > >> > >
> > > >> > > > > > >> >
> > > >> > > > > > >>
> > > >> > > > > > >>
> > > >> > > > > > >>
> > > >> > > > > > >> --
> > > >> > > > > > >> Milinda Pathirage
> > > >> > > > > > >>
> > > >> > > > > > >> PhD Student | Research Assistant
> > > >> > > > > > >> School of Informatics and Computing | Data to Insight
> > > Center
> > > >> > > > > > >> Indiana University
> > > >> > > > > > >>
> > > >> > > > > > >> twitter: milindalakmal
> > > >> > > > > > >> skype: milinda.pathirage
> > > >> > > > > > >> blog: http://milinda.pathirage.org
> > > >> > > > > > >>
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >
> > > >
> > > >
> > > > --
> > > > Milinda Pathirage
> > > >
> > > > PhD Student | Research Assistant
> > > > School of Informatics and Computing | Data to Insight Center
> > > > Indiana University
> > > >
> > > > twitter: milindalakmal
> > > > skype: milinda.pathirage
> > > > blog: http://milinda.pathirage.org
> > > >
> > >
> > >
> > >
> > > --
> > > Milinda Pathirage
> > >
> > > PhD Student | Research Assistant
> > > School of Informatics and Computing | Data to Insight Center
> > > Indiana University
> > >
> > > twitter: milindalakmal
> > > skype: milinda.pathirage
> > > blog: http://milinda.pathirage.org
> > >
> >
>
>
>
> --
> Milinda Pathirage
>
> PhD Student | Research Assistant
> School of Informatics and Computing | Data to Insight Center
> Indiana University
>
> twitter: milindalakmal
> skype: milinda.pathirage
> blog: http://milinda.pathirage.org
>
Re: [DISCUSS] SQL workflow
Posted by Milinda Pathirage <mp...@umail.iu.edu>.
No, SAMZA-483 doesn't depend on SAMZA-484. But we need to add Calcite as a
dependency and Calcite streaming support is only in Julian's branch (
https://github.com/julianhyde/incubator-calcite/tree/chi). What can we do
about that?
Milinda
On Wed, Feb 11, 2015 at 12:30 PM, Chris Riccomini <cr...@apache.org>
wrote:
> Hey Milinda,
>
> That'd be great. Do you have any dependency on SAMZA-484? I haven't had a
> chance to commit that one yet.
>
> Cheers,
> Chris
>
> On Wed, Feb 11, 2015 at 9:23 AM, Milinda Pathirage <mp...@umail.iu.edu>
> wrote:
>
> > /need/want
> >
> > Milinda
> >
> > On Wed, Feb 11, 2015 at 12:22 PM, Milinda Pathirage <
> mpathira@umail.iu.edu
> > >
> > wrote:
> >
> > > Hi Chris,
> > >
> > > Do you need me to create the SAMZA-483 patch against latest master with
> > > SAMZA-482 patch? I think that will make it easier to review the patch?
> > >
> > > Thanks
> > > Milinda
> > >
> > > On Mon, Feb 9, 2015 at 11:39 AM, Chris Riccomini <
> criccomini@apache.org>
> > > wrote:
> > >
> > >> Cool. Looks like we've got consensus. I'll move forward with RTC on
> some
> > >> of
> > >> the early SAMZA-390 tickets. (SAMZA-482, SAMZA-483, SAMZA-484)
> > >>
> > >> On Mon, Feb 9, 2015 at 8:35 AM, Yan Fang <ya...@gmail.com>
> wrote:
> > >>
> > >> > +1 on this.
> > >> >
> > >> > Fang, Yan
> > >> > yanfang724@gmail.com
> > >> > +1 (206) 849-4108
> > >> >
> > >> > On Fri, Feb 6, 2015 at 4:38 PM, Chris Riccomini <
> > criccomini@apache.org>
> > >> > wrote:
> > >> >
> > >> > > Hey all,
> > >> > >
> > >> > > Are we +1 on this? I think Jakob was the only one who was curious
> > >> about
> > >> > it.
> > >> > >
> > >> > > Cheers,
> > >> > > Chris
> > >> > >
> > >> > > On Thu, Feb 5, 2015 at 1:22 PM, Yi Pan <ni...@gmail.com>
> wrote:
> > >> > >
> > >> > > > Hi, Jakob,
> > >> > > >
> > >> > > >
> > >> > > > > > Eh? Not sure what this means...
> > >> > > > >
> > >> > > > > I mean SAMZA-484 depends on SAMZA-482, and neither are
> > committed.
> > >> So
> > >> > > > Navina
> > >> > > > > is having to post Yi's patch, as well as her own, on the JIRA.
> > It
> > >> > makes
> > >> > > > it
> > >> > > > > really hard to do code reviews because you can't tell whether
> Yi
> > >> made
> > >> > > the
> > >> > > > > changes or Navina did.
> > >> > > > >
> > >> > > > >
> > >> > > > Just to add to the point. It is also difficult to always see a
> > long
> > >> > list
> > >> > > of
> > >> > > > changed files if the RB request is always based on the master.
> It
> > is
> > >> > > > possible to have RB request based on another RB request (I have
> > >> tried
> > >> > it
> > >> > > > before). But what happens if the base RB request is
> > >> > cancelled/discarded?
> > >> > > RB
> > >> > > > is not designed to track the revision changes in a dependency
> > chain.
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > > Cheers,
> > >> > > > > Chris
> > >> > > > >
> > >> > > > > On Thu, Feb 5, 2015 at 12:16 PM, Jakob Homan <
> jghoman@gmail.com
> > >
> > >> > > wrote:
> > >> > > > >
> > >> > > > > > > I want to avoid branches,
> > >> > > > > > Just curious, any reason for this?
> > >> > > > > >
> > >> > > > > > > and I also want to avoid revision control over JIRA
> > >> > > > > > Eh? Not sure what this means...
> > >> > > > > >
> > >> > > > > > Thanks,
> > >> > > > > > jg
> > >> > > > > >
> > >> > > > > > On 4 February 2015 at 17:11, Chris Riccomini <
> > >> > criccomini@apache.org>
> > >> > > > > > wrote:
> > >> > > > > > > Hey all,
> > >> > > > > > >
> > >> > > > > > > @Jakob, yeah I was thinking we'll follow our normal flow.
> > >> RTC. I
> > >> > > just
> > >> > > > > > > wanted to set expectation that the code committed might be
> > >> not up
> > >> > > to
> > >> > > > > our
> > >> > > > > > > normal quality initially (missing docs, no tests, etc).
> > Until
> > >> the
> > >> > > > > quality
> > >> > > > > > > is raised, we should think of this module as experimental.
> > >> > > > > > >
> > >> > > > > > > @Milinda, awesome! Thanks. :)
> > >> > > > > > >
> > >> > > > > > > Cheers,
> > >> > > > > > > Chris
> > >> > > > > > >
> > >> > > > > > > On Wed, Feb 4, 2015 at 11:57 AM, Milinda Pathirage <
> > >> > > > > > mpathira@umail.iu.edu>
> > >> > > > > > > wrote:
> > >> > > > > > >
> > >> > > > > > >> Hi Chris,
> > >> > > > > > >>
> > >> > > > > > >> Hope we no longer need the SQL API. I'll create a RB for
> > >> Calcite
> > >> > > > > > >> integration.
> > >> > > > > > >>
> > >> > > > > > >> Thanks
> > >> > > > > > >> Milinda
> > >> > > > > > >>
> > >> > > > > > >> On Wed, Feb 4, 2015 at 1:31 PM, Chris Riccomini <
> > >> > > > > criccomini@apache.org>
> > >> > > > > > >> wrote:
> > >> > > > > > >>
> > >> > > > > > >> > I think so. There was some RB downtime, but it just got
> > >> fixed.
> > >> > > Yi,
> > >> > > > > > >> Navina,
> > >> > > > > > >> > Milinda, can you make sure your JIRAs have up to date
> > RBs?
> > >> > > > > > >> >
> > >> > > > > > >> > On Wed, Feb 4, 2015 at 10:24 AM, sriram <
> > >> sriram.sub@gmail.com
> > >> > >
> > >> > > > > wrote:
> > >> > > > > > >> >
> > >> > > > > > >> > > Can we have updated RBs for all the three sub tasks
> > >> before
> > >> > we
> > >> > > > > > commit?
> > >> > > > > > >> > This
> > >> > > > > > >> > > would help us to review even after we commit.
> > >> > > > > > >> > >
> > >> > > > > > >> > > On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini <
> > >> > > > > > >> criccomini@apache.org>
> > >> > > > > > >> > > wrote:
> > >> > > > > > >> > >
> > >> > > > > > >> > > > Hey all,
> > >> > > > > > >> > > >
> > >> > > > > > >> > > > Yi, Navina, and Milinda have been working on
> > SAMZA-390
> > >> > > > > sub-tickets
> > >> > > > > > >> > > related
> > >> > > > > > >> > > > to SQL operators. We're getting to the point where
> > the
> > >> > > amount
> > >> > > > of
> > >> > > > > > work
> > >> > > > > > >> > > > floating around is quite large, and some tickets
> > build
> > >> off
> > >> > > of
> > >> > > > > > others.
> > >> > > > > > >> > > >
> > >> > > > > > >> > > > I'm proposing that we commit this work into a
> > samza-sql
> > >> > > > > submodule
> > >> > > > > > on
> > >> > > > > > >> > > > master, and treat this module as experimental. I
> want
> > >> to
> > >> > > avoid
> > >> > > > > > >> > branches,
> > >> > > > > > >> > > > and I also want to avoid revision control over
> JIRA.
> > >> This
> > >> > > > means
> > >> > > > > > that
> > >> > > > > > >> > > there
> > >> > > > > > >> > > > will probably be a fair amount of commits/JIRAs on
> > this
> > >> > > module
> > >> > > > > as
> > >> > > > > > we
> > >> > > > > > >> > > > iterate, but I think that's OK.
> > >> > > > > > >> > > >
> > >> > > > > > >> > > > Does this sound good to everyone?
> > >> > > > > > >> > > >
> > >> > > > > > >> > > > Cheers,
> > >> > > > > > >> > > > Chris
> > >> > > > > > >> > > >
> > >> > > > > > >> > >
> > >> > > > > > >> >
> > >> > > > > > >>
> > >> > > > > > >>
> > >> > > > > > >>
> > >> > > > > > >> --
> > >> > > > > > >> Milinda Pathirage
> > >> > > > > > >>
> > >> > > > > > >> PhD Student | Research Assistant
> > >> > > > > > >> School of Informatics and Computing | Data to Insight
> > Center
> > >> > > > > > >> Indiana University
> > >> > > > > > >>
> > >> > > > > > >> twitter: milindalakmal
> > >> > > > > > >> skype: milinda.pathirage
> > >> > > > > > >> blog: http://milinda.pathirage.org
> > >> > > > > > >>
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> > >
> > > --
> > > Milinda Pathirage
> > >
> > > PhD Student | Research Assistant
> > > School of Informatics and Computing | Data to Insight Center
> > > Indiana University
> > >
> > > twitter: milindalakmal
> > > skype: milinda.pathirage
> > > blog: http://milinda.pathirage.org
> > >
> >
> >
> >
> > --
> > Milinda Pathirage
> >
> > PhD Student | Research Assistant
> > School of Informatics and Computing | Data to Insight Center
> > Indiana University
> >
> > twitter: milindalakmal
> > skype: milinda.pathirage
> > blog: http://milinda.pathirage.org
> >
>
--
Milinda Pathirage
PhD Student | Research Assistant
School of Informatics and Computing | Data to Insight Center
Indiana University
twitter: milindalakmal
skype: milinda.pathirage
blog: http://milinda.pathirage.org
Re: [DISCUSS] SQL workflow
Posted by Chris Riccomini <cr...@apache.org>.
Hey Milinda,
That'd be great. Do you have any dependency on SAMZA-484? I haven't had a
chance to commit that one yet.
Cheers,
Chris
On Wed, Feb 11, 2015 at 9:23 AM, Milinda Pathirage <mp...@umail.iu.edu>
wrote:
> /need/want
>
> Milinda
>
> On Wed, Feb 11, 2015 at 12:22 PM, Milinda Pathirage <mpathira@umail.iu.edu
> >
> wrote:
>
> > Hi Chris,
> >
> > Do you need me to create the SAMZA-483 patch against latest master with
> > SAMZA-482 patch? I think that will make it easier to review the patch?
> >
> > Thanks
> > Milinda
> >
> > On Mon, Feb 9, 2015 at 11:39 AM, Chris Riccomini <cr...@apache.org>
> > wrote:
> >
> >> Cool. Looks like we've got consensus. I'll move forward with RTC on some
> >> of
> >> the early SAMZA-390 tickets. (SAMZA-482, SAMZA-483, SAMZA-484)
> >>
> >> On Mon, Feb 9, 2015 at 8:35 AM, Yan Fang <ya...@gmail.com> wrote:
> >>
> >> > +1 on this.
> >> >
> >> > Fang, Yan
> >> > yanfang724@gmail.com
> >> > +1 (206) 849-4108
> >> >
> >> > On Fri, Feb 6, 2015 at 4:38 PM, Chris Riccomini <
> criccomini@apache.org>
> >> > wrote:
> >> >
> >> > > Hey all,
> >> > >
> >> > > Are we +1 on this? I think Jakob was the only one who was curious
> >> about
> >> > it.
> >> > >
> >> > > Cheers,
> >> > > Chris
> >> > >
> >> > > On Thu, Feb 5, 2015 at 1:22 PM, Yi Pan <ni...@gmail.com> wrote:
> >> > >
> >> > > > Hi, Jakob,
> >> > > >
> >> > > >
> >> > > > > > Eh? Not sure what this means...
> >> > > > >
> >> > > > > I mean SAMZA-484 depends on SAMZA-482, and neither are
> committed.
> >> So
> >> > > > Navina
> >> > > > > is having to post Yi's patch, as well as her own, on the JIRA.
> It
> >> > makes
> >> > > > it
> >> > > > > really hard to do code reviews because you can't tell whether Yi
> >> made
> >> > > the
> >> > > > > changes or Navina did.
> >> > > > >
> >> > > > >
> >> > > > Just to add to the point. It is also difficult to always see a
> long
> >> > list
> >> > > of
> >> > > > changed files if the RB request is always based on the master. It
> is
> >> > > > possible to have RB request based on another RB request (I have
> >> tried
> >> > it
> >> > > > before). But what happens if the base RB request is
> >> > cancelled/discarded?
> >> > > RB
> >> > > > is not designed to track the revision changes in a dependency
> chain.
> >> > > >
> >> > > >
> >> > > >
> >> > > > > Cheers,
> >> > > > > Chris
> >> > > > >
> >> > > > > On Thu, Feb 5, 2015 at 12:16 PM, Jakob Homan <jghoman@gmail.com
> >
> >> > > wrote:
> >> > > > >
> >> > > > > > > I want to avoid branches,
> >> > > > > > Just curious, any reason for this?
> >> > > > > >
> >> > > > > > > and I also want to avoid revision control over JIRA
> >> > > > > > Eh? Not sure what this means...
> >> > > > > >
> >> > > > > > Thanks,
> >> > > > > > jg
> >> > > > > >
> >> > > > > > On 4 February 2015 at 17:11, Chris Riccomini <
> >> > criccomini@apache.org>
> >> > > > > > wrote:
> >> > > > > > > Hey all,
> >> > > > > > >
> >> > > > > > > @Jakob, yeah I was thinking we'll follow our normal flow.
> >> RTC. I
> >> > > just
> >> > > > > > > wanted to set expectation that the code committed might be
> >> not up
> >> > > to
> >> > > > > our
> >> > > > > > > normal quality initially (missing docs, no tests, etc).
> Until
> >> the
> >> > > > > quality
> >> > > > > > > is raised, we should think of this module as experimental.
> >> > > > > > >
> >> > > > > > > @Milinda, awesome! Thanks. :)
> >> > > > > > >
> >> > > > > > > Cheers,
> >> > > > > > > Chris
> >> > > > > > >
> >> > > > > > > On Wed, Feb 4, 2015 at 11:57 AM, Milinda Pathirage <
> >> > > > > > mpathira@umail.iu.edu>
> >> > > > > > > wrote:
> >> > > > > > >
> >> > > > > > >> Hi Chris,
> >> > > > > > >>
> >> > > > > > >> Hope we no longer need the SQL API. I'll create a RB for
> >> Calcite
> >> > > > > > >> integration.
> >> > > > > > >>
> >> > > > > > >> Thanks
> >> > > > > > >> Milinda
> >> > > > > > >>
> >> > > > > > >> On Wed, Feb 4, 2015 at 1:31 PM, Chris Riccomini <
> >> > > > > criccomini@apache.org>
> >> > > > > > >> wrote:
> >> > > > > > >>
> >> > > > > > >> > I think so. There was some RB downtime, but it just got
> >> fixed.
> >> > > Yi,
> >> > > > > > >> Navina,
> >> > > > > > >> > Milinda, can you make sure your JIRAs have up to date
> RBs?
> >> > > > > > >> >
> >> > > > > > >> > On Wed, Feb 4, 2015 at 10:24 AM, sriram <
> >> sriram.sub@gmail.com
> >> > >
> >> > > > > wrote:
> >> > > > > > >> >
> >> > > > > > >> > > Can we have updated RBs for all the three sub tasks
> >> before
> >> > we
> >> > > > > > commit?
> >> > > > > > >> > This
> >> > > > > > >> > > would help us to review even after we commit.
> >> > > > > > >> > >
> >> > > > > > >> > > On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini <
> >> > > > > > >> criccomini@apache.org>
> >> > > > > > >> > > wrote:
> >> > > > > > >> > >
> >> > > > > > >> > > > Hey all,
> >> > > > > > >> > > >
> >> > > > > > >> > > > Yi, Navina, and Milinda have been working on
> SAMZA-390
> >> > > > > sub-tickets
> >> > > > > > >> > > related
> >> > > > > > >> > > > to SQL operators. We're getting to the point where
> the
> >> > > amount
> >> > > > of
> >> > > > > > work
> >> > > > > > >> > > > floating around is quite large, and some tickets
> build
> >> off
> >> > > of
> >> > > > > > others.
> >> > > > > > >> > > >
> >> > > > > > >> > > > I'm proposing that we commit this work into a
> samza-sql
> >> > > > > submodule
> >> > > > > > on
> >> > > > > > >> > > > master, and treat this module as experimental. I want
> >> to
> >> > > avoid
> >> > > > > > >> > branches,
> >> > > > > > >> > > > and I also want to avoid revision control over JIRA.
> >> This
> >> > > > means
> >> > > > > > that
> >> > > > > > >> > > there
> >> > > > > > >> > > > will probably be a fair amount of commits/JIRAs on
> this
> >> > > module
> >> > > > > as
> >> > > > > > we
> >> > > > > > >> > > > iterate, but I think that's OK.
> >> > > > > > >> > > >
> >> > > > > > >> > > > Does this sound good to everyone?
> >> > > > > > >> > > >
> >> > > > > > >> > > > Cheers,
> >> > > > > > >> > > > Chris
> >> > > > > > >> > > >
> >> > > > > > >> > >
> >> > > > > > >> >
> >> > > > > > >>
> >> > > > > > >>
> >> > > > > > >>
> >> > > > > > >> --
> >> > > > > > >> Milinda Pathirage
> >> > > > > > >>
> >> > > > > > >> PhD Student | Research Assistant
> >> > > > > > >> School of Informatics and Computing | Data to Insight
> Center
> >> > > > > > >> Indiana University
> >> > > > > > >>
> >> > > > > > >> twitter: milindalakmal
> >> > > > > > >> skype: milinda.pathirage
> >> > > > > > >> blog: http://milinda.pathirage.org
> >> > > > > > >>
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
> >
> > --
> > Milinda Pathirage
> >
> > PhD Student | Research Assistant
> > School of Informatics and Computing | Data to Insight Center
> > Indiana University
> >
> > twitter: milindalakmal
> > skype: milinda.pathirage
> > blog: http://milinda.pathirage.org
> >
>
>
>
> --
> Milinda Pathirage
>
> PhD Student | Research Assistant
> School of Informatics and Computing | Data to Insight Center
> Indiana University
>
> twitter: milindalakmal
> skype: milinda.pathirage
> blog: http://milinda.pathirage.org
>
Re: [DISCUSS] SQL workflow
Posted by Milinda Pathirage <mp...@umail.iu.edu>.
/need/want
Milinda
On Wed, Feb 11, 2015 at 12:22 PM, Milinda Pathirage <mp...@umail.iu.edu>
wrote:
> Hi Chris,
>
> Do you need me to create the SAMZA-483 patch against latest master with
> SAMZA-482 patch? I think that will make it easier to review the patch?
>
> Thanks
> Milinda
>
> On Mon, Feb 9, 2015 at 11:39 AM, Chris Riccomini <cr...@apache.org>
> wrote:
>
>> Cool. Looks like we've got consensus. I'll move forward with RTC on some
>> of
>> the early SAMZA-390 tickets. (SAMZA-482, SAMZA-483, SAMZA-484)
>>
>> On Mon, Feb 9, 2015 at 8:35 AM, Yan Fang <ya...@gmail.com> wrote:
>>
>> > +1 on this.
>> >
>> > Fang, Yan
>> > yanfang724@gmail.com
>> > +1 (206) 849-4108
>> >
>> > On Fri, Feb 6, 2015 at 4:38 PM, Chris Riccomini <cr...@apache.org>
>> > wrote:
>> >
>> > > Hey all,
>> > >
>> > > Are we +1 on this? I think Jakob was the only one who was curious
>> about
>> > it.
>> > >
>> > > Cheers,
>> > > Chris
>> > >
>> > > On Thu, Feb 5, 2015 at 1:22 PM, Yi Pan <ni...@gmail.com> wrote:
>> > >
>> > > > Hi, Jakob,
>> > > >
>> > > >
>> > > > > > Eh? Not sure what this means...
>> > > > >
>> > > > > I mean SAMZA-484 depends on SAMZA-482, and neither are committed.
>> So
>> > > > Navina
>> > > > > is having to post Yi's patch, as well as her own, on the JIRA. It
>> > makes
>> > > > it
>> > > > > really hard to do code reviews because you can't tell whether Yi
>> made
>> > > the
>> > > > > changes or Navina did.
>> > > > >
>> > > > >
>> > > > Just to add to the point. It is also difficult to always see a long
>> > list
>> > > of
>> > > > changed files if the RB request is always based on the master. It is
>> > > > possible to have RB request based on another RB request (I have
>> tried
>> > it
>> > > > before). But what happens if the base RB request is
>> > cancelled/discarded?
>> > > RB
>> > > > is not designed to track the revision changes in a dependency chain.
>> > > >
>> > > >
>> > > >
>> > > > > Cheers,
>> > > > > Chris
>> > > > >
>> > > > > On Thu, Feb 5, 2015 at 12:16 PM, Jakob Homan <jg...@gmail.com>
>> > > wrote:
>> > > > >
>> > > > > > > I want to avoid branches,
>> > > > > > Just curious, any reason for this?
>> > > > > >
>> > > > > > > and I also want to avoid revision control over JIRA
>> > > > > > Eh? Not sure what this means...
>> > > > > >
>> > > > > > Thanks,
>> > > > > > jg
>> > > > > >
>> > > > > > On 4 February 2015 at 17:11, Chris Riccomini <
>> > criccomini@apache.org>
>> > > > > > wrote:
>> > > > > > > Hey all,
>> > > > > > >
>> > > > > > > @Jakob, yeah I was thinking we'll follow our normal flow.
>> RTC. I
>> > > just
>> > > > > > > wanted to set expectation that the code committed might be
>> not up
>> > > to
>> > > > > our
>> > > > > > > normal quality initially (missing docs, no tests, etc). Until
>> the
>> > > > > quality
>> > > > > > > is raised, we should think of this module as experimental.
>> > > > > > >
>> > > > > > > @Milinda, awesome! Thanks. :)
>> > > > > > >
>> > > > > > > Cheers,
>> > > > > > > Chris
>> > > > > > >
>> > > > > > > On Wed, Feb 4, 2015 at 11:57 AM, Milinda Pathirage <
>> > > > > > mpathira@umail.iu.edu>
>> > > > > > > wrote:
>> > > > > > >
>> > > > > > >> Hi Chris,
>> > > > > > >>
>> > > > > > >> Hope we no longer need the SQL API. I'll create a RB for
>> Calcite
>> > > > > > >> integration.
>> > > > > > >>
>> > > > > > >> Thanks
>> > > > > > >> Milinda
>> > > > > > >>
>> > > > > > >> On Wed, Feb 4, 2015 at 1:31 PM, Chris Riccomini <
>> > > > > criccomini@apache.org>
>> > > > > > >> wrote:
>> > > > > > >>
>> > > > > > >> > I think so. There was some RB downtime, but it just got
>> fixed.
>> > > Yi,
>> > > > > > >> Navina,
>> > > > > > >> > Milinda, can you make sure your JIRAs have up to date RBs?
>> > > > > > >> >
>> > > > > > >> > On Wed, Feb 4, 2015 at 10:24 AM, sriram <
>> sriram.sub@gmail.com
>> > >
>> > > > > wrote:
>> > > > > > >> >
>> > > > > > >> > > Can we have updated RBs for all the three sub tasks
>> before
>> > we
>> > > > > > commit?
>> > > > > > >> > This
>> > > > > > >> > > would help us to review even after we commit.
>> > > > > > >> > >
>> > > > > > >> > > On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini <
>> > > > > > >> criccomini@apache.org>
>> > > > > > >> > > wrote:
>> > > > > > >> > >
>> > > > > > >> > > > Hey all,
>> > > > > > >> > > >
>> > > > > > >> > > > Yi, Navina, and Milinda have been working on SAMZA-390
>> > > > > sub-tickets
>> > > > > > >> > > related
>> > > > > > >> > > > to SQL operators. We're getting to the point where the
>> > > amount
>> > > > of
>> > > > > > work
>> > > > > > >> > > > floating around is quite large, and some tickets build
>> off
>> > > of
>> > > > > > others.
>> > > > > > >> > > >
>> > > > > > >> > > > I'm proposing that we commit this work into a samza-sql
>> > > > > submodule
>> > > > > > on
>> > > > > > >> > > > master, and treat this module as experimental. I want
>> to
>> > > avoid
>> > > > > > >> > branches,
>> > > > > > >> > > > and I also want to avoid revision control over JIRA.
>> This
>> > > > means
>> > > > > > that
>> > > > > > >> > > there
>> > > > > > >> > > > will probably be a fair amount of commits/JIRAs on this
>> > > module
>> > > > > as
>> > > > > > we
>> > > > > > >> > > > iterate, but I think that's OK.
>> > > > > > >> > > >
>> > > > > > >> > > > Does this sound good to everyone?
>> > > > > > >> > > >
>> > > > > > >> > > > Cheers,
>> > > > > > >> > > > Chris
>> > > > > > >> > > >
>> > > > > > >> > >
>> > > > > > >> >
>> > > > > > >>
>> > > > > > >>
>> > > > > > >>
>> > > > > > >> --
>> > > > > > >> Milinda Pathirage
>> > > > > > >>
>> > > > > > >> PhD Student | Research Assistant
>> > > > > > >> School of Informatics and Computing | Data to Insight Center
>> > > > > > >> Indiana University
>> > > > > > >>
>> > > > > > >> twitter: milindalakmal
>> > > > > > >> skype: milinda.pathirage
>> > > > > > >> blog: http://milinda.pathirage.org
>> > > > > > >>
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
>
>
>
> --
> Milinda Pathirage
>
> PhD Student | Research Assistant
> School of Informatics and Computing | Data to Insight Center
> Indiana University
>
> twitter: milindalakmal
> skype: milinda.pathirage
> blog: http://milinda.pathirage.org
>
--
Milinda Pathirage
PhD Student | Research Assistant
School of Informatics and Computing | Data to Insight Center
Indiana University
twitter: milindalakmal
skype: milinda.pathirage
blog: http://milinda.pathirage.org
Re: [DISCUSS] SQL workflow
Posted by Milinda Pathirage <mp...@umail.iu.edu>.
Hi Chris,
Do you need me to create the SAMZA-483 patch against latest master with
SAMZA-482 patch? I think that will make it easier to review the patch?
Thanks
Milinda
On Mon, Feb 9, 2015 at 11:39 AM, Chris Riccomini <cr...@apache.org>
wrote:
> Cool. Looks like we've got consensus. I'll move forward with RTC on some of
> the early SAMZA-390 tickets. (SAMZA-482, SAMZA-483, SAMZA-484)
>
> On Mon, Feb 9, 2015 at 8:35 AM, Yan Fang <ya...@gmail.com> wrote:
>
> > +1 on this.
> >
> > Fang, Yan
> > yanfang724@gmail.com
> > +1 (206) 849-4108
> >
> > On Fri, Feb 6, 2015 at 4:38 PM, Chris Riccomini <cr...@apache.org>
> > wrote:
> >
> > > Hey all,
> > >
> > > Are we +1 on this? I think Jakob was the only one who was curious about
> > it.
> > >
> > > Cheers,
> > > Chris
> > >
> > > On Thu, Feb 5, 2015 at 1:22 PM, Yi Pan <ni...@gmail.com> wrote:
> > >
> > > > Hi, Jakob,
> > > >
> > > >
> > > > > > Eh? Not sure what this means...
> > > > >
> > > > > I mean SAMZA-484 depends on SAMZA-482, and neither are committed.
> So
> > > > Navina
> > > > > is having to post Yi's patch, as well as her own, on the JIRA. It
> > makes
> > > > it
> > > > > really hard to do code reviews because you can't tell whether Yi
> made
> > > the
> > > > > changes or Navina did.
> > > > >
> > > > >
> > > > Just to add to the point. It is also difficult to always see a long
> > list
> > > of
> > > > changed files if the RB request is always based on the master. It is
> > > > possible to have RB request based on another RB request (I have tried
> > it
> > > > before). But what happens if the base RB request is
> > cancelled/discarded?
> > > RB
> > > > is not designed to track the revision changes in a dependency chain.
> > > >
> > > >
> > > >
> > > > > Cheers,
> > > > > Chris
> > > > >
> > > > > On Thu, Feb 5, 2015 at 12:16 PM, Jakob Homan <jg...@gmail.com>
> > > wrote:
> > > > >
> > > > > > > I want to avoid branches,
> > > > > > Just curious, any reason for this?
> > > > > >
> > > > > > > and I also want to avoid revision control over JIRA
> > > > > > Eh? Not sure what this means...
> > > > > >
> > > > > > Thanks,
> > > > > > jg
> > > > > >
> > > > > > On 4 February 2015 at 17:11, Chris Riccomini <
> > criccomini@apache.org>
> > > > > > wrote:
> > > > > > > Hey all,
> > > > > > >
> > > > > > > @Jakob, yeah I was thinking we'll follow our normal flow. RTC.
> I
> > > just
> > > > > > > wanted to set expectation that the code committed might be not
> up
> > > to
> > > > > our
> > > > > > > normal quality initially (missing docs, no tests, etc). Until
> the
> > > > > quality
> > > > > > > is raised, we should think of this module as experimental.
> > > > > > >
> > > > > > > @Milinda, awesome! Thanks. :)
> > > > > > >
> > > > > > > Cheers,
> > > > > > > Chris
> > > > > > >
> > > > > > > On Wed, Feb 4, 2015 at 11:57 AM, Milinda Pathirage <
> > > > > > mpathira@umail.iu.edu>
> > > > > > > wrote:
> > > > > > >
> > > > > > >> Hi Chris,
> > > > > > >>
> > > > > > >> Hope we no longer need the SQL API. I'll create a RB for
> Calcite
> > > > > > >> integration.
> > > > > > >>
> > > > > > >> Thanks
> > > > > > >> Milinda
> > > > > > >>
> > > > > > >> On Wed, Feb 4, 2015 at 1:31 PM, Chris Riccomini <
> > > > > criccomini@apache.org>
> > > > > > >> wrote:
> > > > > > >>
> > > > > > >> > I think so. There was some RB downtime, but it just got
> fixed.
> > > Yi,
> > > > > > >> Navina,
> > > > > > >> > Milinda, can you make sure your JIRAs have up to date RBs?
> > > > > > >> >
> > > > > > >> > On Wed, Feb 4, 2015 at 10:24 AM, sriram <
> sriram.sub@gmail.com
> > >
> > > > > wrote:
> > > > > > >> >
> > > > > > >> > > Can we have updated RBs for all the three sub tasks before
> > we
> > > > > > commit?
> > > > > > >> > This
> > > > > > >> > > would help us to review even after we commit.
> > > > > > >> > >
> > > > > > >> > > On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini <
> > > > > > >> criccomini@apache.org>
> > > > > > >> > > wrote:
> > > > > > >> > >
> > > > > > >> > > > Hey all,
> > > > > > >> > > >
> > > > > > >> > > > Yi, Navina, and Milinda have been working on SAMZA-390
> > > > > sub-tickets
> > > > > > >> > > related
> > > > > > >> > > > to SQL operators. We're getting to the point where the
> > > amount
> > > > of
> > > > > > work
> > > > > > >> > > > floating around is quite large, and some tickets build
> off
> > > of
> > > > > > others.
> > > > > > >> > > >
> > > > > > >> > > > I'm proposing that we commit this work into a samza-sql
> > > > > submodule
> > > > > > on
> > > > > > >> > > > master, and treat this module as experimental. I want to
> > > avoid
> > > > > > >> > branches,
> > > > > > >> > > > and I also want to avoid revision control over JIRA.
> This
> > > > means
> > > > > > that
> > > > > > >> > > there
> > > > > > >> > > > will probably be a fair amount of commits/JIRAs on this
> > > module
> > > > > as
> > > > > > we
> > > > > > >> > > > iterate, but I think that's OK.
> > > > > > >> > > >
> > > > > > >> > > > Does this sound good to everyone?
> > > > > > >> > > >
> > > > > > >> > > > Cheers,
> > > > > > >> > > > Chris
> > > > > > >> > > >
> > > > > > >> > >
> > > > > > >> >
> > > > > > >>
> > > > > > >>
> > > > > > >>
> > > > > > >> --
> > > > > > >> Milinda Pathirage
> > > > > > >>
> > > > > > >> PhD Student | Research Assistant
> > > > > > >> School of Informatics and Computing | Data to Insight Center
> > > > > > >> Indiana University
> > > > > > >>
> > > > > > >> twitter: milindalakmal
> > > > > > >> skype: milinda.pathirage
> > > > > > >> blog: http://milinda.pathirage.org
> > > > > > >>
> > > > > >
> > > > >
> > > >
> > >
> >
>
--
Milinda Pathirage
PhD Student | Research Assistant
School of Informatics and Computing | Data to Insight Center
Indiana University
twitter: milindalakmal
skype: milinda.pathirage
blog: http://milinda.pathirage.org
Re: [DISCUSS] SQL workflow
Posted by Chris Riccomini <cr...@apache.org>.
Cool. Looks like we've got consensus. I'll move forward with RTC on some of
the early SAMZA-390 tickets. (SAMZA-482, SAMZA-483, SAMZA-484)
On Mon, Feb 9, 2015 at 8:35 AM, Yan Fang <ya...@gmail.com> wrote:
> +1 on this.
>
> Fang, Yan
> yanfang724@gmail.com
> +1 (206) 849-4108
>
> On Fri, Feb 6, 2015 at 4:38 PM, Chris Riccomini <cr...@apache.org>
> wrote:
>
> > Hey all,
> >
> > Are we +1 on this? I think Jakob was the only one who was curious about
> it.
> >
> > Cheers,
> > Chris
> >
> > On Thu, Feb 5, 2015 at 1:22 PM, Yi Pan <ni...@gmail.com> wrote:
> >
> > > Hi, Jakob,
> > >
> > >
> > > > > Eh? Not sure what this means...
> > > >
> > > > I mean SAMZA-484 depends on SAMZA-482, and neither are committed. So
> > > Navina
> > > > is having to post Yi's patch, as well as her own, on the JIRA. It
> makes
> > > it
> > > > really hard to do code reviews because you can't tell whether Yi made
> > the
> > > > changes or Navina did.
> > > >
> > > >
> > > Just to add to the point. It is also difficult to always see a long
> list
> > of
> > > changed files if the RB request is always based on the master. It is
> > > possible to have RB request based on another RB request (I have tried
> it
> > > before). But what happens if the base RB request is
> cancelled/discarded?
> > RB
> > > is not designed to track the revision changes in a dependency chain.
> > >
> > >
> > >
> > > > Cheers,
> > > > Chris
> > > >
> > > > On Thu, Feb 5, 2015 at 12:16 PM, Jakob Homan <jg...@gmail.com>
> > wrote:
> > > >
> > > > > > I want to avoid branches,
> > > > > Just curious, any reason for this?
> > > > >
> > > > > > and I also want to avoid revision control over JIRA
> > > > > Eh? Not sure what this means...
> > > > >
> > > > > Thanks,
> > > > > jg
> > > > >
> > > > > On 4 February 2015 at 17:11, Chris Riccomini <
> criccomini@apache.org>
> > > > > wrote:
> > > > > > Hey all,
> > > > > >
> > > > > > @Jakob, yeah I was thinking we'll follow our normal flow. RTC. I
> > just
> > > > > > wanted to set expectation that the code committed might be not up
> > to
> > > > our
> > > > > > normal quality initially (missing docs, no tests, etc). Until the
> > > > quality
> > > > > > is raised, we should think of this module as experimental.
> > > > > >
> > > > > > @Milinda, awesome! Thanks. :)
> > > > > >
> > > > > > Cheers,
> > > > > > Chris
> > > > > >
> > > > > > On Wed, Feb 4, 2015 at 11:57 AM, Milinda Pathirage <
> > > > > mpathira@umail.iu.edu>
> > > > > > wrote:
> > > > > >
> > > > > >> Hi Chris,
> > > > > >>
> > > > > >> Hope we no longer need the SQL API. I'll create a RB for Calcite
> > > > > >> integration.
> > > > > >>
> > > > > >> Thanks
> > > > > >> Milinda
> > > > > >>
> > > > > >> On Wed, Feb 4, 2015 at 1:31 PM, Chris Riccomini <
> > > > criccomini@apache.org>
> > > > > >> wrote:
> > > > > >>
> > > > > >> > I think so. There was some RB downtime, but it just got fixed.
> > Yi,
> > > > > >> Navina,
> > > > > >> > Milinda, can you make sure your JIRAs have up to date RBs?
> > > > > >> >
> > > > > >> > On Wed, Feb 4, 2015 at 10:24 AM, sriram <sriram.sub@gmail.com
> >
> > > > wrote:
> > > > > >> >
> > > > > >> > > Can we have updated RBs for all the three sub tasks before
> we
> > > > > commit?
> > > > > >> > This
> > > > > >> > > would help us to review even after we commit.
> > > > > >> > >
> > > > > >> > > On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini <
> > > > > >> criccomini@apache.org>
> > > > > >> > > wrote:
> > > > > >> > >
> > > > > >> > > > Hey all,
> > > > > >> > > >
> > > > > >> > > > Yi, Navina, and Milinda have been working on SAMZA-390
> > > > sub-tickets
> > > > > >> > > related
> > > > > >> > > > to SQL operators. We're getting to the point where the
> > amount
> > > of
> > > > > work
> > > > > >> > > > floating around is quite large, and some tickets build off
> > of
> > > > > others.
> > > > > >> > > >
> > > > > >> > > > I'm proposing that we commit this work into a samza-sql
> > > > submodule
> > > > > on
> > > > > >> > > > master, and treat this module as experimental. I want to
> > avoid
> > > > > >> > branches,
> > > > > >> > > > and I also want to avoid revision control over JIRA. This
> > > means
> > > > > that
> > > > > >> > > there
> > > > > >> > > > will probably be a fair amount of commits/JIRAs on this
> > module
> > > > as
> > > > > we
> > > > > >> > > > iterate, but I think that's OK.
> > > > > >> > > >
> > > > > >> > > > Does this sound good to everyone?
> > > > > >> > > >
> > > > > >> > > > Cheers,
> > > > > >> > > > Chris
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> --
> > > > > >> Milinda Pathirage
> > > > > >>
> > > > > >> PhD Student | Research Assistant
> > > > > >> School of Informatics and Computing | Data to Insight Center
> > > > > >> Indiana University
> > > > > >>
> > > > > >> twitter: milindalakmal
> > > > > >> skype: milinda.pathirage
> > > > > >> blog: http://milinda.pathirage.org
> > > > > >>
> > > > >
> > > >
> > >
> >
>
Re: [DISCUSS] SQL workflow
Posted by Yan Fang <ya...@gmail.com>.
+1 on this.
Fang, Yan
yanfang724@gmail.com
+1 (206) 849-4108
On Fri, Feb 6, 2015 at 4:38 PM, Chris Riccomini <cr...@apache.org>
wrote:
> Hey all,
>
> Are we +1 on this? I think Jakob was the only one who was curious about it.
>
> Cheers,
> Chris
>
> On Thu, Feb 5, 2015 at 1:22 PM, Yi Pan <ni...@gmail.com> wrote:
>
> > Hi, Jakob,
> >
> >
> > > > Eh? Not sure what this means...
> > >
> > > I mean SAMZA-484 depends on SAMZA-482, and neither are committed. So
> > Navina
> > > is having to post Yi's patch, as well as her own, on the JIRA. It makes
> > it
> > > really hard to do code reviews because you can't tell whether Yi made
> the
> > > changes or Navina did.
> > >
> > >
> > Just to add to the point. It is also difficult to always see a long list
> of
> > changed files if the RB request is always based on the master. It is
> > possible to have RB request based on another RB request (I have tried it
> > before). But what happens if the base RB request is cancelled/discarded?
> RB
> > is not designed to track the revision changes in a dependency chain.
> >
> >
> >
> > > Cheers,
> > > Chris
> > >
> > > On Thu, Feb 5, 2015 at 12:16 PM, Jakob Homan <jg...@gmail.com>
> wrote:
> > >
> > > > > I want to avoid branches,
> > > > Just curious, any reason for this?
> > > >
> > > > > and I also want to avoid revision control over JIRA
> > > > Eh? Not sure what this means...
> > > >
> > > > Thanks,
> > > > jg
> > > >
> > > > On 4 February 2015 at 17:11, Chris Riccomini <cr...@apache.org>
> > > > wrote:
> > > > > Hey all,
> > > > >
> > > > > @Jakob, yeah I was thinking we'll follow our normal flow. RTC. I
> just
> > > > > wanted to set expectation that the code committed might be not up
> to
> > > our
> > > > > normal quality initially (missing docs, no tests, etc). Until the
> > > quality
> > > > > is raised, we should think of this module as experimental.
> > > > >
> > > > > @Milinda, awesome! Thanks. :)
> > > > >
> > > > > Cheers,
> > > > > Chris
> > > > >
> > > > > On Wed, Feb 4, 2015 at 11:57 AM, Milinda Pathirage <
> > > > mpathira@umail.iu.edu>
> > > > > wrote:
> > > > >
> > > > >> Hi Chris,
> > > > >>
> > > > >> Hope we no longer need the SQL API. I'll create a RB for Calcite
> > > > >> integration.
> > > > >>
> > > > >> Thanks
> > > > >> Milinda
> > > > >>
> > > > >> On Wed, Feb 4, 2015 at 1:31 PM, Chris Riccomini <
> > > criccomini@apache.org>
> > > > >> wrote:
> > > > >>
> > > > >> > I think so. There was some RB downtime, but it just got fixed.
> Yi,
> > > > >> Navina,
> > > > >> > Milinda, can you make sure your JIRAs have up to date RBs?
> > > > >> >
> > > > >> > On Wed, Feb 4, 2015 at 10:24 AM, sriram <sr...@gmail.com>
> > > wrote:
> > > > >> >
> > > > >> > > Can we have updated RBs for all the three sub tasks before we
> > > > commit?
> > > > >> > This
> > > > >> > > would help us to review even after we commit.
> > > > >> > >
> > > > >> > > On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini <
> > > > >> criccomini@apache.org>
> > > > >> > > wrote:
> > > > >> > >
> > > > >> > > > Hey all,
> > > > >> > > >
> > > > >> > > > Yi, Navina, and Milinda have been working on SAMZA-390
> > > sub-tickets
> > > > >> > > related
> > > > >> > > > to SQL operators. We're getting to the point where the
> amount
> > of
> > > > work
> > > > >> > > > floating around is quite large, and some tickets build off
> of
> > > > others.
> > > > >> > > >
> > > > >> > > > I'm proposing that we commit this work into a samza-sql
> > > submodule
> > > > on
> > > > >> > > > master, and treat this module as experimental. I want to
> avoid
> > > > >> > branches,
> > > > >> > > > and I also want to avoid revision control over JIRA. This
> > means
> > > > that
> > > > >> > > there
> > > > >> > > > will probably be a fair amount of commits/JIRAs on this
> module
> > > as
> > > > we
> > > > >> > > > iterate, but I think that's OK.
> > > > >> > > >
> > > > >> > > > Does this sound good to everyone?
> > > > >> > > >
> > > > >> > > > Cheers,
> > > > >> > > > Chris
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > > >>
> > > > >>
> > > > >> --
> > > > >> Milinda Pathirage
> > > > >>
> > > > >> PhD Student | Research Assistant
> > > > >> School of Informatics and Computing | Data to Insight Center
> > > > >> Indiana University
> > > > >>
> > > > >> twitter: milindalakmal
> > > > >> skype: milinda.pathirage
> > > > >> blog: http://milinda.pathirage.org
> > > > >>
> > > >
> > >
> >
>
Re: [DISCUSS] SQL workflow
Posted by Jakob Homan <jg...@gmail.com>.
+1.
On 6 February 2015 at 16:38, Chris Riccomini <cr...@apache.org> wrote:
> Hey all,
>
> Are we +1 on this? I think Jakob was the only one who was curious about it.
>
> Cheers,
> Chris
>
> On Thu, Feb 5, 2015 at 1:22 PM, Yi Pan <ni...@gmail.com> wrote:
>
>> Hi, Jakob,
>>
>>
>> > > Eh? Not sure what this means...
>> >
>> > I mean SAMZA-484 depends on SAMZA-482, and neither are committed. So
>> Navina
>> > is having to post Yi's patch, as well as her own, on the JIRA. It makes
>> it
>> > really hard to do code reviews because you can't tell whether Yi made the
>> > changes or Navina did.
>> >
>> >
>> Just to add to the point. It is also difficult to always see a long list of
>> changed files if the RB request is always based on the master. It is
>> possible to have RB request based on another RB request (I have tried it
>> before). But what happens if the base RB request is cancelled/discarded? RB
>> is not designed to track the revision changes in a dependency chain.
>>
>>
>>
>> > Cheers,
>> > Chris
>> >
>> > On Thu, Feb 5, 2015 at 12:16 PM, Jakob Homan <jg...@gmail.com> wrote:
>> >
>> > > > I want to avoid branches,
>> > > Just curious, any reason for this?
>> > >
>> > > > and I also want to avoid revision control over JIRA
>> > > Eh? Not sure what this means...
>> > >
>> > > Thanks,
>> > > jg
>> > >
>> > > On 4 February 2015 at 17:11, Chris Riccomini <cr...@apache.org>
>> > > wrote:
>> > > > Hey all,
>> > > >
>> > > > @Jakob, yeah I was thinking we'll follow our normal flow. RTC. I just
>> > > > wanted to set expectation that the code committed might be not up to
>> > our
>> > > > normal quality initially (missing docs, no tests, etc). Until the
>> > quality
>> > > > is raised, we should think of this module as experimental.
>> > > >
>> > > > @Milinda, awesome! Thanks. :)
>> > > >
>> > > > Cheers,
>> > > > Chris
>> > > >
>> > > > On Wed, Feb 4, 2015 at 11:57 AM, Milinda Pathirage <
>> > > mpathira@umail.iu.edu>
>> > > > wrote:
>> > > >
>> > > >> Hi Chris,
>> > > >>
>> > > >> Hope we no longer need the SQL API. I'll create a RB for Calcite
>> > > >> integration.
>> > > >>
>> > > >> Thanks
>> > > >> Milinda
>> > > >>
>> > > >> On Wed, Feb 4, 2015 at 1:31 PM, Chris Riccomini <
>> > criccomini@apache.org>
>> > > >> wrote:
>> > > >>
>> > > >> > I think so. There was some RB downtime, but it just got fixed. Yi,
>> > > >> Navina,
>> > > >> > Milinda, can you make sure your JIRAs have up to date RBs?
>> > > >> >
>> > > >> > On Wed, Feb 4, 2015 at 10:24 AM, sriram <sr...@gmail.com>
>> > wrote:
>> > > >> >
>> > > >> > > Can we have updated RBs for all the three sub tasks before we
>> > > commit?
>> > > >> > This
>> > > >> > > would help us to review even after we commit.
>> > > >> > >
>> > > >> > > On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini <
>> > > >> criccomini@apache.org>
>> > > >> > > wrote:
>> > > >> > >
>> > > >> > > > Hey all,
>> > > >> > > >
>> > > >> > > > Yi, Navina, and Milinda have been working on SAMZA-390
>> > sub-tickets
>> > > >> > > related
>> > > >> > > > to SQL operators. We're getting to the point where the amount
>> of
>> > > work
>> > > >> > > > floating around is quite large, and some tickets build off of
>> > > others.
>> > > >> > > >
>> > > >> > > > I'm proposing that we commit this work into a samza-sql
>> > submodule
>> > > on
>> > > >> > > > master, and treat this module as experimental. I want to avoid
>> > > >> > branches,
>> > > >> > > > and I also want to avoid revision control over JIRA. This
>> means
>> > > that
>> > > >> > > there
>> > > >> > > > will probably be a fair amount of commits/JIRAs on this module
>> > as
>> > > we
>> > > >> > > > iterate, but I think that's OK.
>> > > >> > > >
>> > > >> > > > Does this sound good to everyone?
>> > > >> > > >
>> > > >> > > > Cheers,
>> > > >> > > > Chris
>> > > >> > > >
>> > > >> > >
>> > > >> >
>> > > >>
>> > > >>
>> > > >>
>> > > >> --
>> > > >> Milinda Pathirage
>> > > >>
>> > > >> PhD Student | Research Assistant
>> > > >> School of Informatics and Computing | Data to Insight Center
>> > > >> Indiana University
>> > > >>
>> > > >> twitter: milindalakmal
>> > > >> skype: milinda.pathirage
>> > > >> blog: http://milinda.pathirage.org
>> > > >>
>> > >
>> >
>>
Re: [DISCUSS] SQL workflow
Posted by Chris Riccomini <cr...@apache.org>.
Hey all,
Are we +1 on this? I think Jakob was the only one who was curious about it.
Cheers,
Chris
On Thu, Feb 5, 2015 at 1:22 PM, Yi Pan <ni...@gmail.com> wrote:
> Hi, Jakob,
>
>
> > > Eh? Not sure what this means...
> >
> > I mean SAMZA-484 depends on SAMZA-482, and neither are committed. So
> Navina
> > is having to post Yi's patch, as well as her own, on the JIRA. It makes
> it
> > really hard to do code reviews because you can't tell whether Yi made the
> > changes or Navina did.
> >
> >
> Just to add to the point. It is also difficult to always see a long list of
> changed files if the RB request is always based on the master. It is
> possible to have RB request based on another RB request (I have tried it
> before). But what happens if the base RB request is cancelled/discarded? RB
> is not designed to track the revision changes in a dependency chain.
>
>
>
> > Cheers,
> > Chris
> >
> > On Thu, Feb 5, 2015 at 12:16 PM, Jakob Homan <jg...@gmail.com> wrote:
> >
> > > > I want to avoid branches,
> > > Just curious, any reason for this?
> > >
> > > > and I also want to avoid revision control over JIRA
> > > Eh? Not sure what this means...
> > >
> > > Thanks,
> > > jg
> > >
> > > On 4 February 2015 at 17:11, Chris Riccomini <cr...@apache.org>
> > > wrote:
> > > > Hey all,
> > > >
> > > > @Jakob, yeah I was thinking we'll follow our normal flow. RTC. I just
> > > > wanted to set expectation that the code committed might be not up to
> > our
> > > > normal quality initially (missing docs, no tests, etc). Until the
> > quality
> > > > is raised, we should think of this module as experimental.
> > > >
> > > > @Milinda, awesome! Thanks. :)
> > > >
> > > > Cheers,
> > > > Chris
> > > >
> > > > On Wed, Feb 4, 2015 at 11:57 AM, Milinda Pathirage <
> > > mpathira@umail.iu.edu>
> > > > wrote:
> > > >
> > > >> Hi Chris,
> > > >>
> > > >> Hope we no longer need the SQL API. I'll create a RB for Calcite
> > > >> integration.
> > > >>
> > > >> Thanks
> > > >> Milinda
> > > >>
> > > >> On Wed, Feb 4, 2015 at 1:31 PM, Chris Riccomini <
> > criccomini@apache.org>
> > > >> wrote:
> > > >>
> > > >> > I think so. There was some RB downtime, but it just got fixed. Yi,
> > > >> Navina,
> > > >> > Milinda, can you make sure your JIRAs have up to date RBs?
> > > >> >
> > > >> > On Wed, Feb 4, 2015 at 10:24 AM, sriram <sr...@gmail.com>
> > wrote:
> > > >> >
> > > >> > > Can we have updated RBs for all the three sub tasks before we
> > > commit?
> > > >> > This
> > > >> > > would help us to review even after we commit.
> > > >> > >
> > > >> > > On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini <
> > > >> criccomini@apache.org>
> > > >> > > wrote:
> > > >> > >
> > > >> > > > Hey all,
> > > >> > > >
> > > >> > > > Yi, Navina, and Milinda have been working on SAMZA-390
> > sub-tickets
> > > >> > > related
> > > >> > > > to SQL operators. We're getting to the point where the amount
> of
> > > work
> > > >> > > > floating around is quite large, and some tickets build off of
> > > others.
> > > >> > > >
> > > >> > > > I'm proposing that we commit this work into a samza-sql
> > submodule
> > > on
> > > >> > > > master, and treat this module as experimental. I want to avoid
> > > >> > branches,
> > > >> > > > and I also want to avoid revision control over JIRA. This
> means
> > > that
> > > >> > > there
> > > >> > > > will probably be a fair amount of commits/JIRAs on this module
> > as
> > > we
> > > >> > > > iterate, but I think that's OK.
> > > >> > > >
> > > >> > > > Does this sound good to everyone?
> > > >> > > >
> > > >> > > > Cheers,
> > > >> > > > Chris
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> Milinda Pathirage
> > > >>
> > > >> PhD Student | Research Assistant
> > > >> School of Informatics and Computing | Data to Insight Center
> > > >> Indiana University
> > > >>
> > > >> twitter: milindalakmal
> > > >> skype: milinda.pathirage
> > > >> blog: http://milinda.pathirage.org
> > > >>
> > >
> >
>
Re: [DISCUSS] SQL workflow
Posted by Yi Pan <ni...@gmail.com>.
Hi, Jakob,
> > Eh? Not sure what this means...
>
> I mean SAMZA-484 depends on SAMZA-482, and neither are committed. So Navina
> is having to post Yi's patch, as well as her own, on the JIRA. It makes it
> really hard to do code reviews because you can't tell whether Yi made the
> changes or Navina did.
>
>
Just to add to the point. It is also difficult to always see a long list of
changed files if the RB request is always based on the master. It is
possible to have RB request based on another RB request (I have tried it
before). But what happens if the base RB request is cancelled/discarded? RB
is not designed to track the revision changes in a dependency chain.
> Cheers,
> Chris
>
> On Thu, Feb 5, 2015 at 12:16 PM, Jakob Homan <jg...@gmail.com> wrote:
>
> > > I want to avoid branches,
> > Just curious, any reason for this?
> >
> > > and I also want to avoid revision control over JIRA
> > Eh? Not sure what this means...
> >
> > Thanks,
> > jg
> >
> > On 4 February 2015 at 17:11, Chris Riccomini <cr...@apache.org>
> > wrote:
> > > Hey all,
> > >
> > > @Jakob, yeah I was thinking we'll follow our normal flow. RTC. I just
> > > wanted to set expectation that the code committed might be not up to
> our
> > > normal quality initially (missing docs, no tests, etc). Until the
> quality
> > > is raised, we should think of this module as experimental.
> > >
> > > @Milinda, awesome! Thanks. :)
> > >
> > > Cheers,
> > > Chris
> > >
> > > On Wed, Feb 4, 2015 at 11:57 AM, Milinda Pathirage <
> > mpathira@umail.iu.edu>
> > > wrote:
> > >
> > >> Hi Chris,
> > >>
> > >> Hope we no longer need the SQL API. I'll create a RB for Calcite
> > >> integration.
> > >>
> > >> Thanks
> > >> Milinda
> > >>
> > >> On Wed, Feb 4, 2015 at 1:31 PM, Chris Riccomini <
> criccomini@apache.org>
> > >> wrote:
> > >>
> > >> > I think so. There was some RB downtime, but it just got fixed. Yi,
> > >> Navina,
> > >> > Milinda, can you make sure your JIRAs have up to date RBs?
> > >> >
> > >> > On Wed, Feb 4, 2015 at 10:24 AM, sriram <sr...@gmail.com>
> wrote:
> > >> >
> > >> > > Can we have updated RBs for all the three sub tasks before we
> > commit?
> > >> > This
> > >> > > would help us to review even after we commit.
> > >> > >
> > >> > > On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini <
> > >> criccomini@apache.org>
> > >> > > wrote:
> > >> > >
> > >> > > > Hey all,
> > >> > > >
> > >> > > > Yi, Navina, and Milinda have been working on SAMZA-390
> sub-tickets
> > >> > > related
> > >> > > > to SQL operators. We're getting to the point where the amount of
> > work
> > >> > > > floating around is quite large, and some tickets build off of
> > others.
> > >> > > >
> > >> > > > I'm proposing that we commit this work into a samza-sql
> submodule
> > on
> > >> > > > master, and treat this module as experimental. I want to avoid
> > >> > branches,
> > >> > > > and I also want to avoid revision control over JIRA. This means
> > that
> > >> > > there
> > >> > > > will probably be a fair amount of commits/JIRAs on this module
> as
> > we
> > >> > > > iterate, but I think that's OK.
> > >> > > >
> > >> > > > Does this sound good to everyone?
> > >> > > >
> > >> > > > Cheers,
> > >> > > > Chris
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >>
> > >>
> > >> --
> > >> Milinda Pathirage
> > >>
> > >> PhD Student | Research Assistant
> > >> School of Informatics and Computing | Data to Insight Center
> > >> Indiana University
> > >>
> > >> twitter: milindalakmal
> > >> skype: milinda.pathirage
> > >> blog: http://milinda.pathirage.org
> > >>
> >
>
Re: [DISCUSS] SQL workflow
Posted by Chris Riccomini <cr...@apache.org>.
Hey Jakob,
> Just curious, any reason for this?
Every time I've seen major feature work done on a branch, the branch
diverges from master to the point where no one feels comfortable merging
it, or it gets abandoned, or something. Example A would be the
transactional Kafka branch. Example B would be the Kafka offset manager
branch, which Joel had to totally re-do.
> Eh? Not sure what this means...
I mean SAMZA-484 depends on SAMZA-482, and neither are committed. So Navina
is having to post Yi's patch, as well as her own, on the JIRA. It makes it
really hard to do code reviews because you can't tell whether Yi made the
changes or Navina did.
Cheers,
Chris
On Thu, Feb 5, 2015 at 12:16 PM, Jakob Homan <jg...@gmail.com> wrote:
> > I want to avoid branches,
> Just curious, any reason for this?
>
> > and I also want to avoid revision control over JIRA
> Eh? Not sure what this means...
>
> Thanks,
> jg
>
> On 4 February 2015 at 17:11, Chris Riccomini <cr...@apache.org>
> wrote:
> > Hey all,
> >
> > @Jakob, yeah I was thinking we'll follow our normal flow. RTC. I just
> > wanted to set expectation that the code committed might be not up to our
> > normal quality initially (missing docs, no tests, etc). Until the quality
> > is raised, we should think of this module as experimental.
> >
> > @Milinda, awesome! Thanks. :)
> >
> > Cheers,
> > Chris
> >
> > On Wed, Feb 4, 2015 at 11:57 AM, Milinda Pathirage <
> mpathira@umail.iu.edu>
> > wrote:
> >
> >> Hi Chris,
> >>
> >> Hope we no longer need the SQL API. I'll create a RB for Calcite
> >> integration.
> >>
> >> Thanks
> >> Milinda
> >>
> >> On Wed, Feb 4, 2015 at 1:31 PM, Chris Riccomini <cr...@apache.org>
> >> wrote:
> >>
> >> > I think so. There was some RB downtime, but it just got fixed. Yi,
> >> Navina,
> >> > Milinda, can you make sure your JIRAs have up to date RBs?
> >> >
> >> > On Wed, Feb 4, 2015 at 10:24 AM, sriram <sr...@gmail.com> wrote:
> >> >
> >> > > Can we have updated RBs for all the three sub tasks before we
> commit?
> >> > This
> >> > > would help us to review even after we commit.
> >> > >
> >> > > On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini <
> >> criccomini@apache.org>
> >> > > wrote:
> >> > >
> >> > > > Hey all,
> >> > > >
> >> > > > Yi, Navina, and Milinda have been working on SAMZA-390 sub-tickets
> >> > > related
> >> > > > to SQL operators. We're getting to the point where the amount of
> work
> >> > > > floating around is quite large, and some tickets build off of
> others.
> >> > > >
> >> > > > I'm proposing that we commit this work into a samza-sql submodule
> on
> >> > > > master, and treat this module as experimental. I want to avoid
> >> > branches,
> >> > > > and I also want to avoid revision control over JIRA. This means
> that
> >> > > there
> >> > > > will probably be a fair amount of commits/JIRAs on this module as
> we
> >> > > > iterate, but I think that's OK.
> >> > > >
> >> > > > Does this sound good to everyone?
> >> > > >
> >> > > > Cheers,
> >> > > > Chris
> >> > > >
> >> > >
> >> >
> >>
> >>
> >>
> >> --
> >> Milinda Pathirage
> >>
> >> PhD Student | Research Assistant
> >> School of Informatics and Computing | Data to Insight Center
> >> Indiana University
> >>
> >> twitter: milindalakmal
> >> skype: milinda.pathirage
> >> blog: http://milinda.pathirage.org
> >>
>
Re: [DISCUSS] SQL workflow
Posted by Jakob Homan <jg...@gmail.com>.
> I want to avoid branches,
Just curious, any reason for this?
> and I also want to avoid revision control over JIRA
Eh? Not sure what this means...
Thanks,
jg
On 4 February 2015 at 17:11, Chris Riccomini <cr...@apache.org> wrote:
> Hey all,
>
> @Jakob, yeah I was thinking we'll follow our normal flow. RTC. I just
> wanted to set expectation that the code committed might be not up to our
> normal quality initially (missing docs, no tests, etc). Until the quality
> is raised, we should think of this module as experimental.
>
> @Milinda, awesome! Thanks. :)
>
> Cheers,
> Chris
>
> On Wed, Feb 4, 2015 at 11:57 AM, Milinda Pathirage <mp...@umail.iu.edu>
> wrote:
>
>> Hi Chris,
>>
>> Hope we no longer need the SQL API. I'll create a RB for Calcite
>> integration.
>>
>> Thanks
>> Milinda
>>
>> On Wed, Feb 4, 2015 at 1:31 PM, Chris Riccomini <cr...@apache.org>
>> wrote:
>>
>> > I think so. There was some RB downtime, but it just got fixed. Yi,
>> Navina,
>> > Milinda, can you make sure your JIRAs have up to date RBs?
>> >
>> > On Wed, Feb 4, 2015 at 10:24 AM, sriram <sr...@gmail.com> wrote:
>> >
>> > > Can we have updated RBs for all the three sub tasks before we commit?
>> > This
>> > > would help us to review even after we commit.
>> > >
>> > > On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini <
>> criccomini@apache.org>
>> > > wrote:
>> > >
>> > > > Hey all,
>> > > >
>> > > > Yi, Navina, and Milinda have been working on SAMZA-390 sub-tickets
>> > > related
>> > > > to SQL operators. We're getting to the point where the amount of work
>> > > > floating around is quite large, and some tickets build off of others.
>> > > >
>> > > > I'm proposing that we commit this work into a samza-sql submodule on
>> > > > master, and treat this module as experimental. I want to avoid
>> > branches,
>> > > > and I also want to avoid revision control over JIRA. This means that
>> > > there
>> > > > will probably be a fair amount of commits/JIRAs on this module as we
>> > > > iterate, but I think that's OK.
>> > > >
>> > > > Does this sound good to everyone?
>> > > >
>> > > > Cheers,
>> > > > Chris
>> > > >
>> > >
>> >
>>
>>
>>
>> --
>> Milinda Pathirage
>>
>> PhD Student | Research Assistant
>> School of Informatics and Computing | Data to Insight Center
>> Indiana University
>>
>> twitter: milindalakmal
>> skype: milinda.pathirage
>> blog: http://milinda.pathirage.org
>>
Re: [DISCUSS] SQL workflow
Posted by Chris Riccomini <cr...@apache.org>.
Hey all,
@Jakob, yeah I was thinking we'll follow our normal flow. RTC. I just
wanted to set expectation that the code committed might be not up to our
normal quality initially (missing docs, no tests, etc). Until the quality
is raised, we should think of this module as experimental.
@Milinda, awesome! Thanks. :)
Cheers,
Chris
On Wed, Feb 4, 2015 at 11:57 AM, Milinda Pathirage <mp...@umail.iu.edu>
wrote:
> Hi Chris,
>
> Hope we no longer need the SQL API. I'll create a RB for Calcite
> integration.
>
> Thanks
> Milinda
>
> On Wed, Feb 4, 2015 at 1:31 PM, Chris Riccomini <cr...@apache.org>
> wrote:
>
> > I think so. There was some RB downtime, but it just got fixed. Yi,
> Navina,
> > Milinda, can you make sure your JIRAs have up to date RBs?
> >
> > On Wed, Feb 4, 2015 at 10:24 AM, sriram <sr...@gmail.com> wrote:
> >
> > > Can we have updated RBs for all the three sub tasks before we commit?
> > This
> > > would help us to review even after we commit.
> > >
> > > On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini <
> criccomini@apache.org>
> > > wrote:
> > >
> > > > Hey all,
> > > >
> > > > Yi, Navina, and Milinda have been working on SAMZA-390 sub-tickets
> > > related
> > > > to SQL operators. We're getting to the point where the amount of work
> > > > floating around is quite large, and some tickets build off of others.
> > > >
> > > > I'm proposing that we commit this work into a samza-sql submodule on
> > > > master, and treat this module as experimental. I want to avoid
> > branches,
> > > > and I also want to avoid revision control over JIRA. This means that
> > > there
> > > > will probably be a fair amount of commits/JIRAs on this module as we
> > > > iterate, but I think that's OK.
> > > >
> > > > Does this sound good to everyone?
> > > >
> > > > Cheers,
> > > > Chris
> > > >
> > >
> >
>
>
>
> --
> Milinda Pathirage
>
> PhD Student | Research Assistant
> School of Informatics and Computing | Data to Insight Center
> Indiana University
>
> twitter: milindalakmal
> skype: milinda.pathirage
> blog: http://milinda.pathirage.org
>
Re: [DISCUSS] SQL workflow
Posted by Milinda Pathirage <mp...@umail.iu.edu>.
Hi Chris,
Hope we no longer need the SQL API. I'll create a RB for Calcite
integration.
Thanks
Milinda
On Wed, Feb 4, 2015 at 1:31 PM, Chris Riccomini <cr...@apache.org>
wrote:
> I think so. There was some RB downtime, but it just got fixed. Yi, Navina,
> Milinda, can you make sure your JIRAs have up to date RBs?
>
> On Wed, Feb 4, 2015 at 10:24 AM, sriram <sr...@gmail.com> wrote:
>
> > Can we have updated RBs for all the three sub tasks before we commit?
> This
> > would help us to review even after we commit.
> >
> > On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini <cr...@apache.org>
> > wrote:
> >
> > > Hey all,
> > >
> > > Yi, Navina, and Milinda have been working on SAMZA-390 sub-tickets
> > related
> > > to SQL operators. We're getting to the point where the amount of work
> > > floating around is quite large, and some tickets build off of others.
> > >
> > > I'm proposing that we commit this work into a samza-sql submodule on
> > > master, and treat this module as experimental. I want to avoid
> branches,
> > > and I also want to avoid revision control over JIRA. This means that
> > there
> > > will probably be a fair amount of commits/JIRAs on this module as we
> > > iterate, but I think that's OK.
> > >
> > > Does this sound good to everyone?
> > >
> > > Cheers,
> > > Chris
> > >
> >
>
--
Milinda Pathirage
PhD Student | Research Assistant
School of Informatics and Computing | Data to Insight Center
Indiana University
twitter: milindalakmal
skype: milinda.pathirage
blog: http://milinda.pathirage.org
Re: [DISCUSS] SQL workflow
Posted by Chris Riccomini <cr...@apache.org>.
I think so. There was some RB downtime, but it just got fixed. Yi, Navina,
Milinda, can you make sure your JIRAs have up to date RBs?
On Wed, Feb 4, 2015 at 10:24 AM, sriram <sr...@gmail.com> wrote:
> Can we have updated RBs for all the three sub tasks before we commit? This
> would help us to review even after we commit.
>
> On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini <cr...@apache.org>
> wrote:
>
> > Hey all,
> >
> > Yi, Navina, and Milinda have been working on SAMZA-390 sub-tickets
> related
> > to SQL operators. We're getting to the point where the amount of work
> > floating around is quite large, and some tickets build off of others.
> >
> > I'm proposing that we commit this work into a samza-sql submodule on
> > master, and treat this module as experimental. I want to avoid branches,
> > and I also want to avoid revision control over JIRA. This means that
> there
> > will probably be a fair amount of commits/JIRAs on this module as we
> > iterate, but I think that's OK.
> >
> > Does this sound good to everyone?
> >
> > Cheers,
> > Chris
> >
>
Re: [DISCUSS] SQL workflow
Posted by sriram <sr...@gmail.com>.
Can we have updated RBs for all the three sub tasks before we commit? This
would help us to review even after we commit.
On Wed, Feb 4, 2015 at 10:15 AM, Chris Riccomini <cr...@apache.org>
wrote:
> Hey all,
>
> Yi, Navina, and Milinda have been working on SAMZA-390 sub-tickets related
> to SQL operators. We're getting to the point where the amount of work
> floating around is quite large, and some tickets build off of others.
>
> I'm proposing that we commit this work into a samza-sql submodule on
> master, and treat this module as experimental. I want to avoid branches,
> and I also want to avoid revision control over JIRA. This means that there
> will probably be a fair amount of commits/JIRAs on this module as we
> iterate, but I think that's OK.
>
> Does this sound good to everyone?
>
> Cheers,
> Chris
>