You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@asterixdb.apache.org by Yingyi Bu <bu...@gmail.com> on 2017/03/16 16:22:08 UTC
New cluster installation options
Hi dev,
We recently added two cluster installation options and their
documentation is online now:
Ansible: https://ci.apache.org/projects/asterixdb/ansible.html
AWS: https://ci.apache.org/projects/asterixdb/aws.html
With the Ansible option, you can deploy, start, stop, and erase an
AsterixDB cluster from your client with a simple shell scripts.
With the AWS option, you can start/stop an AsterixDB cluster on AWS
with a simple shell scripts. The scripts automate both AWS and AsterixDB
setup.
Managix is marked as "deprecated" and probably is going to retire
soon, as (1) it's not well maintained and (2) some of its functionalities
are already provided by existing tools such as Ansible. It was written at
the time when there're not many automation tools that we can rely on, which
is no longer the case now.
Please share your thoughts and feedbacks.
Thanks!
Best,
Yingyi
Re: New cluster installation options
Posted by Xikui <xi...@uci.edu>.
The library jars will be placed under asterix-installer-0.9.0-binary-assembly/clusters/local/working_dir/library directly.
I think UDF package itself will not be stored on each NCs. Only CC is aware of where the UDFs’ jars are. When CC start, it registers the UDFs with their class loaders [1] and provide the function instances when they are being called[2]. Maybe @abudullah can help to confirm this as well. :)
[1] org.apache.asterix.app.external.ExternalLibraryUtils#setUpExternaLibraries
[2] org.apache.asterix.external.library.ExternalFunction#ExternalFunction
Best,
Xikui
> On Apr 5, 2017, at 3:06 PM, Taewoo Kim <wa...@gmail.com> wrote:
>
> @Gift, @Xikui: could you answer? :-)
>
> Best,
> Taewoo
>
> On Wed, Apr 5, 2017 at 3:01 PM, Yingyi Bu <bu...@gmail.com> wrote:
>
>> In the install library command:
>> managix install -n my_asterix -d feeds -l testlib -p
>> extlibs/asterix-external-data-0.8.7-binary-assembly.zip
>>
>> Does any one know what the resulting file system location on each NC of
>> asterix-external-data-0.8.7-binary-assembly.zip or its enclosed jars is?
>> Will those user defined jars be placed together with the standard AsterixDB
>> binary jars?
>>
>> Best,
>> Yingyi
>>
>>
>> On Wed, Apr 5, 2017 at 7:36 AM, Mike Carey <dt...@gmail.com> wrote:
>>
>>> It's definitely the one that Taewoo and Co plan to use in the short-term.
>>> Hopefully all of the feeds stuff works seamlessly in the cloud too?
>> (They
>>> will be feeding Tweets from one of the Twitter adaptors into *DB and
>> using
>>> UDFs to process them on the way in and in queries.)
>>>
>>>
>>>
>>> On 4/4/17 4:04 PM, Yingyi Bu wrote:
>>>
>>>> Thanks for the feedback.
>>>> It hasn't supported yet. But it should be easy to add.
>>>>
>>>> Is Java UDF support the only missing feature in those two installation
>>>> options?
>>>>
>>>> Unless they don't support it, it may be early to deprecate the managix?
>>>>>>
>>>>> It's deprecated for common use cases -- I'm not sure how many people
>> are
>>>> using the Java UDF?
>>>>
>>>> Best,
>>>> Yingyi
>>>>
>>>>
>>>> On Tue, Apr 4, 2017 at 3:54 PM, Taewoo Kim <wa...@gmail.com> wrote:
>>>>
>>>> Sorry for the late response.
>>>>>
>>>>> Do Ansible and AWS support "installing libraries" like the managix
>> does?
>>>>> Just checked the documentation. I'm talking about this:
>>>>> https://ci.apache.org/projects/asterixdb/udf.html. Unless they don't
>>>>> support it, it may be early to deprecate the managix?
>>>>>
>>>>> Best,
>>>>> Taewoo
>>>>>
>>>>> On Thu, Mar 16, 2017 at 9:22 AM, Yingyi Bu <bu...@gmail.com> wrote:
>>>>>
>>>>> Hi dev,
>>>>>>
>>>>>> We recently added two cluster installation options and their
>>>>>> documentation is online now:
>>>>>>
>>>>>> Ansible: https://ci.apache.org/projects/asterixdb/ansible.html
>>>>>> AWS: https://ci.apache.org/projects/asterixdb/aws.html
>>>>>>
>>>>>> With the Ansible option, you can deploy, start, stop, and erase
>> an
>>>>>> AsterixDB cluster from your client with a simple shell scripts.
>>>>>> With the AWS option, you can start/stop an AsterixDB cluster on
>>>>>> AWS
>>>>>> with a simple shell scripts. The scripts automate both AWS and
>> AsterixDB
>>>>>> setup.
>>>>>>
>>>>>> Managix is marked as "deprecated" and probably is going to
>> retire
>>>>>> soon, as (1) it's not well maintained and (2) some of its
>>>>>> functionalities
>>>>>> are already provided by existing tools such as Ansible. It was written
>>>>>> at
>>>>>> the time when there're not many automation tools that we can rely on,
>>>>>>
>>>>> which
>>>>>
>>>>>> is no longer the case now.
>>>>>>
>>>>>> Please share your thoughts and feedbacks.
>>>>>> Thanks!
>>>>>>
>>>>>> Best,
>>>>>> Yingyi
>>>>>>
>>>>>>
>>>
>>
Re: New cluster installation options
Posted by Taewoo Kim <wa...@gmail.com>.
@Gift, @Xikui: could you answer? :-)
Best,
Taewoo
On Wed, Apr 5, 2017 at 3:01 PM, Yingyi Bu <bu...@gmail.com> wrote:
> In the install library command:
> managix install -n my_asterix -d feeds -l testlib -p
> extlibs/asterix-external-data-0.8.7-binary-assembly.zip
>
> Does any one know what the resulting file system location on each NC of
> asterix-external-data-0.8.7-binary-assembly.zip or its enclosed jars is?
> Will those user defined jars be placed together with the standard AsterixDB
> binary jars?
>
> Best,
> Yingyi
>
>
> On Wed, Apr 5, 2017 at 7:36 AM, Mike Carey <dt...@gmail.com> wrote:
>
> > It's definitely the one that Taewoo and Co plan to use in the short-term.
> > Hopefully all of the feeds stuff works seamlessly in the cloud too?
> (They
> > will be feeding Tweets from one of the Twitter adaptors into *DB and
> using
> > UDFs to process them on the way in and in queries.)
> >
> >
> >
> > On 4/4/17 4:04 PM, Yingyi Bu wrote:
> >
> >> Thanks for the feedback.
> >> It hasn't supported yet. But it should be easy to add.
> >>
> >> Is Java UDF support the only missing feature in those two installation
> >> options?
> >>
> >> Unless they don't support it, it may be early to deprecate the managix?
> >>>>
> >>> It's deprecated for common use cases -- I'm not sure how many people
> are
> >> using the Java UDF?
> >>
> >> Best,
> >> Yingyi
> >>
> >>
> >> On Tue, Apr 4, 2017 at 3:54 PM, Taewoo Kim <wa...@gmail.com> wrote:
> >>
> >> Sorry for the late response.
> >>>
> >>> Do Ansible and AWS support "installing libraries" like the managix
> does?
> >>> Just checked the documentation. I'm talking about this:
> >>> https://ci.apache.org/projects/asterixdb/udf.html. Unless they don't
> >>> support it, it may be early to deprecate the managix?
> >>>
> >>> Best,
> >>> Taewoo
> >>>
> >>> On Thu, Mar 16, 2017 at 9:22 AM, Yingyi Bu <bu...@gmail.com> wrote:
> >>>
> >>> Hi dev,
> >>>>
> >>>> We recently added two cluster installation options and their
> >>>> documentation is online now:
> >>>>
> >>>> Ansible: https://ci.apache.org/projects/asterixdb/ansible.html
> >>>> AWS: https://ci.apache.org/projects/asterixdb/aws.html
> >>>>
> >>>> With the Ansible option, you can deploy, start, stop, and erase
> an
> >>>> AsterixDB cluster from your client with a simple shell scripts.
> >>>> With the AWS option, you can start/stop an AsterixDB cluster on
> >>>> AWS
> >>>> with a simple shell scripts. The scripts automate both AWS and
> AsterixDB
> >>>> setup.
> >>>>
> >>>> Managix is marked as "deprecated" and probably is going to
> retire
> >>>> soon, as (1) it's not well maintained and (2) some of its
> >>>> functionalities
> >>>> are already provided by existing tools such as Ansible. It was written
> >>>> at
> >>>> the time when there're not many automation tools that we can rely on,
> >>>>
> >>> which
> >>>
> >>>> is no longer the case now.
> >>>>
> >>>> Please share your thoughts and feedbacks.
> >>>> Thanks!
> >>>>
> >>>> Best,
> >>>> Yingyi
> >>>>
> >>>>
> >
>
Re: New cluster installation options
Posted by Yingyi Bu <bu...@gmail.com>.
In the install library command:
managix install -n my_asterix -d feeds -l testlib -p
extlibs/asterix-external-data-0.8.7-binary-assembly.zip
Does any one know what the resulting file system location on each NC of
asterix-external-data-0.8.7-binary-assembly.zip or its enclosed jars is?
Will those user defined jars be placed together with the standard AsterixDB
binary jars?
Best,
Yingyi
On Wed, Apr 5, 2017 at 7:36 AM, Mike Carey <dt...@gmail.com> wrote:
> It's definitely the one that Taewoo and Co plan to use in the short-term.
> Hopefully all of the feeds stuff works seamlessly in the cloud too? (They
> will be feeding Tweets from one of the Twitter adaptors into *DB and using
> UDFs to process them on the way in and in queries.)
>
>
>
> On 4/4/17 4:04 PM, Yingyi Bu wrote:
>
>> Thanks for the feedback.
>> It hasn't supported yet. But it should be easy to add.
>>
>> Is Java UDF support the only missing feature in those two installation
>> options?
>>
>> Unless they don't support it, it may be early to deprecate the managix?
>>>>
>>> It's deprecated for common use cases -- I'm not sure how many people are
>> using the Java UDF?
>>
>> Best,
>> Yingyi
>>
>>
>> On Tue, Apr 4, 2017 at 3:54 PM, Taewoo Kim <wa...@gmail.com> wrote:
>>
>> Sorry for the late response.
>>>
>>> Do Ansible and AWS support "installing libraries" like the managix does?
>>> Just checked the documentation. I'm talking about this:
>>> https://ci.apache.org/projects/asterixdb/udf.html. Unless they don't
>>> support it, it may be early to deprecate the managix?
>>>
>>> Best,
>>> Taewoo
>>>
>>> On Thu, Mar 16, 2017 at 9:22 AM, Yingyi Bu <bu...@gmail.com> wrote:
>>>
>>> Hi dev,
>>>>
>>>> We recently added two cluster installation options and their
>>>> documentation is online now:
>>>>
>>>> Ansible: https://ci.apache.org/projects/asterixdb/ansible.html
>>>> AWS: https://ci.apache.org/projects/asterixdb/aws.html
>>>>
>>>> With the Ansible option, you can deploy, start, stop, and erase an
>>>> AsterixDB cluster from your client with a simple shell scripts.
>>>> With the AWS option, you can start/stop an AsterixDB cluster on
>>>> AWS
>>>> with a simple shell scripts. The scripts automate both AWS and AsterixDB
>>>> setup.
>>>>
>>>> Managix is marked as "deprecated" and probably is going to retire
>>>> soon, as (1) it's not well maintained and (2) some of its
>>>> functionalities
>>>> are already provided by existing tools such as Ansible. It was written
>>>> at
>>>> the time when there're not many automation tools that we can rely on,
>>>>
>>> which
>>>
>>>> is no longer the case now.
>>>>
>>>> Please share your thoughts and feedbacks.
>>>> Thanks!
>>>>
>>>> Best,
>>>> Yingyi
>>>>
>>>>
>
Re: New cluster installation options
Posted by Mike Carey <dt...@gmail.com>.
It's definitely the one that Taewoo and Co plan to use in the
short-term. Hopefully all of the feeds stuff works seamlessly in the
cloud too? (They will be feeding Tweets from one of the Twitter
adaptors into *DB and using UDFs to process them on the way in and in
queries.)
On 4/4/17 4:04 PM, Yingyi Bu wrote:
> Thanks for the feedback.
> It hasn't supported yet. But it should be easy to add.
>
> Is Java UDF support the only missing feature in those two installation
> options?
>
>>> Unless they don't support it, it may be early to deprecate the managix?
> It's deprecated for common use cases -- I'm not sure how many people are
> using the Java UDF?
>
> Best,
> Yingyi
>
>
> On Tue, Apr 4, 2017 at 3:54 PM, Taewoo Kim <wa...@gmail.com> wrote:
>
>> Sorry for the late response.
>>
>> Do Ansible and AWS support "installing libraries" like the managix does?
>> Just checked the documentation. I'm talking about this:
>> https://ci.apache.org/projects/asterixdb/udf.html. Unless they don't
>> support it, it may be early to deprecate the managix?
>>
>> Best,
>> Taewoo
>>
>> On Thu, Mar 16, 2017 at 9:22 AM, Yingyi Bu <bu...@gmail.com> wrote:
>>
>>> Hi dev,
>>>
>>> We recently added two cluster installation options and their
>>> documentation is online now:
>>>
>>> Ansible: https://ci.apache.org/projects/asterixdb/ansible.html
>>> AWS: https://ci.apache.org/projects/asterixdb/aws.html
>>>
>>> With the Ansible option, you can deploy, start, stop, and erase an
>>> AsterixDB cluster from your client with a simple shell scripts.
>>> With the AWS option, you can start/stop an AsterixDB cluster on AWS
>>> with a simple shell scripts. The scripts automate both AWS and AsterixDB
>>> setup.
>>>
>>> Managix is marked as "deprecated" and probably is going to retire
>>> soon, as (1) it's not well maintained and (2) some of its functionalities
>>> are already provided by existing tools such as Ansible. It was written at
>>> the time when there're not many automation tools that we can rely on,
>> which
>>> is no longer the case now.
>>>
>>> Please share your thoughts and feedbacks.
>>> Thanks!
>>>
>>> Best,
>>> Yingyi
>>>
Re: New cluster installation options
Posted by Taewoo Kim <wa...@gmail.com>.
@Yingyi: As far as I know, Gift and Xikui are working on those features.
Best,
Taewoo
On Tue, Apr 4, 2017 at 4:04 PM, Yingyi Bu <bu...@gmail.com> wrote:
> Thanks for the feedback.
> It hasn't supported yet. But it should be easy to add.
>
> Is Java UDF support the only missing feature in those two installation
> options?
>
> >> Unless they don't support it, it may be early to deprecate the managix?
> It's deprecated for common use cases -- I'm not sure how many people are
> using the Java UDF?
>
> Best,
> Yingyi
>
>
> On Tue, Apr 4, 2017 at 3:54 PM, Taewoo Kim <wa...@gmail.com> wrote:
>
> > Sorry for the late response.
> >
> > Do Ansible and AWS support "installing libraries" like the managix does?
> > Just checked the documentation. I'm talking about this:
> > https://ci.apache.org/projects/asterixdb/udf.html. Unless they don't
> > support it, it may be early to deprecate the managix?
> >
> > Best,
> > Taewoo
> >
> > On Thu, Mar 16, 2017 at 9:22 AM, Yingyi Bu <bu...@gmail.com> wrote:
> >
> > > Hi dev,
> > >
> > > We recently added two cluster installation options and their
> > > documentation is online now:
> > >
> > > Ansible: https://ci.apache.org/projects/asterixdb/ansible.html
> > > AWS: https://ci.apache.org/projects/asterixdb/aws.html
> > >
> > > With the Ansible option, you can deploy, start, stop, and erase an
> > > AsterixDB cluster from your client with a simple shell scripts.
> > > With the AWS option, you can start/stop an AsterixDB cluster on
> AWS
> > > with a simple shell scripts. The scripts automate both AWS and
> AsterixDB
> > > setup.
> > >
> > > Managix is marked as "deprecated" and probably is going to retire
> > > soon, as (1) it's not well maintained and (2) some of its
> functionalities
> > > are already provided by existing tools such as Ansible. It was written
> at
> > > the time when there're not many automation tools that we can rely on,
> > which
> > > is no longer the case now.
> > >
> > > Please share your thoughts and feedbacks.
> > > Thanks!
> > >
> > > Best,
> > > Yingyi
> > >
> >
>
Re: New cluster installation options
Posted by Yingyi Bu <bu...@gmail.com>.
Thanks for the feedback.
It hasn't supported yet. But it should be easy to add.
Is Java UDF support the only missing feature in those two installation
options?
>> Unless they don't support it, it may be early to deprecate the managix?
It's deprecated for common use cases -- I'm not sure how many people are
using the Java UDF?
Best,
Yingyi
On Tue, Apr 4, 2017 at 3:54 PM, Taewoo Kim <wa...@gmail.com> wrote:
> Sorry for the late response.
>
> Do Ansible and AWS support "installing libraries" like the managix does?
> Just checked the documentation. I'm talking about this:
> https://ci.apache.org/projects/asterixdb/udf.html. Unless they don't
> support it, it may be early to deprecate the managix?
>
> Best,
> Taewoo
>
> On Thu, Mar 16, 2017 at 9:22 AM, Yingyi Bu <bu...@gmail.com> wrote:
>
> > Hi dev,
> >
> > We recently added two cluster installation options and their
> > documentation is online now:
> >
> > Ansible: https://ci.apache.org/projects/asterixdb/ansible.html
> > AWS: https://ci.apache.org/projects/asterixdb/aws.html
> >
> > With the Ansible option, you can deploy, start, stop, and erase an
> > AsterixDB cluster from your client with a simple shell scripts.
> > With the AWS option, you can start/stop an AsterixDB cluster on AWS
> > with a simple shell scripts. The scripts automate both AWS and AsterixDB
> > setup.
> >
> > Managix is marked as "deprecated" and probably is going to retire
> > soon, as (1) it's not well maintained and (2) some of its functionalities
> > are already provided by existing tools such as Ansible. It was written at
> > the time when there're not many automation tools that we can rely on,
> which
> > is no longer the case now.
> >
> > Please share your thoughts and feedbacks.
> > Thanks!
> >
> > Best,
> > Yingyi
> >
>
Re: New cluster installation options
Posted by Taewoo Kim <wa...@gmail.com>.
Sorry for the late response.
Do Ansible and AWS support "installing libraries" like the managix does?
Just checked the documentation. I'm talking about this:
https://ci.apache.org/projects/asterixdb/udf.html. Unless they don't
support it, it may be early to deprecate the managix?
Best,
Taewoo
On Thu, Mar 16, 2017 at 9:22 AM, Yingyi Bu <bu...@gmail.com> wrote:
> Hi dev,
>
> We recently added two cluster installation options and their
> documentation is online now:
>
> Ansible: https://ci.apache.org/projects/asterixdb/ansible.html
> AWS: https://ci.apache.org/projects/asterixdb/aws.html
>
> With the Ansible option, you can deploy, start, stop, and erase an
> AsterixDB cluster from your client with a simple shell scripts.
> With the AWS option, you can start/stop an AsterixDB cluster on AWS
> with a simple shell scripts. The scripts automate both AWS and AsterixDB
> setup.
>
> Managix is marked as "deprecated" and probably is going to retire
> soon, as (1) it's not well maintained and (2) some of its functionalities
> are already provided by existing tools such as Ansible. It was written at
> the time when there're not many automation tools that we can rely on, which
> is no longer the case now.
>
> Please share your thoughts and feedbacks.
> Thanks!
>
> Best,
> Yingyi
>