You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@apex.apache.org by Mohammad Kargar <mk...@phemi.com> on 2017/03/22 23:19:25 UTC
Apex installation instructions
Hi,
Is there any instructions for installing open source Apex (no DataTorrent
edition) in a production environment? Haven't had any luck searching online
documents here <http://apex.apache.org/docs/apex-3.5/>.
Thanks,
Re: Apex installation instructions
Posted by Pramod Immaneni <pr...@datatorrent.com>.
What is the error message you are seeing?
On Wed, Mar 22, 2017 at 5:51 PM, Mohammad Kargar <mk...@phemi.com> wrote:
> That works in a Dev environment fine. However in a test cluster
> environment (where Apex script copied manually) it's failing to find
> configurations.
>
> On Mar 22, 2017 4:54 PM, "Munagala Ramanath" <ra...@datatorrent.com> wrote:
>
>> Please take a look at: http://apex.apache.org/docs.html
>> The Beginner's Guide is a good place to start. Briefly stated, you'll
>> need to build your application package
>> using maven and deploy it using the commandline tool "apex" that is in
>> the apex-core repository.
>>
>> Ram
>>
>> On Wed, Mar 22, 2017 at 4:19 PM, Mohammad Kargar <mk...@phemi.com>
>> wrote:
>>
>>> Hi,
>>>
>>> Is there any instructions for installing open source Apex (no
>>> DataTorrent edition) in a production environment? Haven't had any luck
>>> searching online documents here <http://apex.apache.org/docs/apex-3.5/>
>>> .
>>>
>>> Thanks,
>>>
>>
>>
>>
>> --
>>
>> _______________________________________________________
>>
>> Munagala V. Ramanath
>>
>> Software Engineer
>>
>> E: ram@datatorrent.com | M: (408) 331-5034 | Twitter: @UnknownRam
>>
>> www.datatorrent.com | apex.apache.org
>>
>>
>>
Re: Apex installation instructions
Posted by Pramod Immaneni <pr...@datatorrent.com>.
Oh, I didn't mean to suggest a distribution. The Apex engine is the same
underneath as far as production use goes.
On Thu, Mar 23, 2017 at 9:29 AM, Mohammad Kargar <mk...@phemi.com> wrote:
> Hi Pramond,
>
> DataTorrent RTS seems like a great bundle and that's how we started our
> evaluation process. However its free edition is very limited and this is
> why I'm looking for a Apex only (no DataTorrent specific additions)
> solution.
>
> Thanks,
>
> On Thu, Mar 23, 2017 at 8:41 AM, Pramod Immaneni <pr...@datatorrent.com>
> wrote:
>
>> Hi Mohammad,
>>
>> It is being used in production in various places, at least from a
>> DataTorrent perspective we bundle it in DataTorrent RTS and have multiple
>> customers using it in production. The reason you see com.datatorrent is it
>> was originally created by DataTorrent as a proprietary software and later
>> open sourced under Apache. We have been moving package names progressively
>> without breaking backward compatibility for existing applications and that
>> is why you are seeing the mix.
>>
>> Thanks
>>
>> On Thu, Mar 23, 2017 at 8:01 AM, Mohammad Kargar <mk...@phemi.com>
>> wrote:
>>
>>> HI Thomas,
>>>
>>> I tried the new cli binary and it worked fine. Thanks for the quick
>>> solution!
>>>
>>> Looking into the apex
>>> <https://github.com/apache/apex-core/blob/master/engine/src/main/scripts/apex> script
>>> I can see hard-coded development settings (e.g. BUILD_DIR="$( dirname "$0"
>>> )/../../../target") which explains why it didn't work in the first place. I
>>> also see most of the code are still under com.datatorrent packages as
>>> opposed to org.apache.apex. Is Apex (the open source project) ready for
>>> *production* use?
>>>
>>> Thanks,
>>> Mohammad
>>>
>>> On Wed, Mar 22, 2017 at 8:00 PM, Thomas Weise <th...@apache.org> wrote:
>>>
>>>> Hi,
>>>>
>>>> Those instructions are indeed for running in a dev environment only.
>>>> There are a few other download options:
>>>>
>>>> http://apex.apache.org/downloads.html
>>>>
>>>> The Bigtop binaries can be used, but they are designed to also install
>>>> Hadoop.
>>>>
>>>> I believe you are looking to just install the Apex CLI on an edge node?
>>>> There wasn't a binary package available for that purpose so far, but now
>>>> there is:
>>>>
>>>> https://github.com/atrato/apex-cli-package/releases/tag/v3.5.0
>>>>
>>>> Give it a try and let me know how it works. It can probably at some
>>>> point become part of the official release also.
>>>>
>>>> Thanks,
>>>> Thomas
>>>>
>>>>
>>>>
>>>> On Wed, Mar 22, 2017 at 5:51 PM, Mohammad Kargar <mk...@phemi.com>
>>>> wrote:
>>>>
>>>>> That works in a Dev environment fine. However in a test cluster
>>>>> environment (where Apex script copied manually) it's failing to find
>>>>> configurations.
>>>>>
>>>>> On Mar 22, 2017 4:54 PM, "Munagala Ramanath" <ra...@datatorrent.com>
>>>>> wrote:
>>>>>
>>>>>> Please take a look at: http://apex.apache.org/docs.html
>>>>>> The Beginner's Guide is a good place to start. Briefly stated, you'll
>>>>>> need to build your application package
>>>>>> using maven and deploy it using the commandline tool "apex" that is
>>>>>> in the apex-core repository.
>>>>>>
>>>>>> Ram
>>>>>>
>>>>>> On Wed, Mar 22, 2017 at 4:19 PM, Mohammad Kargar <mk...@phemi.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Is there any instructions for installing open source Apex (no
>>>>>>> DataTorrent edition) in a production environment? Haven't had any luck
>>>>>>> searching online documents here
>>>>>>> <http://apex.apache.org/docs/apex-3.5/>.
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> _______________________________________________________
>>>>>>
>>>>>> Munagala V. Ramanath
>>>>>>
>>>>>> Software Engineer
>>>>>>
>>>>>> E: ram@datatorrent.com | M: (408) 331-5034 | Twitter: @UnknownRam
>>>>>>
>>>>>> www.datatorrent.com | apex.apache.org
>>>>>>
>>>>>>
>>>>>>
>>>>
>>>
>>
>
Re: Apex installation instructions
Posted by Mohammad Kargar <mk...@phemi.com>.
Hi Pramond,
DataTorrent RTS seems like a great bundle and that's how we started our
evaluation process. However its free edition is very limited and this is
why I'm looking for a Apex only (no DataTorrent specific additions)
solution.
Thanks,
On Thu, Mar 23, 2017 at 8:41 AM, Pramod Immaneni <pr...@datatorrent.com>
wrote:
> Hi Mohammad,
>
> It is being used in production in various places, at least from a
> DataTorrent perspective we bundle it in DataTorrent RTS and have multiple
> customers using it in production. The reason you see com.datatorrent is it
> was originally created by DataTorrent as a proprietary software and later
> open sourced under Apache. We have been moving package names progressively
> without breaking backward compatibility for existing applications and that
> is why you are seeing the mix.
>
> Thanks
>
> On Thu, Mar 23, 2017 at 8:01 AM, Mohammad Kargar <mk...@phemi.com>
> wrote:
>
>> HI Thomas,
>>
>> I tried the new cli binary and it worked fine. Thanks for the quick
>> solution!
>>
>> Looking into the apex
>> <https://github.com/apache/apex-core/blob/master/engine/src/main/scripts/apex> script
>> I can see hard-coded development settings (e.g. BUILD_DIR="$( dirname "$0"
>> )/../../../target") which explains why it didn't work in the first place. I
>> also see most of the code are still under com.datatorrent packages as
>> opposed to org.apache.apex. Is Apex (the open source project) ready for
>> *production* use?
>>
>> Thanks,
>> Mohammad
>>
>> On Wed, Mar 22, 2017 at 8:00 PM, Thomas Weise <th...@apache.org> wrote:
>>
>>> Hi,
>>>
>>> Those instructions are indeed for running in a dev environment only.
>>> There are a few other download options:
>>>
>>> http://apex.apache.org/downloads.html
>>>
>>> The Bigtop binaries can be used, but they are designed to also install
>>> Hadoop.
>>>
>>> I believe you are looking to just install the Apex CLI on an edge node?
>>> There wasn't a binary package available for that purpose so far, but now
>>> there is:
>>>
>>> https://github.com/atrato/apex-cli-package/releases/tag/v3.5.0
>>>
>>> Give it a try and let me know how it works. It can probably at some
>>> point become part of the official release also.
>>>
>>> Thanks,
>>> Thomas
>>>
>>>
>>>
>>> On Wed, Mar 22, 2017 at 5:51 PM, Mohammad Kargar <mk...@phemi.com>
>>> wrote:
>>>
>>>> That works in a Dev environment fine. However in a test cluster
>>>> environment (where Apex script copied manually) it's failing to find
>>>> configurations.
>>>>
>>>> On Mar 22, 2017 4:54 PM, "Munagala Ramanath" <ra...@datatorrent.com>
>>>> wrote:
>>>>
>>>>> Please take a look at: http://apex.apache.org/docs.html
>>>>> The Beginner's Guide is a good place to start. Briefly stated, you'll
>>>>> need to build your application package
>>>>> using maven and deploy it using the commandline tool "apex" that is in
>>>>> the apex-core repository.
>>>>>
>>>>> Ram
>>>>>
>>>>> On Wed, Mar 22, 2017 at 4:19 PM, Mohammad Kargar <mk...@phemi.com>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Is there any instructions for installing open source Apex (no
>>>>>> DataTorrent edition) in a production environment? Haven't had any luck
>>>>>> searching online documents here
>>>>>> <http://apex.apache.org/docs/apex-3.5/>.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> _______________________________________________________
>>>>>
>>>>> Munagala V. Ramanath
>>>>>
>>>>> Software Engineer
>>>>>
>>>>> E: ram@datatorrent.com | M: (408) 331-5034 | Twitter: @UnknownRam
>>>>>
>>>>> www.datatorrent.com | apex.apache.org
>>>>>
>>>>>
>>>>>
>>>
>>
>
Re: Apex installation instructions
Posted by Pramod Immaneni <pr...@datatorrent.com>.
Hi Mohammad,
It is being used in production in various places, at least from a
DataTorrent perspective we bundle it in DataTorrent RTS and have multiple
customers using it in production. The reason you see com.datatorrent is it
was originally created by DataTorrent as a proprietary software and later
open sourced under Apache. We have been moving package names progressively
without breaking backward compatibility for existing applications and that
is why you are seeing the mix.
Thanks
On Thu, Mar 23, 2017 at 8:01 AM, Mohammad Kargar <mk...@phemi.com> wrote:
> HI Thomas,
>
> I tried the new cli binary and it worked fine. Thanks for the quick
> solution!
>
> Looking into the apex
> <https://github.com/apache/apex-core/blob/master/engine/src/main/scripts/apex> script
> I can see hard-coded development settings (e.g. BUILD_DIR="$( dirname "$0"
> )/../../../target") which explains why it didn't work in the first place. I
> also see most of the code are still under com.datatorrent packages as
> opposed to org.apache.apex. Is Apex (the open source project) ready for
> *production* use?
>
> Thanks,
> Mohammad
>
> On Wed, Mar 22, 2017 at 8:00 PM, Thomas Weise <th...@apache.org> wrote:
>
>> Hi,
>>
>> Those instructions are indeed for running in a dev environment only.
>> There are a few other download options:
>>
>> http://apex.apache.org/downloads.html
>>
>> The Bigtop binaries can be used, but they are designed to also install
>> Hadoop.
>>
>> I believe you are looking to just install the Apex CLI on an edge node?
>> There wasn't a binary package available for that purpose so far, but now
>> there is:
>>
>> https://github.com/atrato/apex-cli-package/releases/tag/v3.5.0
>>
>> Give it a try and let me know how it works. It can probably at some point
>> become part of the official release also.
>>
>> Thanks,
>> Thomas
>>
>>
>>
>> On Wed, Mar 22, 2017 at 5:51 PM, Mohammad Kargar <mk...@phemi.com>
>> wrote:
>>
>>> That works in a Dev environment fine. However in a test cluster
>>> environment (where Apex script copied manually) it's failing to find
>>> configurations.
>>>
>>> On Mar 22, 2017 4:54 PM, "Munagala Ramanath" <ra...@datatorrent.com>
>>> wrote:
>>>
>>>> Please take a look at: http://apex.apache.org/docs.html
>>>> The Beginner's Guide is a good place to start. Briefly stated, you'll
>>>> need to build your application package
>>>> using maven and deploy it using the commandline tool "apex" that is in
>>>> the apex-core repository.
>>>>
>>>> Ram
>>>>
>>>> On Wed, Mar 22, 2017 at 4:19 PM, Mohammad Kargar <mk...@phemi.com>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Is there any instructions for installing open source Apex (no
>>>>> DataTorrent edition) in a production environment? Haven't had any luck
>>>>> searching online documents here
>>>>> <http://apex.apache.org/docs/apex-3.5/>.
>>>>>
>>>>> Thanks,
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> _______________________________________________________
>>>>
>>>> Munagala V. Ramanath
>>>>
>>>> Software Engineer
>>>>
>>>> E: ram@datatorrent.com | M: (408) 331-5034 | Twitter: @UnknownRam
>>>>
>>>> www.datatorrent.com | apex.apache.org
>>>>
>>>>
>>>>
>>
>
Re: Apex installation instructions
Posted by Sandesh Hegde <sa...@datatorrent.com>.
Hello Mohammad
Apex code was made open source and aventually donated by Datatorrent after
3 years of development and production use. Community is working on renaming
the packages, definitely welcome your contribution as well.
Unlike many academic big data projects, Apex was built to handle production
load.
On Thu, Mar 23, 2017 at 8:02 AM Mohammad Kargar <mk...@phemi.com> wrote:
> HI Thomas,
>
> I tried the new cli binary and it worked fine. Thanks for the quick
> solution!
>
> Looking into the apex
> <https://github.com/apache/apex-core/blob/master/engine/src/main/scripts/apex> script
> I can see hard-coded development settings (e.g. BUILD_DIR="$( dirname "$0"
> )/../../../target") which explains why it didn't work in the first place. I
> also see most of the code are still under com.datatorrent packages as
> opposed to org.apache.apex. Is Apex (the open source project) ready for
> *production* use?
>
> Thanks,
> Mohammad
>
> On Wed, Mar 22, 2017 at 8:00 PM, Thomas Weise <th...@apache.org> wrote:
>
> Hi,
>
> Those instructions are indeed for running in a dev environment only. There
> are a few other download options:
>
> http://apex.apache.org/downloads.html
>
> The Bigtop binaries can be used, but they are designed to also install
> Hadoop.
>
> I believe you are looking to just install the Apex CLI on an edge node?
> There wasn't a binary package available for that purpose so far, but now
> there is:
>
> https://github.com/atrato/apex-cli-package/releases/tag/v3.5.0
>
> Give it a try and let me know how it works. It can probably at some point
> become part of the official release also.
>
> Thanks,
> Thomas
>
>
>
> On Wed, Mar 22, 2017 at 5:51 PM, Mohammad Kargar <mk...@phemi.com>
> wrote:
>
> That works in a Dev environment fine. However in a test cluster
> environment (where Apex script copied manually) it's failing to find
> configurations.
>
> On Mar 22, 2017 4:54 PM, "Munagala Ramanath" <ra...@datatorrent.com> wrote:
>
> Please take a look at: http://apex.apache.org/docs.html
> The Beginner's Guide is a good place to start. Briefly stated, you'll need
> to build your application package
> using maven and deploy it using the commandline tool "apex" that is in the
> apex-core repository.
>
> Ram
>
> On Wed, Mar 22, 2017 at 4:19 PM, Mohammad Kargar <mk...@phemi.com>
> wrote:
>
> Hi,
>
> Is there any instructions for installing open source Apex (no DataTorrent
> edition) in a production environment? Haven't had any luck searching online
> documents here <http://apex.apache.org/docs/apex-3.5/>.
>
> Thanks,
>
>
>
>
> --
>
> _______________________________________________________
>
> Munagala V. Ramanath
>
> Software Engineer
>
> E: ram@datatorrent.com | M: (408) 331-5034 | Twitter: @UnknownRam
>
> www.datatorrent.com | apex.apache.org
>
>
>
>
> --
*Join us at Apex Big Data World-San Jose
<http://www.apexbigdata.com/san-jose.html>, April 4, 2017!*
[image: http://www.apexbigdata.com/san-jose-register.html]
Re: Apex installation instructions
Posted by Mohammad Kargar <mk...@phemi.com>.
HI Thomas,
I tried the new cli binary and it worked fine. Thanks for the quick
solution!
Looking into the apex
<https://github.com/apache/apex-core/blob/master/engine/src/main/scripts/apex>
script
I can see hard-coded development settings (e.g. BUILD_DIR="$( dirname "$0"
)/../../../target") which explains why it didn't work in the first place. I
also see most of the code are still under com.datatorrent packages as
opposed to org.apache.apex. Is Apex (the open source project) ready for
*production* use?
Thanks,
Mohammad
On Wed, Mar 22, 2017 at 8:00 PM, Thomas Weise <th...@apache.org> wrote:
> Hi,
>
> Those instructions are indeed for running in a dev environment only. There
> are a few other download options:
>
> http://apex.apache.org/downloads.html
>
> The Bigtop binaries can be used, but they are designed to also install
> Hadoop.
>
> I believe you are looking to just install the Apex CLI on an edge node?
> There wasn't a binary package available for that purpose so far, but now
> there is:
>
> https://github.com/atrato/apex-cli-package/releases/tag/v3.5.0
>
> Give it a try and let me know how it works. It can probably at some point
> become part of the official release also.
>
> Thanks,
> Thomas
>
>
>
> On Wed, Mar 22, 2017 at 5:51 PM, Mohammad Kargar <mk...@phemi.com>
> wrote:
>
>> That works in a Dev environment fine. However in a test cluster
>> environment (where Apex script copied manually) it's failing to find
>> configurations.
>>
>> On Mar 22, 2017 4:54 PM, "Munagala Ramanath" <ra...@datatorrent.com> wrote:
>>
>>> Please take a look at: http://apex.apache.org/docs.html
>>> The Beginner's Guide is a good place to start. Briefly stated, you'll
>>> need to build your application package
>>> using maven and deploy it using the commandline tool "apex" that is in
>>> the apex-core repository.
>>>
>>> Ram
>>>
>>> On Wed, Mar 22, 2017 at 4:19 PM, Mohammad Kargar <mk...@phemi.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> Is there any instructions for installing open source Apex (no
>>>> DataTorrent edition) in a production environment? Haven't had any luck
>>>> searching online documents here <http://apex.apache.org/docs/apex-3.5/>
>>>> .
>>>>
>>>> Thanks,
>>>>
>>>
>>>
>>>
>>> --
>>>
>>> _______________________________________________________
>>>
>>> Munagala V. Ramanath
>>>
>>> Software Engineer
>>>
>>> E: ram@datatorrent.com | M: (408) 331-5034 | Twitter: @UnknownRam
>>>
>>> www.datatorrent.com | apex.apache.org
>>>
>>>
>>>
>
Re: Apex installation instructions
Posted by Thomas Weise <th...@apache.org>.
Hi,
Those instructions are indeed for running in a dev environment only. There
are a few other download options:
http://apex.apache.org/downloads.html
The Bigtop binaries can be used, but they are designed to also install
Hadoop.
I believe you are looking to just install the Apex CLI on an edge node?
There wasn't a binary package available for that purpose so far, but now
there is:
https://github.com/atrato/apex-cli-package/releases/tag/v3.5.0
Give it a try and let me know how it works. It can probably at some point
become part of the official release also.
Thanks,
Thomas
On Wed, Mar 22, 2017 at 5:51 PM, Mohammad Kargar <mk...@phemi.com> wrote:
> That works in a Dev environment fine. However in a test cluster
> environment (where Apex script copied manually) it's failing to find
> configurations.
>
> On Mar 22, 2017 4:54 PM, "Munagala Ramanath" <ra...@datatorrent.com> wrote:
>
>> Please take a look at: http://apex.apache.org/docs.html
>> The Beginner's Guide is a good place to start. Briefly stated, you'll
>> need to build your application package
>> using maven and deploy it using the commandline tool "apex" that is in
>> the apex-core repository.
>>
>> Ram
>>
>> On Wed, Mar 22, 2017 at 4:19 PM, Mohammad Kargar <mk...@phemi.com>
>> wrote:
>>
>>> Hi,
>>>
>>> Is there any instructions for installing open source Apex (no
>>> DataTorrent edition) in a production environment? Haven't had any luck
>>> searching online documents here <http://apex.apache.org/docs/apex-3.5/>
>>> .
>>>
>>> Thanks,
>>>
>>
>>
>>
>> --
>>
>> _______________________________________________________
>>
>> Munagala V. Ramanath
>>
>> Software Engineer
>>
>> E: ram@datatorrent.com | M: (408) 331-5034 | Twitter: @UnknownRam
>>
>> www.datatorrent.com | apex.apache.org
>>
>>
>>
Re: Apex installation instructions
Posted by Mohammad Kargar <mk...@phemi.com>.
That works in a Dev environment fine. However in a test cluster environment
(where Apex script copied manually) it's failing to find configurations.
On Mar 22, 2017 4:54 PM, "Munagala Ramanath" <ra...@datatorrent.com> wrote:
> Please take a look at: http://apex.apache.org/docs.html
> The Beginner's Guide is a good place to start. Briefly stated, you'll need
> to build your application package
> using maven and deploy it using the commandline tool "apex" that is in the
> apex-core repository.
>
> Ram
>
> On Wed, Mar 22, 2017 at 4:19 PM, Mohammad Kargar <mk...@phemi.com>
> wrote:
>
>> Hi,
>>
>> Is there any instructions for installing open source Apex (no DataTorrent
>> edition) in a production environment? Haven't had any luck searching online
>> documents here <http://apex.apache.org/docs/apex-3.5/>.
>>
>> Thanks,
>>
>
>
>
> --
>
> _______________________________________________________
>
> Munagala V. Ramanath
>
> Software Engineer
>
> E: ram@datatorrent.com | M: (408) 331-5034 | Twitter: @UnknownRam
>
> www.datatorrent.com | apex.apache.org
>
>
>
Re: Apex installation instructions
Posted by Munagala Ramanath <ra...@datatorrent.com>.
Please take a look at: http://apex.apache.org/docs.html
The Beginner's Guide is a good place to start. Briefly stated, you'll need
to build your application package
using maven and deploy it using the commandline tool "apex" that is in the
apex-core repository.
Ram
On Wed, Mar 22, 2017 at 4:19 PM, Mohammad Kargar <mk...@phemi.com> wrote:
> Hi,
>
> Is there any instructions for installing open source Apex (no DataTorrent
> edition) in a production environment? Haven't had any luck searching online
> documents here <http://apex.apache.org/docs/apex-3.5/>.
>
> Thanks,
>
--
_______________________________________________________
Munagala V. Ramanath
Software Engineer
E: ram@datatorrent.com | M: (408) 331-5034 | Twitter: @UnknownRam
www.datatorrent.com | apex.apache.org