You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lens.apache.org by Yash Sharma <ya...@gmail.com> on 2015/05/25 09:08:13 UTC
Need information on Lens ML setup
Hi All,
I was trying to play around with Lens ML module and was not able to get lot
of information on the setup and starting guidelines.
I was going through the deployment section in docs [1] but could not figure
out lot of things. Is there any resource for setting up ML module and
trying out some sample models through the CLI.
Also what is the recommended version of Spark.
Any tips would be appreciated. I would be happy to enhance the docs based
on my findings.
Thanks
1. https://lens.incubator.apache.org/user/ml.html
Re: Need information on Lens ML setup
Posted by Sharad Agarwal <sh...@apache.org>.
check if the ml module jar is in the LENS_EXT_CLASSPATH
Re: Need information on Lens ML setup
Posted by Yash Sharma <ya...@gmail.com>.
I keep stumbling on this error. I wonder if this a known error ?
Will continue checking it tomorrow.
Regards
----------------------------------------------------------------------------------------------------------------------------------------------------
$bin/lens-run-class.sh org.apache.lens.ml.MLRunner
$LENS_ML/data/naive_bayes --conf
/usr/local/lens/client/conf-pseudo-distr/lens-client-site.xml
opt_classname is org.apache.lens.ml.MLRunner
args are
/home/yash/work/git/incubator-lens/lens-ml-dist/target/apache-lens-2.2.0-beta-incubating-SNAPSHOT-ml/data/naive_bayes
Executing class org.apache.lens.ml.MLRunner
Adding hadoop libs in classpath from /usr/local/hadoop
HIVE_HOME is set, adding /usr/local/hive/lib/* into lens classpath
Error: Could not find or load main class org.apache.lens.ml.MLRunner
----------------------------------------------------------------------------------------------------------------------------------------------------
$echo $LENS_ML
/home/yash/work/git/incubator-lens/lens-ml-dist/target/apache-lens-2.2.0-beta-incubating-SNAPSHOT-ml
On Tue, May 26, 2015 at 3:33 PM, Yash Sharma <ya...@gmail.com> wrote:
> Sure, I would start exploring it and post updates.
>
> Regards
>
> On Tue, May 26, 2015 at 3:30 PM, Sharad Agarwal <sh...@apache.org> wrote:
>
>> Yash,
>>
>> You may want to look at https://issues.apache.org/jira/browse/LENS-515
>> feel free to assign to yourself if it seems interesting to you.
>>
>> Sharad
>>
>>
>> On Tue, May 26, 2015 at 10:52 AM, Yash Sharma <ya...@gmail.com> wrote:
>>
>> > Awesome. Exactly what I was looking for. Thanks Sharad.
>> >
>> > Regards
>> >
>> > On Tue, May 26, 2015 at 10:39 AM, Sharad Agarwal <sh...@apache.org>
>> > wrote:
>> >
>> > > lens ml module is an optional add on service
>> > >
>> > > - the dependencies of lens ml are not loaded by default in the lens
>> > server.
>> > > the ml dependencies have to be exported to LENS_EXT_CLASSPATH, which
>> is
>> > > automatically added into the lens server classpath.
>> > >
>> > > - there is a separate distribution that gets created for lens ml. Use
>> > >
>> >
>> LENS_EXT_CLASSPATH=$LENS_EXT_CLASSPATH:`$LENS_ML/bin/lens-ml-classpath.sh`
>> > > to export the lens ml classpath before the starting the lens server
>> > >
>> > > - you can find the sample conf that loads the ml module in
>> > > tools/conf-pseudo-distr/server
>> > >
>> > > - it has been tested with spark 1.3
>> > >
>> > > Alternatively you can try running via docker. Docker sets the above
>> > > environment for you automatically.
>> > >
>> > > The ml module does not have the CLI shell commands as of now. So only
>> the
>> > > way to run some models is via running MLRunner:
>> > >
>> > > bin/lens-run-class.sh org.apache.lens.ml.MLRunner
>> > $LENS_ML/data/naive_bayes
>> > > --conf $LENS_CLIENT_CONF
>> > >
>> > > Sharad
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > > On Mon, May 25, 2015 at 12:38 PM, Yash Sharma <ya...@gmail.com>
>> wrote:
>> > >
>> > > > Hi All,
>> > > > I was trying to play around with Lens ML module and was not able to
>> get
>> > > lot
>> > > > of information on the setup and starting guidelines.
>> > > >
>> > > > I was going through the deployment section in docs [1] but could not
>> > > figure
>> > > > out lot of things. Is there any resource for setting up ML module
>> and
>> > > > trying out some sample models through the CLI.
>> > > > Also what is the recommended version of Spark.
>> > > >
>> > > > Any tips would be appreciated. I would be happy to enhance the docs
>> > based
>> > > > on my findings.
>> > > >
>> > > > Thanks
>> > > >
>> > > > 1. https://lens.incubator.apache.org/user/ml.html
>> > > >
>> > >
>> >
>>
>
>
Re: Need information on Lens ML setup
Posted by Yash Sharma <ya...@gmail.com>.
Sure, I would start exploring it and post updates.
Regards
On Tue, May 26, 2015 at 3:30 PM, Sharad Agarwal <sh...@apache.org> wrote:
> Yash,
>
> You may want to look at https://issues.apache.org/jira/browse/LENS-515
> feel free to assign to yourself if it seems interesting to you.
>
> Sharad
>
>
> On Tue, May 26, 2015 at 10:52 AM, Yash Sharma <ya...@gmail.com> wrote:
>
> > Awesome. Exactly what I was looking for. Thanks Sharad.
> >
> > Regards
> >
> > On Tue, May 26, 2015 at 10:39 AM, Sharad Agarwal <sh...@apache.org>
> > wrote:
> >
> > > lens ml module is an optional add on service
> > >
> > > - the dependencies of lens ml are not loaded by default in the lens
> > server.
> > > the ml dependencies have to be exported to LENS_EXT_CLASSPATH, which is
> > > automatically added into the lens server classpath.
> > >
> > > - there is a separate distribution that gets created for lens ml. Use
> > >
> >
> LENS_EXT_CLASSPATH=$LENS_EXT_CLASSPATH:`$LENS_ML/bin/lens-ml-classpath.sh`
> > > to export the lens ml classpath before the starting the lens server
> > >
> > > - you can find the sample conf that loads the ml module in
> > > tools/conf-pseudo-distr/server
> > >
> > > - it has been tested with spark 1.3
> > >
> > > Alternatively you can try running via docker. Docker sets the above
> > > environment for you automatically.
> > >
> > > The ml module does not have the CLI shell commands as of now. So only
> the
> > > way to run some models is via running MLRunner:
> > >
> > > bin/lens-run-class.sh org.apache.lens.ml.MLRunner
> > $LENS_ML/data/naive_bayes
> > > --conf $LENS_CLIENT_CONF
> > >
> > > Sharad
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Mon, May 25, 2015 at 12:38 PM, Yash Sharma <ya...@gmail.com>
> wrote:
> > >
> > > > Hi All,
> > > > I was trying to play around with Lens ML module and was not able to
> get
> > > lot
> > > > of information on the setup and starting guidelines.
> > > >
> > > > I was going through the deployment section in docs [1] but could not
> > > figure
> > > > out lot of things. Is there any resource for setting up ML module and
> > > > trying out some sample models through the CLI.
> > > > Also what is the recommended version of Spark.
> > > >
> > > > Any tips would be appreciated. I would be happy to enhance the docs
> > based
> > > > on my findings.
> > > >
> > > > Thanks
> > > >
> > > > 1. https://lens.incubator.apache.org/user/ml.html
> > > >
> > >
> >
>
Re: Need information on Lens ML setup
Posted by Sharad Agarwal <sh...@apache.org>.
Yash,
You may want to look at https://issues.apache.org/jira/browse/LENS-515
feel free to assign to yourself if it seems interesting to you.
Sharad
On Tue, May 26, 2015 at 10:52 AM, Yash Sharma <ya...@gmail.com> wrote:
> Awesome. Exactly what I was looking for. Thanks Sharad.
>
> Regards
>
> On Tue, May 26, 2015 at 10:39 AM, Sharad Agarwal <sh...@apache.org>
> wrote:
>
> > lens ml module is an optional add on service
> >
> > - the dependencies of lens ml are not loaded by default in the lens
> server.
> > the ml dependencies have to be exported to LENS_EXT_CLASSPATH, which is
> > automatically added into the lens server classpath.
> >
> > - there is a separate distribution that gets created for lens ml. Use
> >
> LENS_EXT_CLASSPATH=$LENS_EXT_CLASSPATH:`$LENS_ML/bin/lens-ml-classpath.sh`
> > to export the lens ml classpath before the starting the lens server
> >
> > - you can find the sample conf that loads the ml module in
> > tools/conf-pseudo-distr/server
> >
> > - it has been tested with spark 1.3
> >
> > Alternatively you can try running via docker. Docker sets the above
> > environment for you automatically.
> >
> > The ml module does not have the CLI shell commands as of now. So only the
> > way to run some models is via running MLRunner:
> >
> > bin/lens-run-class.sh org.apache.lens.ml.MLRunner
> $LENS_ML/data/naive_bayes
> > --conf $LENS_CLIENT_CONF
> >
> > Sharad
> >
> >
> >
> >
> >
> >
> > On Mon, May 25, 2015 at 12:38 PM, Yash Sharma <ya...@gmail.com> wrote:
> >
> > > Hi All,
> > > I was trying to play around with Lens ML module and was not able to get
> > lot
> > > of information on the setup and starting guidelines.
> > >
> > > I was going through the deployment section in docs [1] but could not
> > figure
> > > out lot of things. Is there any resource for setting up ML module and
> > > trying out some sample models through the CLI.
> > > Also what is the recommended version of Spark.
> > >
> > > Any tips would be appreciated. I would be happy to enhance the docs
> based
> > > on my findings.
> > >
> > > Thanks
> > >
> > > 1. https://lens.incubator.apache.org/user/ml.html
> > >
> >
>
Re: Need information on Lens ML setup
Posted by Yash Sharma <ya...@gmail.com>.
Awesome. Exactly what I was looking for. Thanks Sharad.
Regards
On Tue, May 26, 2015 at 10:39 AM, Sharad Agarwal <sh...@apache.org> wrote:
> lens ml module is an optional add on service
>
> - the dependencies of lens ml are not loaded by default in the lens server.
> the ml dependencies have to be exported to LENS_EXT_CLASSPATH, which is
> automatically added into the lens server classpath.
>
> - there is a separate distribution that gets created for lens ml. Use
> LENS_EXT_CLASSPATH=$LENS_EXT_CLASSPATH:`$LENS_ML/bin/lens-ml-classpath.sh`
> to export the lens ml classpath before the starting the lens server
>
> - you can find the sample conf that loads the ml module in
> tools/conf-pseudo-distr/server
>
> - it has been tested with spark 1.3
>
> Alternatively you can try running via docker. Docker sets the above
> environment for you automatically.
>
> The ml module does not have the CLI shell commands as of now. So only the
> way to run some models is via running MLRunner:
>
> bin/lens-run-class.sh org.apache.lens.ml.MLRunner $LENS_ML/data/naive_bayes
> --conf $LENS_CLIENT_CONF
>
> Sharad
>
>
>
>
>
>
> On Mon, May 25, 2015 at 12:38 PM, Yash Sharma <ya...@gmail.com> wrote:
>
> > Hi All,
> > I was trying to play around with Lens ML module and was not able to get
> lot
> > of information on the setup and starting guidelines.
> >
> > I was going through the deployment section in docs [1] but could not
> figure
> > out lot of things. Is there any resource for setting up ML module and
> > trying out some sample models through the CLI.
> > Also what is the recommended version of Spark.
> >
> > Any tips would be appreciated. I would be happy to enhance the docs based
> > on my findings.
> >
> > Thanks
> >
> > 1. https://lens.incubator.apache.org/user/ml.html
> >
>
Re: Need information on Lens ML setup
Posted by Sharad Agarwal <sh...@apache.org>.
lens ml module is an optional add on service
- the dependencies of lens ml are not loaded by default in the lens server.
the ml dependencies have to be exported to LENS_EXT_CLASSPATH, which is
automatically added into the lens server classpath.
- there is a separate distribution that gets created for lens ml. Use
LENS_EXT_CLASSPATH=$LENS_EXT_CLASSPATH:`$LENS_ML/bin/lens-ml-classpath.sh`
to export the lens ml classpath before the starting the lens server
- you can find the sample conf that loads the ml module in
tools/conf-pseudo-distr/server
- it has been tested with spark 1.3
Alternatively you can try running via docker. Docker sets the above
environment for you automatically.
The ml module does not have the CLI shell commands as of now. So only the
way to run some models is via running MLRunner:
bin/lens-run-class.sh org.apache.lens.ml.MLRunner $LENS_ML/data/naive_bayes
--conf $LENS_CLIENT_CONF
Sharad
On Mon, May 25, 2015 at 12:38 PM, Yash Sharma <ya...@gmail.com> wrote:
> Hi All,
> I was trying to play around with Lens ML module and was not able to get lot
> of information on the setup and starting guidelines.
>
> I was going through the deployment section in docs [1] but could not figure
> out lot of things. Is there any resource for setting up ML module and
> trying out some sample models through the CLI.
> Also what is the recommended version of Spark.
>
> Any tips would be appreciated. I would be happy to enhance the docs based
> on my findings.
>
> Thanks
>
> 1. https://lens.incubator.apache.org/user/ml.html
>