You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Li Jiajia <ji...@gatech.edu> on 2016/02/24 04:50:29 UTC
About Tensor Factorization in Spark
Hi,
I wonder if there are tensor algorithms or tensor data structures supported by Spark MLlib or GraphX. In a Spark intro slide, tensor factorization is mentioned as one of the algorithms in GraphX, but I didn't find it in the guide. If not, do you plan to implement them in the future?
I’m new to Spark, please give some detailed explanation if possible. Thanks in advance.
Best regards!
Jiajia Li
------------------------------------------
E-mail: jiajiali@gatech.edu
Tel: +1 (404)9404603
Computational Science & Engineering
Georgia Institute of Technology
Re: About Tensor Factorization in Spark
Posted by Li Jiajia <ji...@gatech.edu>.
I see. Thanks very much, Nick! I’m thinking to take this as my class project. :-)
Best regards!
Jiajia Li
------------------------------------------
E-mail: jiajiali@gatech.edu
Tel: +1 (404)9404603
Computational Science & Engineering
Georgia Institute of Technology
> On Feb 24, 2016, at 1:10 AM, Nick Pentreath <ni...@gmail.com> wrote:
>
> Not that I'm aware of - it would be a great addition as a Spark package!
>
> On Wed, 24 Feb 2016 at 06:33 Li Jiajia <jiajiali@gatech.edu <ma...@gatech.edu>> wrote:
> Thanks Nick. I found this one. This library is focusing on a particular application I guess, seems only implemented one tensor factorization algorithm by far, and only for three dimensional tensors. Are there Spark powered libraries supporting general tensors and the algorithms?
>
> Best regards!
> Jiajia Li
>
> ------------------------------------------
> E-mail: jiajiali@gatech.edu <ma...@gatech.edu>
> Tel: +1 (404)9404603
> Computational Science & Engineering
> Georgia Institute of Technology
>
>> On Feb 23, 2016, at 11:12 PM, Nick Pentreath <nick.pentreath@gmail.com <ma...@gmail.com>> wrote:
>>
>> There is this library that I've come across - https://github.com/FurongHuang/SpectralLDA-TensorSpark <https://github.com/FurongHuang/SpectralLDA-TensorSpark>
>> On Wed, 24 Feb 2016 at 05:50, Li Jiajia <jiajiali@gatech.edu <ma...@gatech.edu>> wrote:
>> Hi,
>> I wonder if there are tensor algorithms or tensor data structures supported by Spark MLlib or GraphX. In a Spark intro slide, tensor factorization is mentioned as one of the algorithms in GraphX, but I didn't find it in the guide. If not, do you plan to implement them in the future?
>> I’m new to Spark, please give some detailed explanation if possible. Thanks in advance.
>>
>> Best regards!
>> Jiajia Li
>>
>> ------------------------------------------
>> E-mail: jiajiali@gatech.edu <ma...@gatech.edu>
>> Tel: +1 (404)9404603
>> Computational Science & Engineering
>> Georgia Institute of Technology
>>
>
Re: About Tensor Factorization in Spark
Posted by Li Jiajia <ji...@gatech.edu>.
Thanks Pooja! This is basically for TensorFlow. Seems don’t have many tensor features, basically matrix operations after the input part.
Best regards!
Jiajia Li
------------------------------------------
E-mail: jiajiali@gatech.edu
Tel: +1 (404)9404603
Computational Science & Engineering
Georgia Institute of Technology
> On Feb 24, 2016, at 2:42 PM, Chadha Pooja <Ch...@bcg.com> wrote:
>
> Hi,
>
> Here is a link that might help - https://databricks.com/blog/2016/01/25/deep-learning-with-spark-and-tensorflow.html <https://databricks.com/blog/2016/01/25/deep-learning-with-spark-and-tensorflow.html>
>
> Thanks
> Pooja
>
> From: Nick Pentreath [mailto:nick.pentreath@gmail.com <ma...@gmail.com>]
> Sent: Wednesday, February 24, 2016 1:11 AM
> To: user@spark.apache.org <ma...@spark.apache.org>
> Subject: Re: About Tensor Factorization in Spark
>
> Not that I'm aware of - it would be a great addition as a Spark package!
>
> On Wed, 24 Feb 2016 at 06:33 Li Jiajia <jiajiali@gatech.edu <ma...@gatech.edu>> wrote:
> Thanks Nick. I found this one. This library is focusing on a particular application I guess, seems only implemented one tensor factorization algorithm by far, and only for three dimensional tensors. Are there Spark powered libraries supporting general tensors and the algorithms?
>
> Best regards!
> Jiajia Li
>
> ------------------------------------------
> E-mail: jiajiali@gatech.edu <ma...@gatech.edu>
> Tel: +1 (404)9404603
> Computational Science & Engineering
> Georgia Institute of Technology
>
> On Feb 23, 2016, at 11:12 PM, Nick Pentreath <nick.pentreath@gmail.com <ma...@gmail.com>> wrote:
>
> There is this library that I've come across - https://github.com/FurongHuang/SpectralLDA-TensorSpark <https://github.com/FurongHuang/SpectralLDA-TensorSpark>
>
> On Wed, 24 Feb 2016 at 05:50, Li Jiajia <jiajiali@gatech.edu <ma...@gatech.edu>> wrote:
> Hi,
> I wonder if there are tensor algorithms or tensor data structures supported by Spark MLlib or GraphX. In a Spark intro slide, tensor factorization is mentioned as one of the algorithms in GraphX, but I didn't find it in the guide. If not, do you plan to implement them in the future?
> I’m new to Spark, please give some detailed explanation if possible. Thanks in advance.
>
> Best regards!
> Jiajia Li
>
> ------------------------------------------
> E-mail: jiajiali@gatech.edu <ma...@gatech.edu>
> Tel: +1 (404)9404603
> Computational Science & Engineering
> Georgia Institute of Technology
>
>
>
>
> The Boston Consulting Group, Inc.
>
> This e-mail message may contain confidential and/or privileged information. If you are not an addressee or otherwise authorized to receive this message, you should not use, copy, disclose or take any action based on this e-mail or any information contained in the message. If you have received this material in error, please advise the sender immediately by reply e-mail and delete this message. Thank you.
RE: About Tensor Factorization in Spark
Posted by Chadha Pooja <Ch...@bcg.com>.
Hi,
Here is a link that might help - https://databricks.com/blog/2016/01/25/deep-learning-with-spark-and-tensorflow.html
Thanks
Pooja
From: Nick Pentreath [mailto:nick.pentreath@gmail.com]
Sent: Wednesday, February 24, 2016 1:11 AM
To: user@spark.apache.org
Subject: Re: About Tensor Factorization in Spark
Not that I'm aware of - it would be a great addition as a Spark package!
On Wed, 24 Feb 2016 at 06:33 Li Jiajia <ji...@gatech.edu>> wrote:
Thanks Nick. I found this one. This library is focusing on a particular application I guess, seems only implemented one tensor factorization algorithm by far, and only for three dimensional tensors. Are there Spark powered libraries supporting general tensors and the algorithms?
Best regards!
Jiajia Li
------------------------------------------
E-mail: jiajiali@gatech.edu<ma...@gatech.edu>
Tel: +1 (404)9404603
Computational Science & Engineering
Georgia Institute of Technology
On Feb 23, 2016, at 11:12 PM, Nick Pentreath <ni...@gmail.com>> wrote:
There is this library that I've come across - https://github.com/FurongHuang/SpectralLDA-TensorSpark
On Wed, 24 Feb 2016 at 05:50, Li Jiajia <ji...@gatech.edu>> wrote:
Hi,
I wonder if there are tensor algorithms or tensor data structures supported by Spark MLlib or GraphX. In a Spark intro slide, tensor factorization is mentioned as one of the algorithms in GraphX, but I didn't find it in the guide. If not, do you plan to implement them in the future?
I’m new to Spark, please give some detailed explanation if possible. Thanks in advance.
Best regards!
Jiajia Li
------------------------------------------
E-mail: jiajiali@gatech.edu<ma...@gatech.edu>
Tel: +1 (404)9404603
Computational Science & Engineering
Georgia Institute of Technology
______________________________________________________________________________
The Boston Consulting Group, Inc.
This e-mail message may contain confidential and/or privileged information.
If you are not an addressee or otherwise authorized to receive this message,
you should not use, copy, disclose or take any action based on this e-mail or
any information contained in the message. If you have received this material
in error, please advise the sender immediately by reply e-mail and delete this
message. Thank you.
Re: About Tensor Factorization in Spark
Posted by Nick Pentreath <ni...@gmail.com>.
Not that I'm aware of - it would be a great addition as a Spark package!
On Wed, 24 Feb 2016 at 06:33 Li Jiajia <ji...@gatech.edu> wrote:
> Thanks Nick. I found this one. This library is focusing on a particular
> application I guess, seems only implemented one tensor factorization
> algorithm by far, and only for three dimensional tensors. Are there Spark
> powered libraries supporting general tensors and the algorithms?
>
> Best regards!
> Jiajia Li
>
> ------------------------------------------
> E-mail: jiajiali@gatech.edu
> Tel: +1 (404)9404603
> Computational Science & Engineering
> Georgia Institute of Technology
>
> On Feb 23, 2016, at 11:12 PM, Nick Pentreath <ni...@gmail.com>
> wrote:
>
> There is this library that I've come across -
> https://github.com/FurongHuang/SpectralLDA-TensorSpark
>
> On Wed, 24 Feb 2016 at 05:50, Li Jiajia <ji...@gatech.edu> wrote:
>
>> Hi,
>> I wonder if there are tensor algorithms or tensor data structures
>> supported by Spark MLlib or GraphX. In a Spark intro slide, tensor
>> factorization is mentioned as one of the algorithms in GraphX, but I didn't
>> find it in the guide. If not, do you plan to implement them in the future?
>> I’m new to Spark, please give some detailed explanation if
>> possible. Thanks in advance.
>>
>> Best regards!
>> Jiajia Li
>>
>> ------------------------------------------
>> E-mail: jiajiali@gatech.edu
>> Tel: +1 (404)9404603
>> Computational Science & Engineering
>> Georgia Institute of Technology
>>
>>
>
Re: About Tensor Factorization in Spark
Posted by Li Jiajia <ji...@gatech.edu>.
Thanks Nick. I found this one. This library is focusing on a particular application I guess, seems only implemented one tensor factorization algorithm by far, and only for three dimensional tensors. Are there Spark powered libraries supporting general tensors and the algorithms?
Best regards!
Jiajia Li
------------------------------------------
E-mail: jiajiali@gatech.edu
Tel: +1 (404)9404603
Computational Science & Engineering
Georgia Institute of Technology
> On Feb 23, 2016, at 11:12 PM, Nick Pentreath <ni...@gmail.com> wrote:
>
> There is this library that I've come across - https://github.com/FurongHuang/SpectralLDA-TensorSpark <https://github.com/FurongHuang/SpectralLDA-TensorSpark>
> On Wed, 24 Feb 2016 at 05:50, Li Jiajia <jiajiali@gatech.edu <ma...@gatech.edu>> wrote:
> Hi,
> I wonder if there are tensor algorithms or tensor data structures supported by Spark MLlib or GraphX. In a Spark intro slide, tensor factorization is mentioned as one of the algorithms in GraphX, but I didn't find it in the guide. If not, do you plan to implement them in the future?
> I’m new to Spark, please give some detailed explanation if possible. Thanks in advance.
>
> Best regards!
> Jiajia Li
>
> ------------------------------------------
> E-mail: jiajiali@gatech.edu <ma...@gatech.edu>
> Tel: +1 (404)9404603
> Computational Science & Engineering
> Georgia Institute of Technology
>
Re: About Tensor Factorization in Spark
Posted by Nick Pentreath <ni...@gmail.com>.
There is this library that I've come across -
https://github.com/FurongHuang/SpectralLDA-TensorSpark
On Wed, 24 Feb 2016 at 05:50, Li Jiajia <ji...@gatech.edu> wrote:
> Hi,
> I wonder if there are tensor algorithms or tensor data structures
> supported by Spark MLlib or GraphX. In a Spark intro slide, tensor
> factorization is mentioned as one of the algorithms in GraphX, but I didn't
> find it in the guide. If not, do you plan to implement them in the future?
> I’m new to Spark, please give some detailed explanation if
> possible. Thanks in advance.
>
> Best regards!
> Jiajia Li
>
> ------------------------------------------
> E-mail: jiajiali@gatech.edu
> Tel: +1 (404)9404603
> Computational Science & Engineering
> Georgia Institute of Technology
>
>