You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@spark.apache.org by Yasemin Kaya <go...@gmail.com> on 2015/06/03 14:04:32 UTC
ALS Rating Object
Hi,
I want to use Spark's ALS in my project. I have the userid
like 30011397223227125563254 and Rating Object which is the Object of ALS
wants Integer as a userid so the id field does not fit into a 32 bit
Integer. How can I solve that ? Thanks.
Best,
yasemin
--
hiç ender hiç
Re: ALS Rating Object
Posted by Yasemin Kaya <go...@gmail.com>.
Hi Joseph,
I think about converting IDS but there will be birthday problem. The
probability of a Hash Collision
<http://preshing.com/20110504/hash-collision-probabilities/> is important
for me because of the user number. I don't know how can I modify ALS to use
Integer.
yasemin
2015-06-04 2:28 GMT+03:00 Joseph Bradley <jo...@databricks.com>:
> Hi Yasemin,
>
> If you can convert your user IDs to Integers in pre-processing (if you
> have < a couple billion users), that would work. Otherwise...
> In Spark 1.3: You may need to modify ALS to use Long instead of Int.
> In Spark 1.4: spark.ml.recommendation.ALS (in the Pipeline API) exposes
> ALS.train as a DeveloperApi to allow users to use Long instead of Int.
> We're also thinking about better ways to permit Long IDs.
>
> Joseph
>
> On Wed, Jun 3, 2015 at 5:04 AM, Yasemin Kaya <go...@gmail.com> wrote:
>
>> Hi,
>>
>> I want to use Spark's ALS in my project. I have the userid
>> like 30011397223227125563254 and Rating Object which is the Object of ALS
>> wants Integer as a userid so the id field does not fit into a 32 bit
>> Integer. How can I solve that ? Thanks.
>>
>> Best,
>> yasemin
>> --
>> hiç ender hiç
>>
>
>
--
hiç ender hiç
Re: ALS Rating Object
Posted by Joseph Bradley <jo...@databricks.com>.
Hi Yasemin,
If you can convert your user IDs to Integers in pre-processing (if you have
< a couple billion users), that would work. Otherwise...
In Spark 1.3: You may need to modify ALS to use Long instead of Int.
In Spark 1.4: spark.ml.recommendation.ALS (in the Pipeline API) exposes
ALS.train as a DeveloperApi to allow users to use Long instead of Int.
We're also thinking about better ways to permit Long IDs.
Joseph
On Wed, Jun 3, 2015 at 5:04 AM, Yasemin Kaya <go...@gmail.com> wrote:
> Hi,
>
> I want to use Spark's ALS in my project. I have the userid
> like 30011397223227125563254 and Rating Object which is the Object of ALS
> wants Integer as a userid so the id field does not fit into a 32 bit
> Integer. How can I solve that ? Thanks.
>
> Best,
> yasemin
> --
> hiç ender hiç
>