You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@phoenix.apache.org by "Gabriel Reid (JIRA)" <ji...@apache.org> on 2014/12/11 13:39:13 UTC
[jira] [Commented] (PHOENIX-1516) Add RANDOM built-in function
[ https://issues.apache.org/jira/browse/PHOENIX-1516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14242481#comment-14242481 ]
Gabriel Reid commented on PHOENIX-1516:
---------------------------------------
+1 to the idea of adding this kind of function.
It seems ok to me that it isn't possible to set a random seed -- in my own use cases, setting a random seed is typically only useful for making sure that something is totally deterministic, but it seems like there are enough things going on all over the place (in different JVMs, different threads, etc), that it would be difficult to ensure that we have a fully deterministic random value coming out for each row.
The fact that the same random will be generated for multiple columns in the same row seems like a bit of a show-stopper to me. As a user, I would expect that multiple calls to RANDOM() will give multiple (different) random values.
One point that also is probably worth looking into is the creation of a new Random() in each call to evaluate(). The default constructor of Random() calls some code that has class-level locks, so there could be some lock contention if multiple threads in the same JVM are calling {{new Random()}} over and over.
> Add RANDOM built-in function
> ----------------------------
>
> Key: PHOENIX-1516
> URL: https://issues.apache.org/jira/browse/PHOENIX-1516
> Project: Phoenix
> Issue Type: Bug
> Reporter: Lars Hofhansl
> Assignee: Lars Hofhansl
> Attachments: 1516.txt
>
>
> I often find it useful to generate some rows with random data.
> Here's a simple RANDOM() function that we could use for that.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Re: [jira] [Created] (PHOENIX-1516) Add RANDOM built-in function
Posted by lars hofhansl <la...@apache.org>.
I have a bias against thread local from past experiences. We actually introduced severe performance degradation with ThreadLocal, we have thread state to clean up (potentially) between requests, etc.Let's do it only when we know we have to.
(also let's discuss on the jira :) )
-- Lars
From: Andrew Purtell <ap...@apache.org>
To: "dev@phoenix.apache.org" <de...@phoenix.apache.org>
Sent: Thursday, December 11, 2014 8:19 AM
Subject: Re: [jira] [Created] (PHOENIX-1516) Add RANDOM built-in function
TLR is Java 7 or later. Not a problem if not supporting Java 6. HBase 0.98
still needs to support Java 6. What we did instead is make thread-local
Randoms: https://issues.apache.org/jira/browse/HBASE-12529
On Thu, Dec 11, 2014 at 5:05 AM, Samarth Jain <sa...@gmail.com>
wrote:
> An alternative to get rid of lock contention in Random() would be to use
> ThreadLocalRandom -
>
> https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ThreadLocalRandom.html
>
>
> On Thursday, December 11, 2014, Gabriel Reid (JIRA) <ji...@apache.org>
> wrote:
>
> >
> > [
> >
> https://issues.apache.org/jira/browse/PHOENIX-1516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14242481#comment-14242481
> > ]
> >
> > Gabriel Reid commented on PHOENIX-1516:
> > ---------------------------------------
> >
> > +1 to the idea of adding this kind of function.
> >
> > It seems ok to me that it isn't possible to set a random seed -- in my
> own
> > use cases, setting a random seed is typically only useful for making sure
> > that something is totally deterministic, but it seems like there are
> enough
> > things going on all over the place (in different JVMs, different threads,
> > etc), that it would be difficult to ensure that we have a fully
> > deterministic random value coming out for each row.
> >
> > The fact that the same random will be generated for multiple columns in
> > the same row seems like a bit of a show-stopper to me. As a user, I would
> > expect that multiple calls to RANDOM() will give multiple (different)
> > random values.
> >
> > One point that also is probably worth looking into is the creation of a
> > new Random() in each call to evaluate(). The default constructor of
> > Random() calls some code that has class-level locks, so there could be
> some
> > lock contention if multiple threads in the same JVM are calling {{new
> > Random()}} over and over.
> >
> > > Add RANDOM built-in function
> > > ----------------------------
> > >
> > > Key: PHOENIX-1516
> > > URL:
> https://issues.apache.org/jira/browse/PHOENIX-1516
> > > Project: Phoenix
> > > Issue Type: Bug
> > > Reporter: Lars Hofhansl
> > > Assignee: Lars Hofhansl
> > > Attachments: 1516.txt
> > >
> > >
> > > I often find it useful to generate some rows with random data.
> > > Here's a simple RANDOM() function that we could use for that.
> >
> >
> >
> > --
> > This message was sent by Atlassian JIRA
> > (v6.3.4#6332)
> >
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: [jira] [Created] (PHOENIX-1516) Add RANDOM built-in function
Posted by Andrew Purtell <ap...@apache.org>.
TLR is Java 7 or later. Not a problem if not supporting Java 6. HBase 0.98
still needs to support Java 6. What we did instead is make thread-local
Randoms: https://issues.apache.org/jira/browse/HBASE-12529
On Thu, Dec 11, 2014 at 5:05 AM, Samarth Jain <sa...@gmail.com>
wrote:
> An alternative to get rid of lock contention in Random() would be to use
> ThreadLocalRandom -
>
> https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ThreadLocalRandom.html
>
>
> On Thursday, December 11, 2014, Gabriel Reid (JIRA) <ji...@apache.org>
> wrote:
>
> >
> > [
> >
> https://issues.apache.org/jira/browse/PHOENIX-1516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14242481#comment-14242481
> > ]
> >
> > Gabriel Reid commented on PHOENIX-1516:
> > ---------------------------------------
> >
> > +1 to the idea of adding this kind of function.
> >
> > It seems ok to me that it isn't possible to set a random seed -- in my
> own
> > use cases, setting a random seed is typically only useful for making sure
> > that something is totally deterministic, but it seems like there are
> enough
> > things going on all over the place (in different JVMs, different threads,
> > etc), that it would be difficult to ensure that we have a fully
> > deterministic random value coming out for each row.
> >
> > The fact that the same random will be generated for multiple columns in
> > the same row seems like a bit of a show-stopper to me. As a user, I would
> > expect that multiple calls to RANDOM() will give multiple (different)
> > random values.
> >
> > One point that also is probably worth looking into is the creation of a
> > new Random() in each call to evaluate(). The default constructor of
> > Random() calls some code that has class-level locks, so there could be
> some
> > lock contention if multiple threads in the same JVM are calling {{new
> > Random()}} over and over.
> >
> > > Add RANDOM built-in function
> > > ----------------------------
> > >
> > > Key: PHOENIX-1516
> > > URL:
> https://issues.apache.org/jira/browse/PHOENIX-1516
> > > Project: Phoenix
> > > Issue Type: Bug
> > > Reporter: Lars Hofhansl
> > > Assignee: Lars Hofhansl
> > > Attachments: 1516.txt
> > >
> > >
> > > I often find it useful to generate some rows with random data.
> > > Here's a simple RANDOM() function that we could use for that.
> >
> >
> >
> > --
> > This message was sent by Atlassian JIRA
> > (v6.3.4#6332)
> >
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: [jira] [Created] (PHOENIX-1516) Add RANDOM built-in function
Posted by Samarth Jain <sa...@gmail.com>.
An alternative to get rid of lock contention in Random() would be to use
ThreadLocalRandom -
https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ThreadLocalRandom.html
On Thursday, December 11, 2014, Gabriel Reid (JIRA) <ji...@apache.org> wrote:
>
> [
> https://issues.apache.org/jira/browse/PHOENIX-1516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14242481#comment-14242481
> ]
>
> Gabriel Reid commented on PHOENIX-1516:
> ---------------------------------------
>
> +1 to the idea of adding this kind of function.
>
> It seems ok to me that it isn't possible to set a random seed -- in my own
> use cases, setting a random seed is typically only useful for making sure
> that something is totally deterministic, but it seems like there are enough
> things going on all over the place (in different JVMs, different threads,
> etc), that it would be difficult to ensure that we have a fully
> deterministic random value coming out for each row.
>
> The fact that the same random will be generated for multiple columns in
> the same row seems like a bit of a show-stopper to me. As a user, I would
> expect that multiple calls to RANDOM() will give multiple (different)
> random values.
>
> One point that also is probably worth looking into is the creation of a
> new Random() in each call to evaluate(). The default constructor of
> Random() calls some code that has class-level locks, so there could be some
> lock contention if multiple threads in the same JVM are calling {{new
> Random()}} over and over.
>
> > Add RANDOM built-in function
> > ----------------------------
> >
> > Key: PHOENIX-1516
> > URL: https://issues.apache.org/jira/browse/PHOENIX-1516
> > Project: Phoenix
> > Issue Type: Bug
> > Reporter: Lars Hofhansl
> > Assignee: Lars Hofhansl
> > Attachments: 1516.txt
> >
> >
> > I often find it useful to generate some rows with random data.
> > Here's a simple RANDOM() function that we could use for that.
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)
>