Implementing ARRAY_APPEND built-in function

[Dumindu Buddhika <du...@gmail.com>]

Hi all,

I am a GSOC aspirant this year from Department of Computer Science and
Engineering, University of Moratuwa, Sri Lanka.

I would like to work on PHOENIX-1665, to create missing array built-in
functions. I am currently trying to implement ARRAY_APPEND function. I have
some questions associated with this. It would be great if someone can
answer.
PBinaryArray type would be first input of this function. Second would be a
primitive type element to be appended to the first given array. So I am
guessing it should be PDataType.

The approach I thought was using  PDataType's toObject method to construct
a PhoenixArray from given bytes(from ptr) and add the new element to it
then convert it back to bytes.

When using this approach I have a problem how to convert the second input
argument(the element to be appended to the array) to corresponding object.
if the type of this is known of this input for sure it can be done( ex: if
its PInteger then PInteger.INSTANCE.getCodec().decodeInt can be used). But
here it can be any of the primitive types to my understanding. so how to
convert to the corresponding datatype object without exactly knowing the
type?

It would be really great if someone can help me on this. if this approach
is wrong or if there is a better approach(directly manipulating bytes) I
would also like to know :)

Thank you.

Regards,
Dumindu.

Re: Implementing ARRAY_APPEND built-in function

[Dumindu Buddhika <du...@gmail.com>]

Hi James,

Thanks for the quick response. Thanks for helping out. I will try to
proceed with that information. Next time I will use Jira for communication
specific to the task.

Regards,
Dumindu.

Dumindu Buddhika,
Undergraduate,
Department of Computer Science and Engineering,
University of Moratuwa,
Sri Lanka.
Mobile:+94715764749

On Sat, Mar 7, 2015 at 7:59 AM, James Taylor <ja...@apache.org> wrote:

> Hi Dumindu,
>
> Good questions. For future, best to post questions specific to your
> JIRA on the JIRA itself. All JIRA comments appear in the dev list as
> well.
>
> For the @BuiltInFunction annotation, you'd define your second type as
> PVarbinary, since the element being appended may be of any type. To
> get the actual type at runtime, you'd use the getChildren() method
> which returns a List<Expression>. The children represent the arguments
> to your built-in function. In your case, getChildren().get(0)
> expression would represent the array, while the getChildren().get(1)
> expression would represent the element being appended. From the
> expression, you can call getDataType() to get the actual PDataType at
> runtime, and use the dataType.toBytes() method to get the bytes that
> represent the actual value which you can add to the array. Take a look
> at PArrayDataType for methods that manipulate the array. If possible,
> you'd want to keep the array as byte[], as converting back and forth
> to objects is expensive. You might need to add a new method to
> PArrayDataType that allows the byte[] of an element to be appended to
> the byte[] of the array.
>
> Note that you know the data types at compile time, so it's possible
> that you could create a different expression for the
> ArrayAppendFunction from a custom factory method. An example of this
> is with the RoundFunction - it has a nodeClass attribute in its
> annotation that delegate the creation to the indicated class. If you
> look at that class, you'll see that it ends up creating a
> RoundDateExpression, RoundTimestampExpression, or
> RoundDecimalExpression depending on the type of the argument. In this
> case, I don't think it's worth going this route, though.
>
> HTH. Thanks,
>
> James
>
> On Fri, Mar 6, 2015 at 5:16 PM, Dumindu Buddhika
> <du...@gmail.com> wrote:
> > Hi all,
> >
> > I am a GSOC aspirant this year from Department of Computer Science and
> > Engineering, University of Moratuwa, Sri Lanka.
> >
> > I would like to work on PHOENIX-1665, to create missing array built-in
> > functions. I am currently trying to implement ARRAY_APPEND function. I
> have
> > some questions associated with this. It would be great if someone can
> > answer.
> > PBinaryArray type would be first input of this function. Second would be
> a
> > primitive type element to be appended to the first given array. So I am
> > guessing it should be PDataType.
> >
> > The approach I thought was using  PDataType's toObject method to
> construct
> > a PhoenixArray from given bytes(from ptr) and add the new element to it
> > then convert it back to bytes.
> >
> > When using this approach I have a problem how to convert the second input
> > argument(the element to be appended to the array) to corresponding
> object.
> > if the type of this is known of this input for sure it can be done( ex:
> if
> > its PInteger then PInteger.INSTANCE.getCodec().decodeInt can be used).
> But
> > here it can be any of the primitive types to my understanding. so how to
> > convert to the corresponding datatype object without exactly knowing the
> > type?
> >
> > It would be really great if someone can help me on this. if this approach
> > is wrong or if there is a better approach(directly manipulating bytes) I
> > would also like to know :)
> >
> > Thank you.
> >
> > Regards,
> > Dumindu.
>

Re: Implementing ARRAY_APPEND built-in function

[James Taylor <ja...@apache.org>]

Hi Dumindu,

Good questions. For future, best to post questions specific to your
JIRA on the JIRA itself. All JIRA comments appear in the dev list as
well.

For the @BuiltInFunction annotation, you'd define your second type as
PVarbinary, since the element being appended may be of any type. To
get the actual type at runtime, you'd use the getChildren() method
which returns a List<Expression>. The children represent the arguments
to your built-in function. In your case, getChildren().get(0)
expression would represent the array, while the getChildren().get(1)
expression would represent the element being appended. From the
expression, you can call getDataType() to get the actual PDataType at
runtime, and use the dataType.toBytes() method to get the bytes that
represent the actual value which you can add to the array. Take a look
at PArrayDataType for methods that manipulate the array. If possible,
you'd want to keep the array as byte[], as converting back and forth
to objects is expensive. You might need to add a new method to
PArrayDataType that allows the byte[] of an element to be appended to
the byte[] of the array.

Note that you know the data types at compile time, so it's possible
that you could create a different expression for the
ArrayAppendFunction from a custom factory method. An example of this
is with the RoundFunction - it has a nodeClass attribute in its
annotation that delegate the creation to the indicated class. If you
look at that class, you'll see that it ends up creating a
RoundDateExpression, RoundTimestampExpression, or
RoundDecimalExpression depending on the type of the argument. In this
case, I don't think it's worth going this route, though.

HTH. Thanks,

James

On Fri, Mar 6, 2015 at 5:16 PM, Dumindu Buddhika
<du...@gmail.com> wrote:
> Hi all,
>
> I am a GSOC aspirant this year from Department of Computer Science and
> Engineering, University of Moratuwa, Sri Lanka.
>
> I would like to work on PHOENIX-1665, to create missing array built-in
> functions. I am currently trying to implement ARRAY_APPEND function. I have
> some questions associated with this. It would be great if someone can
> answer.
> PBinaryArray type would be first input of this function. Second would be a
> primitive type element to be appended to the first given array. So I am
> guessing it should be PDataType.
>
> The approach I thought was using  PDataType's toObject method to construct
> a PhoenixArray from given bytes(from ptr) and add the new element to it
> then convert it back to bytes.
>
> When using this approach I have a problem how to convert the second input
> argument(the element to be appended to the array) to corresponding object.
> if the type of this is known of this input for sure it can be done( ex: if
> its PInteger then PInteger.INSTANCE.getCodec().decodeInt can be used). But
> here it can be any of the primitive types to my understanding. so how to
> convert to the corresponding datatype object without exactly knowing the
> type?
>
> It would be really great if someone can help me on this. if this approach
> is wrong or if there is a better approach(directly manipulating bytes) I
> would also like to know :)
>
> Thank you.
>
> Regards,
> Dumindu.