Re: rya parser failed on parsing large numeric type

[David Lotts <dl...@gmail.com>]

> Good idea, but why can’t you use LongEncoder which is also 8 bytes?
It's great that it works for you!  It would probably work for lots of cases
since a long is large.
I am seeing 16 bytes.  Maybe we are looking at something different:
org.calrissian.mango.types.encoders.lexi.LongEncoder.class

However it does not meet the RDF specifications which I believe allow it to
be unlimited length.
I'm going to paste my solution above into RYA-43 in case someone wants to
implement it.

david.

On Wed, Jan 11, 2017 at 1:20 PM, Ly, Kiet <Ki...@finra.org> wrote:

> Good idea, but why can’t you use LongEncoder which is also 8 bytes?
>
> BigInteger.parseInt(array, start, end) required array conversion and extra
> start,end parameters.
>
>
>
> I replaced Integer to Long and it worked for me.
>
> public static final TypeEncoder<Long, String> INTEGER_STRING_TYPE_ENCODER
> = LexiTypeEncoders.longEncoder();
>
>
>
> On 1/11/17, 12:51 PM, "David Lotts" <dl...@gmail.com> wrote:
>
>
>
>     I had an idea to make this work backwardly compatible.  It should not
> break
>
>     existing Rya repositories:
>
>     Encode the java sized integers as-is, then for anything out of range,
> use
>
>     MAX/MIN and concatenate the new big integer encoding.
>
>
>
>     Here is the current way of encoding returning a string:
>
>                 return INTEGER_STRING_TYPE_ENCODER.
>
>     encode(Integer.parseInt(data));
>
>
>
>     Here is my replacement:
>
>
>
>     if  (value >= Integer.MAX) { //  value is a string, fix this with
>
>     parseint() and catch or similar
>
>         return INTEGER_STRING_TYPE_ENCODER.encode(Integer.MAX) +
>
>     bigIntegerEncode(value) ;
>
>     } else if (value <= Integer.MIN) {
>
>         return INTEGER_STRING_TYPE_ENCODER.encode(Integer.MIN)  +
>
>     bigIntegerEncode(value) ;
>
>     } else {
>
>                 return INTEGER_STRING_TYPE_ENCODER.
>
>     encode(Integer.parseInt(data));
>
>     }
>
>
>
>     The only disadvantage I see is that every big integer literal that you
>
>     store will have an extra 8 bytes.  Regular integers are unencumbered.
>
>
>
>     david.
>
>
>
>     On Tue, Jan 10, 2017 at 3:32 PM, David Lotts <dl...@gmail.com> wrote:
>
>
>
>     > > RDF parser confused between large numeric data type with integer.
> Any
>
>     > work around for this?  This is a recent build from master branch
> 3.2.10 I
>
>     > think.
>
>     >
>
>     > This issue is reported here: https://issues.apache.org/
> jira/browse/RYA-43
>
>     >
>
>     > This has a mechanically simple fix, but it breaks existing
>
>     > implementations.  So making it backward compatible is probably why
> it has
>
>     > not been done yet.  Backward compatible can be done by designing a
> way to
>
>     > mix the encoding from LexiTypeEncoders.bigIntegerEncoder() with the
>
>     > existing LexiTypeEncoders.integerEncoder().  Or create a new Lexical
>
>     > encoder, that handles both, or maybe upgrade utility to modify the
> data.
>
>     >
>
>     > So you could do this as a work around in your own code if you have
> luxury
>
>     > of starting from an empty Rya repo:
>
>     >
>
>     > Copy /rya.api/src/main/java/org/apache/rya/api/resolver/impl/
> IntegerRyaTypeResolver.java
>
>     > to LittleIntegerRyaTypeResolver.java
>
>     >
>
>     > Then, in IntegerRyaTypeResolver.java, where-ever it uses a Integer,
>
>     > replace with a BigInteger.
>
>     > Replace LexiTypeEncoders.integerEncoder()  with LexiTypeEncoders.
>
>     > bigIntegerEncoder()
>
>     >     https://github.com/calrissian/mango/blob/master/mango-core/
>
>     > src/main/java/org/calrissian/mango/types/encoders/lexi/
>
>     > BigIntegerEncoder.java
>
>     >
>
>     > Test and done!
>
>     >
>
>     > david.
>
>     >
>
>
>
> Confidentiality Notice::  This email, including attachments, may include
> non-public, proprietary, confidential or legally privileged information.
> If you are not an intended recipient or an authorized agent of an intended
> recipient, you are hereby notified that any dissemination, distribution or
> copying of the information contained in or transmitted with this e-mail is
> unauthorized and strictly prohibited.  If you have received this email in
> error, please notify the sender by replying to this message and permanently
> delete this e-mail, its attachments, and any copies of it immediately.  You
> should not retain, copy or use this e-mail or any attachment for any
> purpose, nor disclose all or any part of the contents to any other person.
> Thank you.
>