Allocating additional memory to the Java Vector objects

[Atul Dambalkar <at...@xoriant.com>]

Hi,

I am creating IntVector in Java as follows -
IntVector intVector = (IntVector) vectorSchemaRoot.getVector(name);
intVector.setInitialCapacity(100);
intVector.allocateNew();

Is there a way that I can allocate additional capacity to the same IntVector object by a defined number? Let's say something like - intVector.allocateAdditional(100), which would only add more capacity to the existing buffer without impacting the existing buffer and data.

There is an API intVector.reAlloc, but it simply doubles the current allocated memory and not what I intend.

Thanks for your inputs,
-Atul

Re: Allocating additional memory to the Java Vector objects

[Siddharth Teotia <si...@dremio.com>]

Yes, explicitly invoking reAlloc() on vectors is not generally needed even
though it is provided as a public API. If the value capacity is not known
upfront or grows dynamically then setSafe() methods will take care of
internally expanding the buffer to store more data -- here we don't have
any fine grained control over the new size as we always double the buffer.

On Fri, Feb 23, 2018 at 4:49 PM, Atul Dambalkar <at...@xoriant.com>
wrote:

> Thanks Sidd. Actually, I was looking at the code in base classes for
> Vector implementation, and it does take care of reallocation itself (which
> I was thinking of doing explicitly in the code). Although it uses "reAlloc"
> which allocates double the current size,  for me it works - as I plan to
> start with moderate initial capacity for the vectors.
>
> -Atul
>
> -----Original Message-----
> From: Siddharth Teotia [mailto:siddharth@dremio.com]
> Sent: Friday, February 23, 2018 12:14 PM
> To: dev@arrow.apache.org
> Subject: Re: Allocating additional memory to the Java Vector objects
>
> Hi Atul,
>
> Currently there is no way for doing this. The only exposed method of
> expanding the vector buffer is reAlloc() and it allocates a new buffer of
> double the original capacity and copies the old contents into the new
> buffer.
>
> Thanks,
> Sidd
>
> On Fri, Feb 23, 2018 at 12:06 PM, Atul Dambalkar <
> atul.dambalkar@xoriant.com
> > wrote:
>
> > Hi,
> >
> > I am creating IntVector in Java as follows - IntVector intVector =
> > (IntVector) vectorSchemaRoot.getVector(name);
> > intVector.setInitialCapacity(100);
> > intVector.allocateNew();
> >
> > Is there a way that I can allocate additional capacity to the same
> > IntVector object by a defined number? Let's say something like -
> > intVector.allocateAdditional(100), which would only add more capacity
> > to the existing buffer without impacting the existing buffer and data.
> >
> > There is an API intVector.reAlloc, but it simply doubles the current
> > allocated memory and not what I intend.
> >
> > Thanks for your inputs,
> > -Atul
> >
> >
>

RE: Allocating additional memory to the Java Vector objects

[Atul Dambalkar <at...@xoriant.com>]

Thanks Sidd. Actually, I was looking at the code in base classes for Vector implementation, and it does take care of reallocation itself (which I was thinking of doing explicitly in the code). Although it uses "reAlloc" which allocates double the current size,  for me it works - as I plan to start with moderate initial capacity for the vectors. 

-Atul

-----Original Message-----
From: Siddharth Teotia [mailto:siddharth@dremio.com] 
Sent: Friday, February 23, 2018 12:14 PM
To: dev@arrow.apache.org
Subject: Re: Allocating additional memory to the Java Vector objects

Hi Atul,

Currently there is no way for doing this. The only exposed method of expanding the vector buffer is reAlloc() and it allocates a new buffer of double the original capacity and copies the old contents into the new buffer.

Thanks,
Sidd

On Fri, Feb 23, 2018 at 12:06 PM, Atul Dambalkar <atul.dambalkar@xoriant.com
> wrote:

> Hi,
>
> I am creating IntVector in Java as follows - IntVector intVector = 
> (IntVector) vectorSchemaRoot.getVector(name); 
> intVector.setInitialCapacity(100);
> intVector.allocateNew();
>
> Is there a way that I can allocate additional capacity to the same 
> IntVector object by a defined number? Let's say something like - 
> intVector.allocateAdditional(100), which would only add more capacity 
> to the existing buffer without impacting the existing buffer and data.
>
> There is an API intVector.reAlloc, but it simply doubles the current 
> allocated memory and not what I intend.
>
> Thanks for your inputs,
> -Atul
>
>

Re: Allocating additional memory to the Java Vector objects

[Siddharth Teotia <si...@dremio.com>]

Hi Atul,

Currently there is no way for doing this. The only exposed method of
expanding the vector buffer is reAlloc() and it allocates a new buffer of
double the original capacity and copies the old contents into the new
buffer.

Thanks,
Sidd

On Fri, Feb 23, 2018 at 12:06 PM, Atul Dambalkar <atul.dambalkar@xoriant.com
> wrote:

> Hi,
>
> I am creating IntVector in Java as follows -
> IntVector intVector = (IntVector) vectorSchemaRoot.getVector(name);
> intVector.setInitialCapacity(100);
> intVector.allocateNew();
>
> Is there a way that I can allocate additional capacity to the same
> IntVector object by a defined number? Let's say something like -
> intVector.allocateAdditional(100), which would only add more capacity to
> the existing buffer without impacting the existing buffer and data.
>
> There is an API intVector.reAlloc, but it simply doubles the current
> allocated memory and not what I intend.
>
> Thanks for your inputs,
> -Atul
>
>