You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by Richard Grossman <ri...@gmail.com> on 2015/05/19 23:36:30 UTC

How to set the file size for parquet Part

Hi

I'm using spark 1.3.1 and now I can't set the size of the part generated
file for parquet.
The size is only 512Kb it's really to small I must made them bigger.
How can set this ?
Thanks

Re: How to set the file size for parquet Part

Posted by Akhil Das <ak...@sigmoidanalytics.com>.

How many part files are you having? Did you try re-partitioning to a
smaller number so that you will have bigger files of smaller number.

Thanks
Best Regards

On Wed, May 20, 2015 at 3:06 AM, Richard Grossman <ri...@gmail.com>
wrote:

> Hi
>
> I'm using spark 1.3.1 and now I can't set the size of the part generated
> file for parquet.
> The size is only 512Kb it's really to small I must made them bigger.
> How can set this ?
> Thanks
>