You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sedona.apache.org by 可为 <mw...@qq.com.INVALID> on 2022/11/29 05:29:02 UTC

Read a big geotiff,it went memory overflow error

&nbsp; &nbsp; Hello, I used Sedona Spark to read a 1GB geotiff, and the memory overflowed. Does Sedona read raster data in pieces? How to set the number of rdd partitions for reading grid data?




可为
mwhei@qq.com



&nbsp;

Re: Read a big geotiff,it went memory overflow error

Posted by Jia Yu <ji...@apache.org>.
Hi,

Each GeoTiff image is treated as a single record in Sedona DataFrame. It
means that the image itself is not partitioned and cannot be too large. The
use scenario of Sedona is that you have tons of small GeoTiff images rather
than a single big image.

Thanks,
Jia

On Thu, Dec 1, 2022 at 11:52 PM 可为 <mw...@qq.com.invalid> wrote:

> &nbsp; &nbsp; Hello, I used Sedona Spark to read a 1GB geotiff, and the
> memory overflowed. Does Sedona read raster data in pieces? How to set the
> number of rdd partitions for reading grid data?
>
>
>
>
> 可为
> mwhei@qq.com
>
>
>
> &nbsp;