[jira] [Commented] (APEXMALHAR-2263) Offsets in AbstractFileInputOperator should be long rather than int

["Munagala V. Ramanath (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/APEXMALHAR-2263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15563965#comment-15563965 ] 

Munagala V. Ramanath commented on APEXMALHAR-2263:
--------------------------------------------------

To provide a bit more detail, imagine a file with small lines, maybe 20 chars each; that's
adequate, for example a timestamp and id of some sort.

At Integer.MAX_VALUE lines, we have 85_899_345_900 bytes or ~79 GB. By today's standards
where TB size files are not uncommon, we can easily overflow ints.

> Offsets in AbstractFileInputOperator should be long rather than int
> -------------------------------------------------------------------
>
>                 Key: APEXMALHAR-2263
>                 URL: https://issues.apache.org/jira/browse/APEXMALHAR-2263
>             Project: Apache Apex Malhar
>          Issue Type: Bug
>          Components: adapters other
>            Reporter: Munagala V. Ramanath
>            Assignee: Matt Zhang
>
> Offsets in AbstractFileInputOperator use the int type which means files with more that (2**31 -1) records will cause overflows and mysterious failures.
> Should be changed to use long.
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)