You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by "Chris Teoh (JIRA)" <ji...@apache.org> on 2017/09/26 23:22:00 UTC
[jira] [Comment Edited] (SQOOP-3225) Mainframe module FTP listing
parser should cater for larger datasets on disk
[ https://issues.apache.org/jira/browse/SQOOP-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16181736#comment-16181736 ]
Chris Teoh edited comment on SQOOP-3225 at 9/26/17 11:21 PM:
-------------------------------------------------------------
Note this is included in diff above which covers SQOOP-3224 and SQOOP-3237 as well.
was (Author: chris.teoh@gmail.com):
Note this is included in diff above which covers SQOOP-3224 and SQOOP-3237
> Mainframe module FTP listing parser should cater for larger datasets on disk
> ----------------------------------------------------------------------------
>
> Key: SQOOP-3225
> URL: https://issues.apache.org/jira/browse/SQOOP-3225
> Project: Sqoop
> Issue Type: Improvement
> Reporter: Chris Teoh
> Assignee: Chris Teoh
> Priority: Minor
>
> The parser can handle smaller datasets but when a dataset reaches a certain size, the FTP listing changes such that the current FTP parser doesn't read the listing correctly. Current workaround is to move the datasets to tape. Sample is below of some tape based datasets followed by a large dataset followed by a small dataset:-
> {{ftp> ls
> 227 Entering Passive Mode (Xx,XX,XXX,XXX,XX,XXX)
> 125 List started OK
> Volume Unit Referred Ext Used Recfm Lrecl BlkSz Dsorg Dsname
> Migrated G1622V00
> Migrated G1623V00
> Migrated G1624V00
> Migrated G1625V00
> Migrated G1626V00
> Migrated G1627V00
> Migrated G1628V00
> Migrated G1629V00
> Migrated G1630V00
> Migrated G1631V00
> Migrated G1632V00
> Migrated G1633V00
> Migrated G1634V00
> Migrated G1635V00
> Migrated G1636V00
> Migrated G1637V00
> Migrated G1638V00
> Migrated G1639V00
> Migrated G1640V00
> xxxx39 3390 2017/08/02 633540 FB 847 27951 PS G1641V00
> xxxx09 3390 2017/08/02 1 15 FB 132 27984 PS OUTPUT}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)