You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by "Chris Teoh (JIRA)" <ji...@apache.org> on 2018/05/29 00:36:00 UTC
[jira] [Updated] (SQOOP-3326) Mainframe FTP listing for GDG should
filter out non-GDG datasets in a heterogeneous listing
[ https://issues.apache.org/jira/browse/SQOOP-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Teoh updated SQOOP-3326:
------------------------------
Description:
The FTP listing will automatically assume the first file in the listing is the most recent GDG file. This is a problem when there are mixed datasets in the listing that the FTP listing doesn't filter these out.
GDG base is : HLQ.ABC.DEF.AB15HUP
The sequential dataset in the middle of the GDG member listing is : HLQ.ABC.DEF.AB15HUP.DATA
The pattern for listing GDG members should be : <<GDG base name>>.G\d\{4}V\d\{2}
Sample below:-
Menu Options View Utilities Compilers Help
ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss
DSLIST - Data Sets Matching HLQ.ABC.DEF.GDGBASE Row 1 of 8
Command ===> Scroll ===> PAGE
Command - Enter "/" to select action Message Volume
-------------------------------------------------------------------------------
HLQ.ABC.DEF.GDGBASE ??????
HLQ.ABC.DEF.GDGBASE.DUMMY SHT331
HLQ.ABC.DEF.GDGBASE.G0034V00 H19761
HLQ.ABC.DEF.GDGBASE.G0035V00 H81751
HLQ.ABC.DEF.GDGBASE.G0035V00.COPY SHT337
HLQ.ABC.DEF.GDGBASE.G0036V00 H73545
HLQ.ABC.DEF.GDGBASE.G0037V00 G10987
HLQ.ABC.DEF.GDGBASE.HELLO SHT33A
***************************** End of Data Set list ****************************
ftp> open some.machine.network.zxc.au
Connected to some.machine.network.zxc.au (11.22.33.44).
220-TCPFTP01 IBM FTP CS V2R1 at some.machine.network.zxc.au, 00:12:29 on 2018-05-29.
220 Connection will close if idle for more than 10 minutes.
Name (some.machine.network.zxc.au:someuser):
331 Send password please.
Password:
230 someuser is logged on. Working directory is "someuser.".
Remote system type is MVS.
ftp> cd 'HLQ.ABC.DEF.GDGBASE'
250 "HLQ.ABC.DEF.GDGBASE." is the working directory name prefix.
ftp> dir
227 Entering Passive Mode (11,22,33,44,55,66)
125 List started OK
Volume Unit Referred Ext Used Recfm Lrecl BlkSz Dsorg Dsname
H19761 Tape G0034V00
H81751 Tape G0035V00
H73545 Tape G0036V00
G10987 Tape G0037V00
SHT331 3390 **NONE** 1 15 VB 114 27998 PS DUMMY
SHT337 3390 **NONE** 1 15 VB 114 27998 PS G0035V00.COPY
SHT33A 3390 **NONE** 1 15 VB 114 27998 PS HELLO
250 List completed successfully.
ftp>
was:
The FTP listing will automatically assume the first file in the listing is the most recent GDG file. This is a problem when there are mixed datasets in the listing that the FTP listing doesn't filter these out.
GDG base is : HLQ.ABC.DEF.AB15HUP
The sequential dataset in the middle of the GDG member listing is : HLQ.ABC.DEF.AB15HUP.DATA
The pattern for listing GDG members should be : <<GDG base name>>.G\d\{4}V\d\{2}
Menu Options View Utilities Compilers Help
ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss
DSLIST - Data Sets Matching HLQ.ABC.DEF.AB15HUP Empty data set or member
Command ===> Scroll ===> PAGE
Command - Enter "/" to select action Message Volume
-------------------------------------------------------------------------------
HLQ.ABC.DEF.AB15HUP ??????
HLQ.ABC.DEF.AB15HUP.DATA ZXC344+
HLQ.ABC.DEF.AB15HUP.G0007V00 H54924
HLQ.ABC.DEF.AB15HUP.G0008V00 G54837
HLQ.ABC.DEF.AB15HUP.G0009V00 G53709
HLQ.ABC.DEF.AB15HUP.G0010V00 G27559
***************************** End of Data Set list ****************************
> Mainframe FTP listing for GDG should filter out non-GDG datasets in a heterogeneous listing
> -------------------------------------------------------------------------------------------
>
> Key: SQOOP-3326
> URL: https://issues.apache.org/jira/browse/SQOOP-3326
> Project: Sqoop
> Issue Type: Improvement
> Reporter: Chris Teoh
> Assignee: Chris Teoh
> Priority: Minor
>
> The FTP listing will automatically assume the first file in the listing is the most recent GDG file. This is a problem when there are mixed datasets in the listing that the FTP listing doesn't filter these out.
>
> GDG base is : HLQ.ABC.DEF.AB15HUP
>
> The sequential dataset in the middle of the GDG member listing is : HLQ.ABC.DEF.AB15HUP.DATA
>
> The pattern for listing GDG members should be : <<GDG base name>>.G\d\{4}V\d\{2}
>
> Sample below:-
> Menu Options View Utilities Compilers Help
> ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss
> DSLIST - Data Sets Matching HLQ.ABC.DEF.GDGBASE Row 1 of 8
> Command ===> Scroll ===> PAGE
>
> Command - Enter "/" to select action Message Volume
> -------------------------------------------------------------------------------
> HLQ.ABC.DEF.GDGBASE ??????
> HLQ.ABC.DEF.GDGBASE.DUMMY SHT331
> HLQ.ABC.DEF.GDGBASE.G0034V00 H19761
> HLQ.ABC.DEF.GDGBASE.G0035V00 H81751
> HLQ.ABC.DEF.GDGBASE.G0035V00.COPY SHT337
> HLQ.ABC.DEF.GDGBASE.G0036V00 H73545
> HLQ.ABC.DEF.GDGBASE.G0037V00 G10987
> HLQ.ABC.DEF.GDGBASE.HELLO SHT33A
> ***************************** End of Data Set list ****************************
> ftp> open some.machine.network.zxc.au
> Connected to some.machine.network.zxc.au (11.22.33.44).
> 220-TCPFTP01 IBM FTP CS V2R1 at some.machine.network.zxc.au, 00:12:29 on 2018-05-29.
> 220 Connection will close if idle for more than 10 minutes.
> Name (some.machine.network.zxc.au:someuser):
> 331 Send password please.
> Password:
> 230 someuser is logged on. Working directory is "someuser.".
> Remote system type is MVS.
> ftp> cd 'HLQ.ABC.DEF.GDGBASE'
> 250 "HLQ.ABC.DEF.GDGBASE." is the working directory name prefix.
> ftp> dir
> 227 Entering Passive Mode (11,22,33,44,55,66)
> 125 List started OK
> Volume Unit Referred Ext Used Recfm Lrecl BlkSz Dsorg Dsname
> H19761 Tape G0034V00
> H81751 Tape G0035V00
> H73545 Tape G0036V00
> G10987 Tape G0037V00
> SHT331 3390 **NONE** 1 15 VB 114 27998 PS DUMMY
> SHT337 3390 **NONE** 1 15 VB 114 27998 PS G0035V00.COPY
> SHT33A 3390 **NONE** 1 15 VB 114 27998 PS HELLO
> 250 List completed successfully.
> ftp>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)