You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Banty Sharma <ba...@gmail.com> on 2014/05/14 08:58:17 UTC
Regarding jira issue
Hii,
i want to work on issue mapreduce-5851.. problem is that
i can see many parameters in Distcp class. in which parameter do we need to
enable regular expressions?
private static final String usage = NAME
+ " [OPTIONS] <srcurl>* <desturl>" +
"\n\nOPTIONS:" +
"\n-p[rbugp] Preserve status" +
"\n r: replication number" +
"\n b: block size" +
"\n u: user" +
"\n g: group" +
"\n p: permission" +
"\n -p alone is equivalent to -prbugp" +
"\n-i Ignore failures" +
"\n-log <logdir> Write logs to <logdir>" +
"\n-m <num_maps> Maximum number of simultaneous copies" +
"\n-overwrite Overwrite destination" +
"\n-update Overwrite if src size different from dst
size" +
"\n-f <urilist_uri> Use list at <urilist_uri> as src list" +
"\n-filelimit <n> Limit the total number of files to be <= n"
+
"\n-sizelimit <n> Limit the total size to be <= n bytes" +
"\n-delete Delete the files existing in the dst but
not in src" +
"\n-mapredSslConf <f> Filename of SSL configuration for mapper
task" +
"\n\nNOTE 1: if -overwrite or -update are set, each source URI is " +
"\n interpreted as an isomorphic update to an existing
directory." +
"\nFor example:" +
"\nhadoop " + NAME + " -p -update \"hdfs://A:8020/user/foo/bar\" " +
"\"hdfs://B:8020/user/foo/baz\"\n" +
"\n would update all descendants of 'baz' also in 'bar'; it would
" +
"\n *not* update /user/foo/baz/bar" +
"\n\nNOTE 2: The parameter <n> in -filelimit and -sizelimit can be " +
"\n specified with symbolic representation. For examples," +
"\n 1230k = 1230 * 1024 = 1259520" +
"\n 891g = 891 * 1024^3 = 956703965184" +
"\n";
Thanks & Regards
Jhanver Sharma.
Re: Regarding jira issue
Posted by Nitin Pawar <ni...@gmail.com>.
If there no parameters defined for it, you can create a new one and add to
documentation that this parameter takes regex in this fashion
On Wed, May 14, 2014 at 12:28 PM, Banty Sharma <ba...@gmail.com>wrote:
> Hii,
>
> i want to work on issue mapreduce-5851.. problem is that
> i can see many parameters in Distcp class. in which parameter do we need
> to enable regular expressions?
>
> private static final String usage = NAME
> + " [OPTIONS] <srcurl>* <desturl>" +
> "\n\nOPTIONS:" +
> "\n-p[rbugp] Preserve status" +
> "\n r: replication number" +
> "\n b: block size" +
> "\n u: user" +
> "\n g: group" +
> "\n p: permission" +
> "\n -p alone is equivalent to -prbugp" +
> "\n-i Ignore failures" +
> "\n-log <logdir> Write logs to <logdir>" +
> "\n-m <num_maps> Maximum number of simultaneous copies" +
> "\n-overwrite Overwrite destination" +
> "\n-update Overwrite if src size different from dst
> size" +
> "\n-f <urilist_uri> Use list at <urilist_uri> as src list" +
> "\n-filelimit <n> Limit the total number of files to be <=
> n" +
> "\n-sizelimit <n> Limit the total size to be <= n bytes" +
> "\n-delete Delete the files existing in the dst but
> not in src" +
> "\n-mapredSslConf <f> Filename of SSL configuration for mapper
> task" +
>
> "\n\nNOTE 1: if -overwrite or -update are set, each source URI is " +
> "\n interpreted as an isomorphic update to an existing
> directory." +
> "\nFor example:" +
> "\nhadoop " + NAME + " -p -update \"hdfs://A:8020/user/foo/bar\" " +
> "\"hdfs://B:8020/user/foo/baz\"\n" +
> "\n would update all descendants of 'baz' also in 'bar'; it
> would " +
> "\n *not* update /user/foo/baz/bar" +
>
> "\n\nNOTE 2: The parameter <n> in -filelimit and -sizelimit can be "
> +
> "\n specified with symbolic representation. For examples," +
> "\n 1230k = 1230 * 1024 = 1259520" +
> "\n 891g = 891 * 1024^3 = 956703965184" +
>
> "\n";
>
>
> Thanks & Regards
> Jhanver Sharma.
>
--
Nitin Pawar
Re: Regarding jira issue
Posted by Nitin Pawar <ni...@gmail.com>.
If there no parameters defined for it, you can create a new one and add to
documentation that this parameter takes regex in this fashion
On Wed, May 14, 2014 at 12:28 PM, Banty Sharma <ba...@gmail.com>wrote:
> Hii,
>
> i want to work on issue mapreduce-5851.. problem is that
> i can see many parameters in Distcp class. in which parameter do we need
> to enable regular expressions?
>
> private static final String usage = NAME
> + " [OPTIONS] <srcurl>* <desturl>" +
> "\n\nOPTIONS:" +
> "\n-p[rbugp] Preserve status" +
> "\n r: replication number" +
> "\n b: block size" +
> "\n u: user" +
> "\n g: group" +
> "\n p: permission" +
> "\n -p alone is equivalent to -prbugp" +
> "\n-i Ignore failures" +
> "\n-log <logdir> Write logs to <logdir>" +
> "\n-m <num_maps> Maximum number of simultaneous copies" +
> "\n-overwrite Overwrite destination" +
> "\n-update Overwrite if src size different from dst
> size" +
> "\n-f <urilist_uri> Use list at <urilist_uri> as src list" +
> "\n-filelimit <n> Limit the total number of files to be <=
> n" +
> "\n-sizelimit <n> Limit the total size to be <= n bytes" +
> "\n-delete Delete the files existing in the dst but
> not in src" +
> "\n-mapredSslConf <f> Filename of SSL configuration for mapper
> task" +
>
> "\n\nNOTE 1: if -overwrite or -update are set, each source URI is " +
> "\n interpreted as an isomorphic update to an existing
> directory." +
> "\nFor example:" +
> "\nhadoop " + NAME + " -p -update \"hdfs://A:8020/user/foo/bar\" " +
> "\"hdfs://B:8020/user/foo/baz\"\n" +
> "\n would update all descendants of 'baz' also in 'bar'; it
> would " +
> "\n *not* update /user/foo/baz/bar" +
>
> "\n\nNOTE 2: The parameter <n> in -filelimit and -sizelimit can be "
> +
> "\n specified with symbolic representation. For examples," +
> "\n 1230k = 1230 * 1024 = 1259520" +
> "\n 891g = 891 * 1024^3 = 956703965184" +
>
> "\n";
>
>
> Thanks & Regards
> Jhanver Sharma.
>
--
Nitin Pawar
Re: Regarding jira issue
Posted by Nitin Pawar <ni...@gmail.com>.
If there no parameters defined for it, you can create a new one and add to
documentation that this parameter takes regex in this fashion
On Wed, May 14, 2014 at 12:28 PM, Banty Sharma <ba...@gmail.com>wrote:
> Hii,
>
> i want to work on issue mapreduce-5851.. problem is that
> i can see many parameters in Distcp class. in which parameter do we need
> to enable regular expressions?
>
> private static final String usage = NAME
> + " [OPTIONS] <srcurl>* <desturl>" +
> "\n\nOPTIONS:" +
> "\n-p[rbugp] Preserve status" +
> "\n r: replication number" +
> "\n b: block size" +
> "\n u: user" +
> "\n g: group" +
> "\n p: permission" +
> "\n -p alone is equivalent to -prbugp" +
> "\n-i Ignore failures" +
> "\n-log <logdir> Write logs to <logdir>" +
> "\n-m <num_maps> Maximum number of simultaneous copies" +
> "\n-overwrite Overwrite destination" +
> "\n-update Overwrite if src size different from dst
> size" +
> "\n-f <urilist_uri> Use list at <urilist_uri> as src list" +
> "\n-filelimit <n> Limit the total number of files to be <=
> n" +
> "\n-sizelimit <n> Limit the total size to be <= n bytes" +
> "\n-delete Delete the files existing in the dst but
> not in src" +
> "\n-mapredSslConf <f> Filename of SSL configuration for mapper
> task" +
>
> "\n\nNOTE 1: if -overwrite or -update are set, each source URI is " +
> "\n interpreted as an isomorphic update to an existing
> directory." +
> "\nFor example:" +
> "\nhadoop " + NAME + " -p -update \"hdfs://A:8020/user/foo/bar\" " +
> "\"hdfs://B:8020/user/foo/baz\"\n" +
> "\n would update all descendants of 'baz' also in 'bar'; it
> would " +
> "\n *not* update /user/foo/baz/bar" +
>
> "\n\nNOTE 2: The parameter <n> in -filelimit and -sizelimit can be "
> +
> "\n specified with symbolic representation. For examples," +
> "\n 1230k = 1230 * 1024 = 1259520" +
> "\n 891g = 891 * 1024^3 = 956703965184" +
>
> "\n";
>
>
> Thanks & Regards
> Jhanver Sharma.
>
--
Nitin Pawar
Re: Regarding jira issue
Posted by Nitin Pawar <ni...@gmail.com>.
If there no parameters defined for it, you can create a new one and add to
documentation that this parameter takes regex in this fashion
On Wed, May 14, 2014 at 12:28 PM, Banty Sharma <ba...@gmail.com>wrote:
> Hii,
>
> i want to work on issue mapreduce-5851.. problem is that
> i can see many parameters in Distcp class. in which parameter do we need
> to enable regular expressions?
>
> private static final String usage = NAME
> + " [OPTIONS] <srcurl>* <desturl>" +
> "\n\nOPTIONS:" +
> "\n-p[rbugp] Preserve status" +
> "\n r: replication number" +
> "\n b: block size" +
> "\n u: user" +
> "\n g: group" +
> "\n p: permission" +
> "\n -p alone is equivalent to -prbugp" +
> "\n-i Ignore failures" +
> "\n-log <logdir> Write logs to <logdir>" +
> "\n-m <num_maps> Maximum number of simultaneous copies" +
> "\n-overwrite Overwrite destination" +
> "\n-update Overwrite if src size different from dst
> size" +
> "\n-f <urilist_uri> Use list at <urilist_uri> as src list" +
> "\n-filelimit <n> Limit the total number of files to be <=
> n" +
> "\n-sizelimit <n> Limit the total size to be <= n bytes" +
> "\n-delete Delete the files existing in the dst but
> not in src" +
> "\n-mapredSslConf <f> Filename of SSL configuration for mapper
> task" +
>
> "\n\nNOTE 1: if -overwrite or -update are set, each source URI is " +
> "\n interpreted as an isomorphic update to an existing
> directory." +
> "\nFor example:" +
> "\nhadoop " + NAME + " -p -update \"hdfs://A:8020/user/foo/bar\" " +
> "\"hdfs://B:8020/user/foo/baz\"\n" +
> "\n would update all descendants of 'baz' also in 'bar'; it
> would " +
> "\n *not* update /user/foo/baz/bar" +
>
> "\n\nNOTE 2: The parameter <n> in -filelimit and -sizelimit can be "
> +
> "\n specified with symbolic representation. For examples," +
> "\n 1230k = 1230 * 1024 = 1259520" +
> "\n 891g = 891 * 1024^3 = 956703965184" +
>
> "\n";
>
>
> Thanks & Regards
> Jhanver Sharma.
>
--
Nitin Pawar