You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by "Pastrana, Rodrigo (RIS-BCT)" <Ro...@lexisnexis.com> on 2013/01/10 15:59:25 UTC
WEBHDFS API GETCONTENTSUMMARY issue
I'm using WEBHDFS to query directory/file information, but the GETCONTENTSUMMARY counts aren't returning expected counts.
For example, when I query content summary for the directory /user/hadoop/tutorial, webhdfs returns the following:
x.y.z.w:50070/webhdfs/v1/user/hadoop/tutorial/?op=GETCONTENTSUMMARY
{"ContentSummary":{"directoryCount":4,"fileCount":10,"length":3204490622,"quota":-1,"spaceConsumed":3204490622,"spaceQuota":-1}}.
But looking at the content of that dir through the web portal, I see 6 files and 3 subdirs:
accounts<http://slave1:50075/browseDirectory.jsp?dir=%2Fuser%2Fhadoop%2Ftutorial%2Faccounts&namenodeInfoPort=50070>
file
812.93 MB
1
64 MB
2012-12-12 10:59
rw-r--r--
hadoop
supergroup
accounts2<http://slave1:50075/browseDirectory.jsp?dir=%2Fuser%2Fhadoop%2Ftutorial%2Faccounts2&namenodeInfoPort=50070>
file
812.93 MB
1
64 MB
2012-07-31 16:48
rw-r--r--
hadoop
supergroup
accounts2-parts<http://slave1:50075/browseDirectory.jsp?dir=%2Fuser%2Fhadoop%2Ftutorial%2Faccounts2-parts&namenodeInfoPort=50070>
dir
2012-08-17 15:34
rwxr-xr-x
hadoop
supergroup
persons<http://slave1:50075/browseDirectory.jsp?dir=%2Fuser%2Fhadoop%2Ftutorial%2Fpersons&namenodeInfoPort=50070>
file
124.38 MB
1
64 MB
2012-07-11 13:19
rw-r--r--
hadoop
supergroup
persons-parts<http://slave1:50075/browseDirectory.jsp?dir=%2Fuser%2Fhadoop%2Ftutorial%2Fpersons-parts&namenodeInfoPort=50070>
dir
2012-12-12 10:35
rwxr-xr-x
hadoop
supergroup
persons2<http://slave1:50075/browseDirectory.jsp?dir=%2Fuser%2Fhadoop%2Ftutorial%2Fpersons2&namenodeInfoPort=50070>
file
124.38 MB
1
64 MB
2012-07-20 13:53
rw-r--r--
hadoop
supergroup
persons3<http://slave1:50075/browseDirectory.jsp?dir=%2Fuser%2Fhadoop%2Ftutorial%2Fpersons3&namenodeInfoPort=50070>
file
124.38 MB
1
64 MB
2012-12-12 10:53
rw-r--r--
hadoop
supergroup
short-accounts<http://slave1:50075/browseDirectory.jsp?dir=%2Fuser%2Fhadoop%2Ftutorial%2Fshort-accounts&namenodeInfoPort=50070>
file
59.88 MB
1
64 MB
2012-12-06 12:26
rw-r--r--
hadoop
supergroup
short-accounts-parts<http://slave1:50075/browseDirectory.jsp?dir=%2Fuser%2Fhadoop%2Ftutorial%2Fshort-accounts-parts&namenodeInfoPort=50070>
dir
2012-12-06 15:00
rwxr-xr-x
hadoop
supergroup
Can anybody help make sense of the summary numbers?
Thanks, Rodrigo.
-----------------------------------------
The information contained in this e-mail message is intended only
for the personal and confidential use of the recipient(s) named
above. This message may be an attorney-client communication and/or
work product and as such is privileged and confidential. If the
reader of this message is not the intended recipient or an agent
responsible for delivering it to the intended recipient, you are
hereby notified that you have received this document in error and
that any review, dissemination, distribution, or copying of this
message is strictly prohibited. If you have received this
communication in error, please notify us immediately by e-mail, and
delete the original message.
Re: WEBHDFS API GETCONTENTSUMMARY issue
Posted by Arpit Gupta <ar...@hortonworks.com>.
forgot to mention that the path you are using in the api will also count towards the directory count.
--
Arpit Gupta
Hortonworks Inc.
http://hortonworks.com/
On Jan 10, 2013, at 3:55 PM, Arpit Gupta <ar...@hortonworks.com> wrote:
> Rodrigo
>
> GETCONTENTSUMMARY will return the summary of everything under the path you specified, even the subdirectories. So i would suggest take a look in the directories and see what content they have and then the numbers should add up.
>
> --
> Arpit Gupta
> Hortonworks Inc.
> http://hortonworks.com/
>
> On Jan 10, 2013, at 6:59 AM, "Pastrana, Rodrigo (RIS-BCT)" <Ro...@lexisnexis.com> wrote:
>
>> I’m using WEBHDFS to query directory/file information, but the GETCONTENTSUMMARY counts aren’t returning expected counts.
>>
>> For example, when I query content summary for the directory /user/hadoop/tutorial, webhdfs returns the following:
>>
>> x.y.z.w:50070/webhdfs/v1/user/hadoop/tutorial/?op=GETCONTENTSUMMARY
>> {"ContentSummary":{"directoryCount":4,"fileCount":10,"length":3204490622,"quota":-1,"spaceConsumed":3204490622,"spaceQuota":-1}}.
>>
>> But looking at the content of that dir through the web portal, I see 6 files and 3 subdirs:
>>
>> accounts
>> file
>> 812.93 MB
>> 1
>> 64 MB
>> 2012-12-12 10:59
>> rw-r--r--
>> hadoop
>> supergroup
>> accounts2
>> file
>> 812.93 MB
>> 1
>> 64 MB
>> 2012-07-31 16:48
>> rw-r--r--
>> hadoop
>> supergroup
>> accounts2-parts
>> dir
>> 2012-08-17 15:34
>> rwxr-xr-x
>> hadoop
>> supergroup
>> persons
>> file
>> 124.38 MB
>> 1
>> 64 MB
>> 2012-07-11 13:19
>> rw-r--r--
>> hadoop
>> supergroup
>> persons-parts
>> dir
>> 2012-12-12 10:35
>> rwxr-xr-x
>> hadoop
>> supergroup
>> persons2
>> file
>> 124.38 MB
>> 1
>> 64 MB
>> 2012-07-20 13:53
>> rw-r--r--
>> hadoop
>> supergroup
>> persons3
>> file
>> 124.38 MB
>> 1
>> 64 MB
>> 2012-12-12 10:53
>> rw-r--r--
>> hadoop
>> supergroup
>> short-accounts
>> file
>> 59.88 MB
>> 1
>> 64 MB
>> 2012-12-06 12:26
>> rw-r--r--
>> hadoop
>> supergroup
>> short-accounts-parts
>> dir
>> 2012-12-06 15:00
>> rwxr-xr-x
>> hadoop
>> supergroup
>>
>> Can anybody help make sense of the summary numbers?
>>
>>
>> Thanks, Rodrigo.
>>
>>
>>
>> The information contained in this e-mail message is intended only for the personal and confidential use of the recipient(s) named above. This message may be an attorney-client communication and/or work product and as such is privileged and confidential. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail, and delete the original message.
>>
>
Re: WEBHDFS API GETCONTENTSUMMARY issue
Posted by Arpit Gupta <ar...@hortonworks.com>.
forgot to mention that the path you are using in the api will also count towards the directory count.
--
Arpit Gupta
Hortonworks Inc.
http://hortonworks.com/
On Jan 10, 2013, at 3:55 PM, Arpit Gupta <ar...@hortonworks.com> wrote:
> Rodrigo
>
> GETCONTENTSUMMARY will return the summary of everything under the path you specified, even the subdirectories. So i would suggest take a look in the directories and see what content they have and then the numbers should add up.
>
> --
> Arpit Gupta
> Hortonworks Inc.
> http://hortonworks.com/
>
> On Jan 10, 2013, at 6:59 AM, "Pastrana, Rodrigo (RIS-BCT)" <Ro...@lexisnexis.com> wrote:
>
>> I’m using WEBHDFS to query directory/file information, but the GETCONTENTSUMMARY counts aren’t returning expected counts.
>>
>> For example, when I query content summary for the directory /user/hadoop/tutorial, webhdfs returns the following:
>>
>> x.y.z.w:50070/webhdfs/v1/user/hadoop/tutorial/?op=GETCONTENTSUMMARY
>> {"ContentSummary":{"directoryCount":4,"fileCount":10,"length":3204490622,"quota":-1,"spaceConsumed":3204490622,"spaceQuota":-1}}.
>>
>> But looking at the content of that dir through the web portal, I see 6 files and 3 subdirs:
>>
>> accounts
>> file
>> 812.93 MB
>> 1
>> 64 MB
>> 2012-12-12 10:59
>> rw-r--r--
>> hadoop
>> supergroup
>> accounts2
>> file
>> 812.93 MB
>> 1
>> 64 MB
>> 2012-07-31 16:48
>> rw-r--r--
>> hadoop
>> supergroup
>> accounts2-parts
>> dir
>> 2012-08-17 15:34
>> rwxr-xr-x
>> hadoop
>> supergroup
>> persons
>> file
>> 124.38 MB
>> 1
>> 64 MB
>> 2012-07-11 13:19
>> rw-r--r--
>> hadoop
>> supergroup
>> persons-parts
>> dir
>> 2012-12-12 10:35
>> rwxr-xr-x
>> hadoop
>> supergroup
>> persons2
>> file
>> 124.38 MB
>> 1
>> 64 MB
>> 2012-07-20 13:53
>> rw-r--r--
>> hadoop
>> supergroup
>> persons3
>> file
>> 124.38 MB
>> 1
>> 64 MB
>> 2012-12-12 10:53
>> rw-r--r--
>> hadoop
>> supergroup
>> short-accounts
>> file
>> 59.88 MB
>> 1
>> 64 MB
>> 2012-12-06 12:26
>> rw-r--r--
>> hadoop
>> supergroup
>> short-accounts-parts
>> dir
>> 2012-12-06 15:00
>> rwxr-xr-x
>> hadoop
>> supergroup
>>
>> Can anybody help make sense of the summary numbers?
>>
>>
>> Thanks, Rodrigo.
>>
>>
>>
>> The information contained in this e-mail message is intended only for the personal and confidential use of the recipient(s) named above. This message may be an attorney-client communication and/or work product and as such is privileged and confidential. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail, and delete the original message.
>>
>
Re: WEBHDFS API GETCONTENTSUMMARY issue
Posted by Arpit Gupta <ar...@hortonworks.com>.
forgot to mention that the path you are using in the api will also count towards the directory count.
--
Arpit Gupta
Hortonworks Inc.
http://hortonworks.com/
On Jan 10, 2013, at 3:55 PM, Arpit Gupta <ar...@hortonworks.com> wrote:
> Rodrigo
>
> GETCONTENTSUMMARY will return the summary of everything under the path you specified, even the subdirectories. So i would suggest take a look in the directories and see what content they have and then the numbers should add up.
>
> --
> Arpit Gupta
> Hortonworks Inc.
> http://hortonworks.com/
>
> On Jan 10, 2013, at 6:59 AM, "Pastrana, Rodrigo (RIS-BCT)" <Ro...@lexisnexis.com> wrote:
>
>> I’m using WEBHDFS to query directory/file information, but the GETCONTENTSUMMARY counts aren’t returning expected counts.
>>
>> For example, when I query content summary for the directory /user/hadoop/tutorial, webhdfs returns the following:
>>
>> x.y.z.w:50070/webhdfs/v1/user/hadoop/tutorial/?op=GETCONTENTSUMMARY
>> {"ContentSummary":{"directoryCount":4,"fileCount":10,"length":3204490622,"quota":-1,"spaceConsumed":3204490622,"spaceQuota":-1}}.
>>
>> But looking at the content of that dir through the web portal, I see 6 files and 3 subdirs:
>>
>> accounts
>> file
>> 812.93 MB
>> 1
>> 64 MB
>> 2012-12-12 10:59
>> rw-r--r--
>> hadoop
>> supergroup
>> accounts2
>> file
>> 812.93 MB
>> 1
>> 64 MB
>> 2012-07-31 16:48
>> rw-r--r--
>> hadoop
>> supergroup
>> accounts2-parts
>> dir
>> 2012-08-17 15:34
>> rwxr-xr-x
>> hadoop
>> supergroup
>> persons
>> file
>> 124.38 MB
>> 1
>> 64 MB
>> 2012-07-11 13:19
>> rw-r--r--
>> hadoop
>> supergroup
>> persons-parts
>> dir
>> 2012-12-12 10:35
>> rwxr-xr-x
>> hadoop
>> supergroup
>> persons2
>> file
>> 124.38 MB
>> 1
>> 64 MB
>> 2012-07-20 13:53
>> rw-r--r--
>> hadoop
>> supergroup
>> persons3
>> file
>> 124.38 MB
>> 1
>> 64 MB
>> 2012-12-12 10:53
>> rw-r--r--
>> hadoop
>> supergroup
>> short-accounts
>> file
>> 59.88 MB
>> 1
>> 64 MB
>> 2012-12-06 12:26
>> rw-r--r--
>> hadoop
>> supergroup
>> short-accounts-parts
>> dir
>> 2012-12-06 15:00
>> rwxr-xr-x
>> hadoop
>> supergroup
>>
>> Can anybody help make sense of the summary numbers?
>>
>>
>> Thanks, Rodrigo.
>>
>>
>>
>> The information contained in this e-mail message is intended only for the personal and confidential use of the recipient(s) named above. This message may be an attorney-client communication and/or work product and as such is privileged and confidential. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail, and delete the original message.
>>
>
Re: WEBHDFS API GETCONTENTSUMMARY issue
Posted by Arpit Gupta <ar...@hortonworks.com>.
forgot to mention that the path you are using in the api will also count towards the directory count.
--
Arpit Gupta
Hortonworks Inc.
http://hortonworks.com/
On Jan 10, 2013, at 3:55 PM, Arpit Gupta <ar...@hortonworks.com> wrote:
> Rodrigo
>
> GETCONTENTSUMMARY will return the summary of everything under the path you specified, even the subdirectories. So i would suggest take a look in the directories and see what content they have and then the numbers should add up.
>
> --
> Arpit Gupta
> Hortonworks Inc.
> http://hortonworks.com/
>
> On Jan 10, 2013, at 6:59 AM, "Pastrana, Rodrigo (RIS-BCT)" <Ro...@lexisnexis.com> wrote:
>
>> I’m using WEBHDFS to query directory/file information, but the GETCONTENTSUMMARY counts aren’t returning expected counts.
>>
>> For example, when I query content summary for the directory /user/hadoop/tutorial, webhdfs returns the following:
>>
>> x.y.z.w:50070/webhdfs/v1/user/hadoop/tutorial/?op=GETCONTENTSUMMARY
>> {"ContentSummary":{"directoryCount":4,"fileCount":10,"length":3204490622,"quota":-1,"spaceConsumed":3204490622,"spaceQuota":-1}}.
>>
>> But looking at the content of that dir through the web portal, I see 6 files and 3 subdirs:
>>
>> accounts
>> file
>> 812.93 MB
>> 1
>> 64 MB
>> 2012-12-12 10:59
>> rw-r--r--
>> hadoop
>> supergroup
>> accounts2
>> file
>> 812.93 MB
>> 1
>> 64 MB
>> 2012-07-31 16:48
>> rw-r--r--
>> hadoop
>> supergroup
>> accounts2-parts
>> dir
>> 2012-08-17 15:34
>> rwxr-xr-x
>> hadoop
>> supergroup
>> persons
>> file
>> 124.38 MB
>> 1
>> 64 MB
>> 2012-07-11 13:19
>> rw-r--r--
>> hadoop
>> supergroup
>> persons-parts
>> dir
>> 2012-12-12 10:35
>> rwxr-xr-x
>> hadoop
>> supergroup
>> persons2
>> file
>> 124.38 MB
>> 1
>> 64 MB
>> 2012-07-20 13:53
>> rw-r--r--
>> hadoop
>> supergroup
>> persons3
>> file
>> 124.38 MB
>> 1
>> 64 MB
>> 2012-12-12 10:53
>> rw-r--r--
>> hadoop
>> supergroup
>> short-accounts
>> file
>> 59.88 MB
>> 1
>> 64 MB
>> 2012-12-06 12:26
>> rw-r--r--
>> hadoop
>> supergroup
>> short-accounts-parts
>> dir
>> 2012-12-06 15:00
>> rwxr-xr-x
>> hadoop
>> supergroup
>>
>> Can anybody help make sense of the summary numbers?
>>
>>
>> Thanks, Rodrigo.
>>
>>
>>
>> The information contained in this e-mail message is intended only for the personal and confidential use of the recipient(s) named above. This message may be an attorney-client communication and/or work product and as such is privileged and confidential. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail, and delete the original message.
>>
>
Re: WEBHDFS API GETCONTENTSUMMARY issue
Posted by Arpit Gupta <ar...@hortonworks.com>.
Rodrigo
GETCONTENTSUMMARY will return the summary of everything under the path you specified, even the subdirectories. So i would suggest take a look in the directories and see what content they have and then the numbers should add up.
--
Arpit Gupta
Hortonworks Inc.
http://hortonworks.com/
On Jan 10, 2013, at 6:59 AM, "Pastrana, Rodrigo (RIS-BCT)" <Ro...@lexisnexis.com> wrote:
> I’m using WEBHDFS to query directory/file information, but the GETCONTENTSUMMARY counts aren’t returning expected counts.
>
> For example, when I query content summary for the directory /user/hadoop/tutorial, webhdfs returns the following:
>
> x.y.z.w:50070/webhdfs/v1/user/hadoop/tutorial/?op=GETCONTENTSUMMARY
> {"ContentSummary":{"directoryCount":4,"fileCount":10,"length":3204490622,"quota":-1,"spaceConsumed":3204490622,"spaceQuota":-1}}.
>
> But looking at the content of that dir through the web portal, I see 6 files and 3 subdirs:
>
> accounts
> file
> 812.93 MB
> 1
> 64 MB
> 2012-12-12 10:59
> rw-r--r--
> hadoop
> supergroup
> accounts2
> file
> 812.93 MB
> 1
> 64 MB
> 2012-07-31 16:48
> rw-r--r--
> hadoop
> supergroup
> accounts2-parts
> dir
> 2012-08-17 15:34
> rwxr-xr-x
> hadoop
> supergroup
> persons
> file
> 124.38 MB
> 1
> 64 MB
> 2012-07-11 13:19
> rw-r--r--
> hadoop
> supergroup
> persons-parts
> dir
> 2012-12-12 10:35
> rwxr-xr-x
> hadoop
> supergroup
> persons2
> file
> 124.38 MB
> 1
> 64 MB
> 2012-07-20 13:53
> rw-r--r--
> hadoop
> supergroup
> persons3
> file
> 124.38 MB
> 1
> 64 MB
> 2012-12-12 10:53
> rw-r--r--
> hadoop
> supergroup
> short-accounts
> file
> 59.88 MB
> 1
> 64 MB
> 2012-12-06 12:26
> rw-r--r--
> hadoop
> supergroup
> short-accounts-parts
> dir
> 2012-12-06 15:00
> rwxr-xr-x
> hadoop
> supergroup
>
> Can anybody help make sense of the summary numbers?
>
>
> Thanks, Rodrigo.
>
>
>
> The information contained in this e-mail message is intended only for the personal and confidential use of the recipient(s) named above. This message may be an attorney-client communication and/or work product and as such is privileged and confidential. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail, and delete the original message.
>
Re: WEBHDFS API GETCONTENTSUMMARY issue
Posted by Arpit Gupta <ar...@hortonworks.com>.
Rodrigo
GETCONTENTSUMMARY will return the summary of everything under the path you specified, even the subdirectories. So i would suggest take a look in the directories and see what content they have and then the numbers should add up.
--
Arpit Gupta
Hortonworks Inc.
http://hortonworks.com/
On Jan 10, 2013, at 6:59 AM, "Pastrana, Rodrigo (RIS-BCT)" <Ro...@lexisnexis.com> wrote:
> I’m using WEBHDFS to query directory/file information, but the GETCONTENTSUMMARY counts aren’t returning expected counts.
>
> For example, when I query content summary for the directory /user/hadoop/tutorial, webhdfs returns the following:
>
> x.y.z.w:50070/webhdfs/v1/user/hadoop/tutorial/?op=GETCONTENTSUMMARY
> {"ContentSummary":{"directoryCount":4,"fileCount":10,"length":3204490622,"quota":-1,"spaceConsumed":3204490622,"spaceQuota":-1}}.
>
> But looking at the content of that dir through the web portal, I see 6 files and 3 subdirs:
>
> accounts
> file
> 812.93 MB
> 1
> 64 MB
> 2012-12-12 10:59
> rw-r--r--
> hadoop
> supergroup
> accounts2
> file
> 812.93 MB
> 1
> 64 MB
> 2012-07-31 16:48
> rw-r--r--
> hadoop
> supergroup
> accounts2-parts
> dir
> 2012-08-17 15:34
> rwxr-xr-x
> hadoop
> supergroup
> persons
> file
> 124.38 MB
> 1
> 64 MB
> 2012-07-11 13:19
> rw-r--r--
> hadoop
> supergroup
> persons-parts
> dir
> 2012-12-12 10:35
> rwxr-xr-x
> hadoop
> supergroup
> persons2
> file
> 124.38 MB
> 1
> 64 MB
> 2012-07-20 13:53
> rw-r--r--
> hadoop
> supergroup
> persons3
> file
> 124.38 MB
> 1
> 64 MB
> 2012-12-12 10:53
> rw-r--r--
> hadoop
> supergroup
> short-accounts
> file
> 59.88 MB
> 1
> 64 MB
> 2012-12-06 12:26
> rw-r--r--
> hadoop
> supergroup
> short-accounts-parts
> dir
> 2012-12-06 15:00
> rwxr-xr-x
> hadoop
> supergroup
>
> Can anybody help make sense of the summary numbers?
>
>
> Thanks, Rodrigo.
>
>
>
> The information contained in this e-mail message is intended only for the personal and confidential use of the recipient(s) named above. This message may be an attorney-client communication and/or work product and as such is privileged and confidential. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail, and delete the original message.
>
Re: WEBHDFS API GETCONTENTSUMMARY issue
Posted by Arpit Gupta <ar...@hortonworks.com>.
Rodrigo
GETCONTENTSUMMARY will return the summary of everything under the path you specified, even the subdirectories. So i would suggest take a look in the directories and see what content they have and then the numbers should add up.
--
Arpit Gupta
Hortonworks Inc.
http://hortonworks.com/
On Jan 10, 2013, at 6:59 AM, "Pastrana, Rodrigo (RIS-BCT)" <Ro...@lexisnexis.com> wrote:
> I’m using WEBHDFS to query directory/file information, but the GETCONTENTSUMMARY counts aren’t returning expected counts.
>
> For example, when I query content summary for the directory /user/hadoop/tutorial, webhdfs returns the following:
>
> x.y.z.w:50070/webhdfs/v1/user/hadoop/tutorial/?op=GETCONTENTSUMMARY
> {"ContentSummary":{"directoryCount":4,"fileCount":10,"length":3204490622,"quota":-1,"spaceConsumed":3204490622,"spaceQuota":-1}}.
>
> But looking at the content of that dir through the web portal, I see 6 files and 3 subdirs:
>
> accounts
> file
> 812.93 MB
> 1
> 64 MB
> 2012-12-12 10:59
> rw-r--r--
> hadoop
> supergroup
> accounts2
> file
> 812.93 MB
> 1
> 64 MB
> 2012-07-31 16:48
> rw-r--r--
> hadoop
> supergroup
> accounts2-parts
> dir
> 2012-08-17 15:34
> rwxr-xr-x
> hadoop
> supergroup
> persons
> file
> 124.38 MB
> 1
> 64 MB
> 2012-07-11 13:19
> rw-r--r--
> hadoop
> supergroup
> persons-parts
> dir
> 2012-12-12 10:35
> rwxr-xr-x
> hadoop
> supergroup
> persons2
> file
> 124.38 MB
> 1
> 64 MB
> 2012-07-20 13:53
> rw-r--r--
> hadoop
> supergroup
> persons3
> file
> 124.38 MB
> 1
> 64 MB
> 2012-12-12 10:53
> rw-r--r--
> hadoop
> supergroup
> short-accounts
> file
> 59.88 MB
> 1
> 64 MB
> 2012-12-06 12:26
> rw-r--r--
> hadoop
> supergroup
> short-accounts-parts
> dir
> 2012-12-06 15:00
> rwxr-xr-x
> hadoop
> supergroup
>
> Can anybody help make sense of the summary numbers?
>
>
> Thanks, Rodrigo.
>
>
>
> The information contained in this e-mail message is intended only for the personal and confidential use of the recipient(s) named above. This message may be an attorney-client communication and/or work product and as such is privileged and confidential. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail, and delete the original message.
>
Re: WEBHDFS API GETCONTENTSUMMARY issue
Posted by Arpit Gupta <ar...@hortonworks.com>.
Rodrigo
GETCONTENTSUMMARY will return the summary of everything under the path you specified, even the subdirectories. So i would suggest take a look in the directories and see what content they have and then the numbers should add up.
--
Arpit Gupta
Hortonworks Inc.
http://hortonworks.com/
On Jan 10, 2013, at 6:59 AM, "Pastrana, Rodrigo (RIS-BCT)" <Ro...@lexisnexis.com> wrote:
> I’m using WEBHDFS to query directory/file information, but the GETCONTENTSUMMARY counts aren’t returning expected counts.
>
> For example, when I query content summary for the directory /user/hadoop/tutorial, webhdfs returns the following:
>
> x.y.z.w:50070/webhdfs/v1/user/hadoop/tutorial/?op=GETCONTENTSUMMARY
> {"ContentSummary":{"directoryCount":4,"fileCount":10,"length":3204490622,"quota":-1,"spaceConsumed":3204490622,"spaceQuota":-1}}.
>
> But looking at the content of that dir through the web portal, I see 6 files and 3 subdirs:
>
> accounts
> file
> 812.93 MB
> 1
> 64 MB
> 2012-12-12 10:59
> rw-r--r--
> hadoop
> supergroup
> accounts2
> file
> 812.93 MB
> 1
> 64 MB
> 2012-07-31 16:48
> rw-r--r--
> hadoop
> supergroup
> accounts2-parts
> dir
> 2012-08-17 15:34
> rwxr-xr-x
> hadoop
> supergroup
> persons
> file
> 124.38 MB
> 1
> 64 MB
> 2012-07-11 13:19
> rw-r--r--
> hadoop
> supergroup
> persons-parts
> dir
> 2012-12-12 10:35
> rwxr-xr-x
> hadoop
> supergroup
> persons2
> file
> 124.38 MB
> 1
> 64 MB
> 2012-07-20 13:53
> rw-r--r--
> hadoop
> supergroup
> persons3
> file
> 124.38 MB
> 1
> 64 MB
> 2012-12-12 10:53
> rw-r--r--
> hadoop
> supergroup
> short-accounts
> file
> 59.88 MB
> 1
> 64 MB
> 2012-12-06 12:26
> rw-r--r--
> hadoop
> supergroup
> short-accounts-parts
> dir
> 2012-12-06 15:00
> rwxr-xr-x
> hadoop
> supergroup
>
> Can anybody help make sense of the summary numbers?
>
>
> Thanks, Rodrigo.
>
>
>
> The information contained in this e-mail message is intended only for the personal and confidential use of the recipient(s) named above. This message may be an attorney-client communication and/or work product and as such is privileged and confidential. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail, and delete the original message.
>