You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hadoop.apache.org by "tesmai4@gmail.com" <te...@gmail.com> on 2015/02/21 09:42:56 UTC
Time taken by -copyFromLocalHost for transferring data
Hi,
How can I measure the time taken by -copyFromLocalHost for transferring my
data from local host to HDFS?
Regards,
Tariq
Re: Time taken by -copyFromLocalHost for transferring data
Posted by Ranadip Chatterjee <ra...@gmail.com>.
$> time hadoop fs -put <local file> <hdfs path>
For small files, I would expect the time to have a significant variance
between runs. For larger files, it should be more consistent (since the
throughput will be bound by the network bandwidth of the local machine).
On 21 Feb 2015 08:43, "tesmai4@gmail.com" <te...@gmail.com> wrote:
> Hi,
>
> How can I measure the time taken by -copyFromLocalHost for transferring my
> data from local host to HDFS?
>
> Regards,
> Tariq
>
Re: Time taken by -copyFromLocalHost for transferring data
Posted by Ranadip Chatterjee <ra...@gmail.com>.
$> time hadoop fs -put <local file> <hdfs path>
For small files, I would expect the time to have a significant variance
between runs. For larger files, it should be more consistent (since the
throughput will be bound by the network bandwidth of the local machine).
On 21 Feb 2015 08:43, "tesmai4@gmail.com" <te...@gmail.com> wrote:
> Hi,
>
> How can I measure the time taken by -copyFromLocalHost for transferring my
> data from local host to HDFS?
>
> Regards,
> Tariq
>
Re: Time taken by -copyFromLocalHost for transferring data
Posted by Ranadip Chatterjee <ra...@gmail.com>.
$> time hadoop fs -put <local file> <hdfs path>
For small files, I would expect the time to have a significant variance
between runs. For larger files, it should be more consistent (since the
throughput will be bound by the network bandwidth of the local machine).
On 21 Feb 2015 08:43, "tesmai4@gmail.com" <te...@gmail.com> wrote:
> Hi,
>
> How can I measure the time taken by -copyFromLocalHost for transferring my
> data from local host to HDFS?
>
> Regards,
> Tariq
>
Re: Time taken by -copyFromLocalHost for transferring data
Posted by Ranadip Chatterjee <ra...@gmail.com>.
$> time hadoop fs -put <local file> <hdfs path>
For small files, I would expect the time to have a significant variance
between runs. For larger files, it should be more consistent (since the
throughput will be bound by the network bandwidth of the local machine).
On 21 Feb 2015 08:43, "tesmai4@gmail.com" <te...@gmail.com> wrote:
> Hi,
>
> How can I measure the time taken by -copyFromLocalHost for transferring my
> data from local host to HDFS?
>
> Regards,
> Tariq
>