You are viewing a plain text version of this content. The canonical link for it is here.

Posted to hdfs-user@hadoop.apache.org by manoj <ma...@gmail.com> on 2015/10/01 23:54:18 UTC

Specify start and end range for Tasks Api

Hello All,


I'm using the following API to get all the tasks relating to a job. The
problem is, if the number of tasks are in say 1million range then the data
returned by this api is too huge to store in memory before I process the
data and save it.

I was wondering if I can specify a range say: start and end of taskID and
get only 1K task at a time.
Basically I want to chop down the day instead of getting all at once.

http://<history server http
address:port>/ws/v1/history/mapreduce/jobs/{jobid}/tasks


Thanks,
--Manoj Kumar M

Re: Specify start and end range for Tasks Api

Posted by manoj <ma...@gmail.com>.

Any suggestions on the following query?

Thanks,
Manoj
On Oct 1, 2015 2:54 PM, "manoj" <ma...@gmail.com> wrote:

> Hello All,
>
>
> I'm using the following API to get all the tasks relating to a job. The
> problem is, if the number of tasks are in say 1million range then the data
> returned by this api is too huge to store in memory before I process the
> data and save it.
>
> I was wondering if I can specify a range say: start and end of taskID and
> get only 1K task at a time.
> Basically I want to chop down the day instead of getting all at once.
>
> http://<history server http address:port>/ws/v1/history/mapreduce/jobs/{jobid}/tasks
>
>
> Thanks,
> --Manoj Kumar M
>

Re: Specify start and end range for Tasks Api

Posted by manoj <ma...@gmail.com>.

Any suggestions on the following query?

Thanks,
Manoj
On Oct 1, 2015 2:54 PM, "manoj" <ma...@gmail.com> wrote:

> Hello All,
>
>
> I'm using the following API to get all the tasks relating to a job. The
> problem is, if the number of tasks are in say 1million range then the data
> returned by this api is too huge to store in memory before I process the
> data and save it.
>
> I was wondering if I can specify a range say: start and end of taskID and
> get only 1K task at a time.
> Basically I want to chop down the day instead of getting all at once.
>
> http://<history server http address:port>/ws/v1/history/mapreduce/jobs/{jobid}/tasks
>
>
> Thanks,
> --Manoj Kumar M
>

Re: Specify start and end range for Tasks Api

Posted by manoj <ma...@gmail.com>.

Any suggestions on the following query?

Thanks,
Manoj
On Oct 1, 2015 2:54 PM, "manoj" <ma...@gmail.com> wrote:

> Hello All,
>
>
> I'm using the following API to get all the tasks relating to a job. The
> problem is, if the number of tasks are in say 1million range then the data
> returned by this api is too huge to store in memory before I process the
> data and save it.
>
> I was wondering if I can specify a range say: start and end of taskID and
> get only 1K task at a time.
> Basically I want to chop down the day instead of getting all at once.
>
> http://<history server http address:port>/ws/v1/history/mapreduce/jobs/{jobid}/tasks
>
>
> Thanks,
> --Manoj Kumar M
>

Re: Specify start and end range for Tasks Api

Posted by manoj <ma...@gmail.com>.

Any suggestions on the following query?

Thanks,
Manoj
On Oct 1, 2015 2:54 PM, "manoj" <ma...@gmail.com> wrote:

> Hello All,
>
>
> I'm using the following API to get all the tasks relating to a job. The
> problem is, if the number of tasks are in say 1million range then the data
> returned by this api is too huge to store in memory before I process the
> data and save it.
>
> I was wondering if I can specify a range say: start and end of taskID and
> get only 1K task at a time.
> Basically I want to chop down the day instead of getting all at once.
>
> http://<history server http address:port>/ws/v1/history/mapreduce/jobs/{jobid}/tasks
>
>
> Thanks,
> --Manoj Kumar M
>