image processing on a low level using PIG...Possible?

[Ifeanyichukwu Osuji <os...@potsdam.edu>]

     Hi all,

          I was wondering if it would be possible to process images on a
low level using PIG. I want to be able to write a pig script
that can differentiate between two images.

Re: image processing on a low level using PIG...Possible?

[Russell Jurney <ru...@gmail.com>]

There was a slide desk sitting around google that had a 'facial recognition'
UDF a while back.

On Mon, Jul 26, 2010 at 7:32 PM, Mridul Muralidharan
<mr...@yahoo-inc.com>wrote:

>
> Hi,
>
>  We have a few projects which do this on hadoop, but I dont see any reason
> why it cant have been done in pig.
> As Alan and Ashutosh mentioned, the image itself will be just bytearray
> (and so you need your own loader, or in our case use a sequence file loader)
> : but you can extract and populate meta-data about the image through udf
> primitives which can then be used in the pig workflow to control how it is
> processed in a scaleout fashion on top of hadoop.
>
>
> Regards,
> Mridul
>
>
> On Tuesday 27 July 2010 12:26 AM, Ifeanyichukwu Osuji wrote:
>
>>
>>
>>      Hi all,
>>
>>           I was wondering if it would be possible to process images on a
>> low level using PIG. I want to be able to write a pig script
>> that can differentiate between two images.
>>
>>
>

Re: image processing on a low level using PIG...Possible?

[Mridul Muralidharan <mr...@yahoo-inc.com>]

Hi,

   We have a few projects which do this on hadoop, but I dont see any 
reason why it cant have been done in pig.
As Alan and Ashutosh mentioned, the image itself will be just bytearray 
(and so you need your own loader, or in our case use a sequence file 
loader) : but you can extract and populate meta-data about the image 
through udf primitives which can then be used in the pig workflow to 
control how it is processed in a scaleout fashion on top of hadoop.


Regards,
Mridul

On Tuesday 27 July 2010 12:26 AM, Ifeanyichukwu Osuji wrote:
>
>
>       Hi all,
>
>            I was wondering if it would be possible to process images on a
> low level using PIG. I want to be able to write a pig script
> that can differentiate between two images.
>

Re: image processing on a low level using PIG...Possible?

[Ashutosh Chauhan <as...@gmail.com>]

To add in bit more details: You basically load images as bytearray
type. Pig wont interpret it. Then you can write a UDF which takes
these bytearrays as input. You can choose to work directly on
bytearray or create image objects out of them, do your processing and
return bytearray back to Pig. You can further do other sql-y
operations on your images (like filter, group etc) using Pig
primitives which will let you leverage Hadoop's strength.
So, in short answer to your question is Yes.

Ashutosh
On Mon, Jul 26, 2010 at 13:27, Alan Gates <ga...@yahoo-inc.com> wrote:
> Pig itself does not contain image processing primitives.  But if you write
> your image processing in a UDF, then Pig can be a great framework for
> dealing with the parallelism, running it on Hadoop, etc.
>
> Alan.
>
> On Jul 26, 2010, at 11:56 AM, Ifeanyichukwu Osuji wrote:
>
>>
>>
>>    Hi all,
>>
>>         I was wondering if it would be possible to process images on a
>> low level using PIG. I want to be able to write a pig script
>> that can differentiate between two images.
>>
>
>

Re: image processing on a low level using PIG...Possible?

[Alan Gates <ga...@yahoo-inc.com>]

Pig itself does not contain image processing primitives.  But if you  
write your image processing in a UDF, then Pig can be a great  
framework for dealing with the parallelism, running it on Hadoop, etc.

Alan.

On Jul 26, 2010, at 11:56 AM, Ifeanyichukwu Osuji wrote:

>
>
>     Hi all,
>
>          I was wondering if it would be possible to process images  
> on a
> low level using PIG. I want to be able to write a pig script
> that can differentiate between two images.
>