Re: Hivemall on Pig,

[Makoto Yui <my...@apache.org>]

Hi Sungwoo,

Honestly, I'm not sure about industrial usage of Apache Pig
but you can find PowerBy Apache Pig list on
https://cwiki.apache.org/confluence/display/PIG/PoweredBy

I think user base of Pig is much smaller (1/5 or less) than Hive
 but certain big companies are still using Pig.

BTW, could you ask Hivemall related questions on
user@hivemall.incubator.apache.org if possible?

http://hivemall.incubator.apache.org/mail-lists.html

It's important in our incubation process.

Warm regards,
Makoto


2016-11-25 23:33 GMT+09:00 Sungwoo Park <gl...@gmail.com>:
> Dear Makoto Yui Sang,
>
> I saw your presentation slides on Hivemall, and I though if you could answer
> small question that I have on Hivemall.
>
> My team (at a university called POSTECH in Korea) are developing a new
> computing engine for the Hadoop ecosystem. It is similar to Tez, but written
> in Scala and offers a few key improvements over Tez. As an application of
> the computing engine, we are also developing a backend for Hive. I am also
> considering a backend for Pig as another application of our computing
> engine, but I am not sure if it is a good decision to commit my team to
> developing a backend for Pig because my team do not have many developers.
>
> I would appreciate it greatly if you could give some feedback on how popular
> Pig is in industrial settings. I know that Hive is an important part of the
> Hadoop ecosystem, so I am sure that we are not wasting our time by
> developing a backend for Hive. However, I am not sure about Pig. For
> example, I wonder if you could share wit me if your company has a lot of
> demand on Hivemall running on Pig.
>
> Sincerely,
>
> --- Sungwoo Park

Re: Hivemall on Pig,

[Reynold Xin <rx...@databricks.com>]

Pig usage is definitely declining, and very rarely do you see new use cases
built on Pig.

IMO, it's primary benefit initially was to support a more modular language
(than SQL) with intermediate variables as well as better UDFs support, and
as a result was good for ETL. However, newer, more language-integrated
programming models (such as the ones provided by Spark) is basically a
strict subset of Spark.


On Mon, Nov 28, 2016 at 12:27 AM, Makoto Yui <my...@apache.org> wrote:

> Hi Sungwoo,
>
> Honestly, I'm not sure about industrial usage of Apache Pig
> but you can find PowerBy Apache Pig list on
> https://cwiki.apache.org/confluence/display/PIG/PoweredBy
>
> I think user base of Pig is much smaller (1/5 or less) than Hive
>  but certain big companies are still using Pig.
>
> BTW, could you ask Hivemall related questions on
> user@hivemall.incubator.apache.org if possible?
>
> http://hivemall.incubator.apache.org/mail-lists.html
>
> It's important in our incubation process.
>
> Warm regards,
> Makoto
>
>
> 2016-11-25 23:33 GMT+09:00 Sungwoo Park <gl...@gmail.com>:
> > Dear Makoto Yui Sang,
> >
> > I saw your presentation slides on Hivemall, and I though if you could
> answer
> > small question that I have on Hivemall.
> >
> > My team (at a university called POSTECH in Korea) are developing a new
> > computing engine for the Hadoop ecosystem. It is similar to Tez, but
> written
> > in Scala and offers a few key improvements over Tez. As an application of
> > the computing engine, we are also developing a backend for Hive. I am
> also
> > considering a backend for Pig as another application of our computing
> > engine, but I am not sure if it is a good decision to commit my team to
> > developing a backend for Pig because my team do not have many developers.
> >
> > I would appreciate it greatly if you could give some feedback on how
> popular
> > Pig is in industrial settings. I know that Hive is an important part of
> the
> > Hadoop ecosystem, so I am sure that we are not wasting our time by
> > developing a backend for Hive. However, I am not sure about Pig. For
> > example, I wonder if you could share wit me if your company has a lot of
> > demand on Hivemall running on Pig.
> >
> > Sincerely,
> >
> > --- Sungwoo Park
>