You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@pig.apache.org by Rohan Rai <ro...@inmobi.com> on 2010/03/04 18:14:17 UTC
Embedded Pig + MQO + Logical Plan
On using embedded Pig Server and registering a pig script for execution
1) Does Multi Query Optimization happens automatically, or has to
explicitly told so.
2) Logical Plan. What one can infer out of it.
3) Does the Block Size (defined in hadoop) has an effect on performance
or the number of map job getting selected.
Regards
Rohan
The information contained in this communication is intended solely for the use of the individual or entity to whom it is addressed and others authorized to receive it. It may contain confidential or legally privileged information. If you are not the intended recipient you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by responding to this email and then delete it from your system. The firm is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt.
Re: Embedded Pig + MQO + Logical Plan
Posted by Rohan Rai <ro...@inmobi.com>.
In addendum
How can I play with Logical Plan?
Rohan Rai wrote:
> On using embedded Pig Server and registering a pig script for execution
>
> 1) Does Multi Query Optimization happens automatically, or has to
> explicitly told so.
>
> 2) Logical Plan. What one can infer out of it.
>
> 3) Does the Block Size (defined in hadoop) has an effect on performance
> or the number of map job getting selected.
>
> Regards
> Rohan
>
> The information contained in this communication is intended solely for the use of the individual or entity to whom it is addressed and others authorized to receive it. It may contain confidential or legally privileged information. If you are not the intended recipient you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by responding to this email and then delete it from your system. The firm is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt.
>
>
The information contained in this communication is intended solely for the use of the individual or entity to whom it is addressed and others authorized to receive it. It may contain confidential or legally privileged information. If you are not the intended recipient you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by responding to this email and then delete it from your system. The firm is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt.
Re: Embedded Pig + MQO + Logical Plan
Posted by Rohan Rai <ro...@inmobi.com>.
Just an after thought
Shouldn't false be given as a parameter by default.
Should this be corrected ?
Regards
Rohan
Rohan Rai wrote:
> Actually I think the evaluation is correct
> Overriding the method to pass FALSE as parameter, enhances the experiences
>
> Secondly as in
> org.apache.pig.Main
> grunt.exec(); is called
> which in turn calls
> parser.parseStopOnError();
> which calls
> parseStopOnError(false);
>
> Regards
> Rohan
>
> Dmitriy Ryaboy wrote:
>
>> This stuff is a bit convoluted, isn't it?
>> I think you may be right (I never use registerScript). Try an experiment?
>>
>> On Thu, Mar 4, 2010 at 11:20 AM, Rohan Rai <ro...@inmobi.com> wrote:
>>
>>
>>
>>> In addition
>>>
>>> Even
>>> org.apache.pig.tools.pigscript.parser.PigScriptParser (.jj)
>>> seems to tell that its not running in batch mode .
>>>
>>> Is the interpretation incorrect
>>>
>>> Regards
>>> Rohan
>>>
>>>
>>> Rohan Rai wrote:
>>>
>>>
>>>
>>>> Thats what makes it confusing
>>>> If you see the the parameter getting passed is true which is sameBatch
>>>> on which it should ideally not call setBatchOn
>>>>
>>>> if (!mInteractive && !sameBatch) {
>>>> setBatchOn();
>>>> }
>>>>
>>>>
>>>>
>>>> Dmitriy Ryaboy wrote:
>>>>
>>>> Looks like it's on automatically.
>>>>
>>>>
>>>>> Code below is from trunk, but I don't think this changed recently. I got
>>>>> rid
>>>>> of exception handling for conciseness.
>>>>>
>>>>> In PigServer:
>>>>>
>>>>> public void registerScript(String fileName) throws IOException {
>>>>> GruntParser grunt = new GruntParser(new FileReader(new
>>>>> File(fileName)));
>>>>> * grunt.setInteractive(false);*
>>>>> grunt.setParams(this);
>>>>> grunt.parseStopOnError(true);
>>>>> }
>>>>>
>>>>>
>>>>> In GruntParser:
>>>>>
>>>>> public int[] parseStopOnError(boolean sameBatch) throws IOException,
>>>>> ParseException {
>>>>> * if (!mInteractive && !sameBatch) {
>>>>> setBatchOn();
>>>>> }
>>>>> * prompt();
>>>>> mDone = false;
>>>>> while(!mDone) {
>>>>> parse();
>>>>> }
>>>>> if (!sameBatch) {
>>>>> executeBatch();
>>>>> }
>>>>> int [] res = { mNumSucceededJobs, mNumFailedJobs };
>>>>> return res;
>>>>> }
>>>>>
>>>>>
>>>>> On Thu, Mar 4, 2010 at 10:00 AM, Rohan Rai <ro...@inmobi.com> wrote:
>>>>>
>>>>>
>>>>>
>>>>> Thanks Dmitriy
>>>>>
>>>>>
>>>>>> Just a question more
>>>>>>
>>>>>> registerScript allows to register a pig script in the embedded mode
>>>>>> So the confusion was does it internally tries to optimize it.
>>>>>> or setBatchOn has to be explicitly called
>>>>>>
>>>>>> Regards
>>>>>> Rohan
>>>>>>
>>>>>>
>>>>>> Dmitriy Ryaboy wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> 1) Automatically, if you call it right. Look for the setBatchOn and
>>>>>>
>>>>>>
>>>>>>> executeBatch methods (I may be slightly off on the method names, going
>>>>>>> off
>>>>>>> memory)
>>>>>>>
>>>>>>> 2) The optimizer moves stuff around and may be executing things in a
>>>>>>> slightly different order then what you tell it. This can mean pushing
>>>>>>> up
>>>>>>> projections, filters, and limits, inserting casts, and doing all kinds
>>>>>>> of
>>>>>>> other manipulations. The logical plan shows you what's going to happen
>>>>>>> without breaking it down into the MR plan. There are further
>>>>>>> optimizations
>>>>>>> at the MR level, so both are worth checking. In practice I usually look
>>>>>>> at
>>>>>>> the logical plan for order-of-operations and general sanity checking,
>>>>>>> and
>>>>>>> at
>>>>>>> the MR plan for number of jobs and whether things like algebraic and
>>>>>>> accumulative interfaces are kicking in.
>>>>>>>
>>>>>>> 3) Yes. Roughly speaking, one map per block will be generated. The
>>>>>>> bigger
>>>>>>> the block, the more work per mapper. The smaller the block, the more
>>>>>>> mappers. Depending on the workload, there's an optimal value.
>>>>>>>
>>>>>>> 4) Playing with logical plan -- don't :-). It's exposed so that you can
>>>>>>> look
>>>>>>> at what's going on, and not intended to let you change execution plans.
>>>>>>> Unless you actually want to hack Pig guts. If that's the case, look at
>>>>>>> the
>>>>>>> optimizer and the MRCompiler classes to see how it's getting modified
>>>>>>> and
>>>>>>> used.
>>>>>>>
>>>>>>> -D
>>>>>>>
>>>>>>> On Thu, Mar 4, 2010 at 9:14 AM, Rohan Rai <ro...@inmobi.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>> On using embedded Pig Server and registering a pig script for
>>>>>>> execution
>>>>>>>
>>>>>>>
>>>>>>> 1) Does Multi Query Optimization happens automatically, or has to
>>>>>>>
>>>>>>>
>>>>>>>> explicitly told so.
>>>>>>>>
>>>>>>>> 2) Logical Plan. What one can infer out of it.
>>>>>>>>
>>>>>>>> 3) Does the Block Size (defined in hadoop) has an effect on
>>>>>>>> performance
>>>>>>>> or the number of map job getting selected.
>>>>>>>>
>>>>>>>> Regards
>>>>>>>> Rohan
>>>>>>>>
>>>>>>>> The information contained in this communication is intended solely for
>>>>>>>> the
>>>>>>>> use of the individual or entity to whom it is addressed and others
>>>>>>>> authorized to receive it. It may contain confidential or legally
>>>>>>>> privileged
>>>>>>>> information. If you are not the intended recipient you are hereby
>>>>>>>> notified
>>>>>>>> that any disclosure, copying, distribution or taking any action in
>>>>>>>> reliance
>>>>>>>> on the contents of this information is strictly prohibited and may be
>>>>>>>> unlawful. If you have received this communication in error, please
>>>>>>>> notify
>>>>>>>> us
>>>>>>>> immediately by responding to this email and then delete it from your
>>>>>>>> system.
>>>>>>>> The firm is neither liable for the proper and complete transmission of
>>>>>>>> the
>>>>>>>> information contained in this communication nor for any delay in its
>>>>>>>> receipt.
>>>>>>>>
>>>>>>>>
>>>>>>>> .
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> The information contained in this communication is intended solely for
>>>>>>>
>>>>>>>
>>>>>> the
>>>>>> use of the individual or entity to whom it is addressed and others
>>>>>> authorized to receive it. It may contain confidential or legally
>>>>>> privileged
>>>>>> information. If you are not the intended recipient you are hereby
>>>>>> notified
>>>>>> that any disclosure, copying, distribution or taking any action in
>>>>>> reliance
>>>>>> on the contents of this information is strictly prohibited and may be
>>>>>> unlawful. If you have received this communication in error, please
>>>>>> notify us
>>>>>> immediately by responding to this email and then delete it from your
>>>>>> system.
>>>>>> The firm is neither liable for the proper and complete transmission of
>>>>>> the
>>>>>> information contained in this communication nor for any delay in its
>>>>>> receipt.
>>>>>>
>>>>>>
>>>>>>
>>>>>> .
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>> The information contained in this communication is intended solely for the
>>>> use of the individual or entity to whom it is addressed and others
>>>> authorized to receive it. It may contain confidential or legally privileged
>>>> information. If you are not the intended recipient you are hereby notified
>>>> that any disclosure, copying, distribution or taking any action in reliance
>>>> on the contents of this information is strictly prohibited and may be
>>>> unlawful. If you have received this communication in error, please notify us
>>>> immediately by responding to this email and then delete it from your system.
>>>> The firm is neither liable for the proper and complete transmission of the
>>>> information contained in this communication nor for any delay in its
>>>> receipt.
>>>> .
>>>>
>>>>
>>>>
>>>>
>>>>
>>> The information contained in this communication is intended solely for the
>>> use of the individual or entity to whom it is addressed and others
>>> authorized to receive it. It may contain confidential or legally privileged
>>> information. If you are not the intended recipient you are hereby notified
>>> that any disclosure, copying, distribution or taking any action in reliance
>>> on the contents of this information is strictly prohibited and may be
>>> unlawful. If you have received this communication in error, please notify us
>>> immediately by responding to this email and then delete it from your system.
>>> The firm is neither liable for the proper and complete transmission of the
>>> information contained in this communication nor for any delay in its
>>> receipt.
>>>
>>>
>>>
>> .
>>
>>
>>
>
>
> The information contained in this communication is intended solely for the use of the individual or entity to whom it is addressed and others authorized to receive it. It may contain confidential or legally privileged information. If you are not the intended recipient you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by responding to this email and then delete it from your system. The firm is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt.
> .
>
>
The information contained in this communication is intended solely for the use of the individual or entity to whom it is addressed and others authorized to receive it. It may contain confidential or legally privileged information. If you are not the intended recipient you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by responding to this email and then delete it from your system. The firm is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt.
Re: Embedded Pig + MQO + Logical Plan
Posted by Rohan Rai <ro...@inmobi.com>.
Actually I think the evaluation is correct
Overriding the method to pass FALSE as parameter, enhances the experiences
Secondly as in
org.apache.pig.Main
grunt.exec(); is called
which in turn calls
parser.parseStopOnError();
which calls
parseStopOnError(false);
Regards
Rohan
Dmitriy Ryaboy wrote:
> This stuff is a bit convoluted, isn't it?
> I think you may be right (I never use registerScript). Try an experiment?
>
> On Thu, Mar 4, 2010 at 11:20 AM, Rohan Rai <ro...@inmobi.com> wrote:
>
>
>> In addition
>>
>> Even
>> org.apache.pig.tools.pigscript.parser.PigScriptParser (.jj)
>> seems to tell that its not running in batch mode .
>>
>> Is the interpretation incorrect
>>
>> Regards
>> Rohan
>>
>>
>> Rohan Rai wrote:
>>
>>
>>> Thats what makes it confusing
>>> If you see the the parameter getting passed is true which is sameBatch
>>> on which it should ideally not call setBatchOn
>>>
>>> if (!mInteractive && !sameBatch) {
>>> setBatchOn();
>>> }
>>>
>>>
>>>
>>> Dmitriy Ryaboy wrote:
>>>
>>> Looks like it's on automatically.
>>>
>>>> Code below is from trunk, but I don't think this changed recently. I got
>>>> rid
>>>> of exception handling for conciseness.
>>>>
>>>> In PigServer:
>>>>
>>>> public void registerScript(String fileName) throws IOException {
>>>> GruntParser grunt = new GruntParser(new FileReader(new
>>>> File(fileName)));
>>>> * grunt.setInteractive(false);*
>>>> grunt.setParams(this);
>>>> grunt.parseStopOnError(true);
>>>> }
>>>>
>>>>
>>>> In GruntParser:
>>>>
>>>> public int[] parseStopOnError(boolean sameBatch) throws IOException,
>>>> ParseException {
>>>> * if (!mInteractive && !sameBatch) {
>>>> setBatchOn();
>>>> }
>>>> * prompt();
>>>> mDone = false;
>>>> while(!mDone) {
>>>> parse();
>>>> }
>>>> if (!sameBatch) {
>>>> executeBatch();
>>>> }
>>>> int [] res = { mNumSucceededJobs, mNumFailedJobs };
>>>> return res;
>>>> }
>>>>
>>>>
>>>> On Thu, Mar 4, 2010 at 10:00 AM, Rohan Rai <ro...@inmobi.com> wrote:
>>>>
>>>>
>>>>
>>>> Thanks Dmitriy
>>>>
>>>>> Just a question more
>>>>>
>>>>> registerScript allows to register a pig script in the embedded mode
>>>>> So the confusion was does it internally tries to optimize it.
>>>>> or setBatchOn has to be explicitly called
>>>>>
>>>>> Regards
>>>>> Rohan
>>>>>
>>>>>
>>>>> Dmitriy Ryaboy wrote:
>>>>>
>>>>>
>>>>>
>>>>> 1) Automatically, if you call it right. Look for the setBatchOn and
>>>>>
>>>>>> executeBatch methods (I may be slightly off on the method names, going
>>>>>> off
>>>>>> memory)
>>>>>>
>>>>>> 2) The optimizer moves stuff around and may be executing things in a
>>>>>> slightly different order then what you tell it. This can mean pushing
>>>>>> up
>>>>>> projections, filters, and limits, inserting casts, and doing all kinds
>>>>>> of
>>>>>> other manipulations. The logical plan shows you what's going to happen
>>>>>> without breaking it down into the MR plan. There are further
>>>>>> optimizations
>>>>>> at the MR level, so both are worth checking. In practice I usually look
>>>>>> at
>>>>>> the logical plan for order-of-operations and general sanity checking,
>>>>>> and
>>>>>> at
>>>>>> the MR plan for number of jobs and whether things like algebraic and
>>>>>> accumulative interfaces are kicking in.
>>>>>>
>>>>>> 3) Yes. Roughly speaking, one map per block will be generated. The
>>>>>> bigger
>>>>>> the block, the more work per mapper. The smaller the block, the more
>>>>>> mappers. Depending on the workload, there's an optimal value.
>>>>>>
>>>>>> 4) Playing with logical plan -- don't :-). It's exposed so that you can
>>>>>> look
>>>>>> at what's going on, and not intended to let you change execution plans.
>>>>>> Unless you actually want to hack Pig guts. If that's the case, look at
>>>>>> the
>>>>>> optimizer and the MRCompiler classes to see how it's getting modified
>>>>>> and
>>>>>> used.
>>>>>>
>>>>>> -D
>>>>>>
>>>>>> On Thu, Mar 4, 2010 at 9:14 AM, Rohan Rai <ro...@inmobi.com>
>>>>>> wrote:
>>>>>>
>>>>>>
>>>>>> On using embedded Pig Server and registering a pig script for
>>>>>> execution
>>>>>>
>>>>>>
>>>>>> 1) Does Multi Query Optimization happens automatically, or has to
>>>>>>
>>>>>>> explicitly told so.
>>>>>>>
>>>>>>> 2) Logical Plan. What one can infer out of it.
>>>>>>>
>>>>>>> 3) Does the Block Size (defined in hadoop) has an effect on
>>>>>>> performance
>>>>>>> or the number of map job getting selected.
>>>>>>>
>>>>>>> Regards
>>>>>>> Rohan
>>>>>>>
>>>>>>> The information contained in this communication is intended solely for
>>>>>>> the
>>>>>>> use of the individual or entity to whom it is addressed and others
>>>>>>> authorized to receive it. It may contain confidential or legally
>>>>>>> privileged
>>>>>>> information. If you are not the intended recipient you are hereby
>>>>>>> notified
>>>>>>> that any disclosure, copying, distribution or taking any action in
>>>>>>> reliance
>>>>>>> on the contents of this information is strictly prohibited and may be
>>>>>>> unlawful. If you have received this communication in error, please
>>>>>>> notify
>>>>>>> us
>>>>>>> immediately by responding to this email and then delete it from your
>>>>>>> system.
>>>>>>> The firm is neither liable for the proper and complete transmission of
>>>>>>> the
>>>>>>> information contained in this communication nor for any delay in its
>>>>>>> receipt.
>>>>>>>
>>>>>>>
>>>>>>> .
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> The information contained in this communication is intended solely for
>>>>>>
>>>>> the
>>>>> use of the individual or entity to whom it is addressed and others
>>>>> authorized to receive it. It may contain confidential or legally
>>>>> privileged
>>>>> information. If you are not the intended recipient you are hereby
>>>>> notified
>>>>> that any disclosure, copying, distribution or taking any action in
>>>>> reliance
>>>>> on the contents of this information is strictly prohibited and may be
>>>>> unlawful. If you have received this communication in error, please
>>>>> notify us
>>>>> immediately by responding to this email and then delete it from your
>>>>> system.
>>>>> The firm is neither liable for the proper and complete transmission of
>>>>> the
>>>>> information contained in this communication nor for any delay in its
>>>>> receipt.
>>>>>
>>>>>
>>>>>
>>>>> .
>>>>>
>>>>
>>>>
>>>>
>>> The information contained in this communication is intended solely for the
>>> use of the individual or entity to whom it is addressed and others
>>> authorized to receive it. It may contain confidential or legally privileged
>>> information. If you are not the intended recipient you are hereby notified
>>> that any disclosure, copying, distribution or taking any action in reliance
>>> on the contents of this information is strictly prohibited and may be
>>> unlawful. If you have received this communication in error, please notify us
>>> immediately by responding to this email and then delete it from your system.
>>> The firm is neither liable for the proper and complete transmission of the
>>> information contained in this communication nor for any delay in its
>>> receipt.
>>> .
>>>
>>>
>>>
>>>
>> The information contained in this communication is intended solely for the
>> use of the individual or entity to whom it is addressed and others
>> authorized to receive it. It may contain confidential or legally privileged
>> information. If you are not the intended recipient you are hereby notified
>> that any disclosure, copying, distribution or taking any action in reliance
>> on the contents of this information is strictly prohibited and may be
>> unlawful. If you have received this communication in error, please notify us
>> immediately by responding to this email and then delete it from your system.
>> The firm is neither liable for the proper and complete transmission of the
>> information contained in this communication nor for any delay in its
>> receipt.
>>
>>
> .
>
>
The information contained in this communication is intended solely for the use of the individual or entity to whom it is addressed and others authorized to receive it. It may contain confidential or legally privileged information. If you are not the intended recipient you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by responding to this email and then delete it from your system. The firm is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt.
Re: Embedded Pig + MQO + Logical Plan
Posted by Dmitriy Ryaboy <dv...@gmail.com>.
This stuff is a bit convoluted, isn't it?
I think you may be right (I never use registerScript). Try an experiment?
On Thu, Mar 4, 2010 at 11:20 AM, Rohan Rai <ro...@inmobi.com> wrote:
> In addition
>
> Even
> org.apache.pig.tools.pigscript.parser.PigScriptParser (.jj)
> seems to tell that its not running in batch mode .
>
> Is the interpretation incorrect
>
> Regards
> Rohan
>
>
> Rohan Rai wrote:
>
>> Thats what makes it confusing
>> If you see the the parameter getting passed is true which is sameBatch
>> on which it should ideally not call setBatchOn
>>
>> if (!mInteractive && !sameBatch) {
>> setBatchOn();
>> }
>>
>>
>>
>> Dmitriy Ryaboy wrote:
>>
>> Looks like it's on automatically.
>>>
>>> Code below is from trunk, but I don't think this changed recently. I got
>>> rid
>>> of exception handling for conciseness.
>>>
>>> In PigServer:
>>>
>>> public void registerScript(String fileName) throws IOException {
>>> GruntParser grunt = new GruntParser(new FileReader(new
>>> File(fileName)));
>>> * grunt.setInteractive(false);*
>>> grunt.setParams(this);
>>> grunt.parseStopOnError(true);
>>> }
>>>
>>>
>>> In GruntParser:
>>>
>>> public int[] parseStopOnError(boolean sameBatch) throws IOException,
>>> ParseException {
>>> * if (!mInteractive && !sameBatch) {
>>> setBatchOn();
>>> }
>>> * prompt();
>>> mDone = false;
>>> while(!mDone) {
>>> parse();
>>> }
>>> if (!sameBatch) {
>>> executeBatch();
>>> }
>>> int [] res = { mNumSucceededJobs, mNumFailedJobs };
>>> return res;
>>> }
>>>
>>>
>>> On Thu, Mar 4, 2010 at 10:00 AM, Rohan Rai <ro...@inmobi.com> wrote:
>>>
>>>
>>>
>>> Thanks Dmitriy
>>>>
>>>> Just a question more
>>>>
>>>> registerScript allows to register a pig script in the embedded mode
>>>> So the confusion was does it internally tries to optimize it.
>>>> or setBatchOn has to be explicitly called
>>>>
>>>> Regards
>>>> Rohan
>>>>
>>>>
>>>> Dmitriy Ryaboy wrote:
>>>>
>>>>
>>>>
>>>> 1) Automatically, if you call it right. Look for the setBatchOn and
>>>>> executeBatch methods (I may be slightly off on the method names, going
>>>>> off
>>>>> memory)
>>>>>
>>>>> 2) The optimizer moves stuff around and may be executing things in a
>>>>> slightly different order then what you tell it. This can mean pushing
>>>>> up
>>>>> projections, filters, and limits, inserting casts, and doing all kinds
>>>>> of
>>>>> other manipulations. The logical plan shows you what's going to happen
>>>>> without breaking it down into the MR plan. There are further
>>>>> optimizations
>>>>> at the MR level, so both are worth checking. In practice I usually look
>>>>> at
>>>>> the logical plan for order-of-operations and general sanity checking,
>>>>> and
>>>>> at
>>>>> the MR plan for number of jobs and whether things like algebraic and
>>>>> accumulative interfaces are kicking in.
>>>>>
>>>>> 3) Yes. Roughly speaking, one map per block will be generated. The
>>>>> bigger
>>>>> the block, the more work per mapper. The smaller the block, the more
>>>>> mappers. Depending on the workload, there's an optimal value.
>>>>>
>>>>> 4) Playing with logical plan -- don't :-). It's exposed so that you can
>>>>> look
>>>>> at what's going on, and not intended to let you change execution plans.
>>>>> Unless you actually want to hack Pig guts. If that's the case, look at
>>>>> the
>>>>> optimizer and the MRCompiler classes to see how it's getting modified
>>>>> and
>>>>> used.
>>>>>
>>>>> -D
>>>>>
>>>>> On Thu, Mar 4, 2010 at 9:14 AM, Rohan Rai <ro...@inmobi.com>
>>>>> wrote:
>>>>>
>>>>>
>>>>> On using embedded Pig Server and registering a pig script for
>>>>> execution
>>>>>
>>>>>
>>>>> 1) Does Multi Query Optimization happens automatically, or has to
>>>>>> explicitly told so.
>>>>>>
>>>>>> 2) Logical Plan. What one can infer out of it.
>>>>>>
>>>>>> 3) Does the Block Size (defined in hadoop) has an effect on
>>>>>> performance
>>>>>> or the number of map job getting selected.
>>>>>>
>>>>>> Regards
>>>>>> Rohan
>>>>>>
>>>>>> The information contained in this communication is intended solely for
>>>>>> the
>>>>>> use of the individual or entity to whom it is addressed and others
>>>>>> authorized to receive it. It may contain confidential or legally
>>>>>> privileged
>>>>>> information. If you are not the intended recipient you are hereby
>>>>>> notified
>>>>>> that any disclosure, copying, distribution or taking any action in
>>>>>> reliance
>>>>>> on the contents of this information is strictly prohibited and may be
>>>>>> unlawful. If you have received this communication in error, please
>>>>>> notify
>>>>>> us
>>>>>> immediately by responding to this email and then delete it from your
>>>>>> system.
>>>>>> The firm is neither liable for the proper and complete transmission of
>>>>>> the
>>>>>> information contained in this communication nor for any delay in its
>>>>>> receipt.
>>>>>>
>>>>>>
>>>>>> .
>>>>>>
>>>>>>
>>>>>>
>>>>> The information contained in this communication is intended solely for
>>>> the
>>>> use of the individual or entity to whom it is addressed and others
>>>> authorized to receive it. It may contain confidential or legally
>>>> privileged
>>>> information. If you are not the intended recipient you are hereby
>>>> notified
>>>> that any disclosure, copying, distribution or taking any action in
>>>> reliance
>>>> on the contents of this information is strictly prohibited and may be
>>>> unlawful. If you have received this communication in error, please
>>>> notify us
>>>> immediately by responding to this email and then delete it from your
>>>> system.
>>>> The firm is neither liable for the proper and complete transmission of
>>>> the
>>>> information contained in this communication nor for any delay in its
>>>> receipt.
>>>>
>>>>
>>>>
>>>> .
>>>
>>>
>>>
>>>
>>
>> The information contained in this communication is intended solely for the
>> use of the individual or entity to whom it is addressed and others
>> authorized to receive it. It may contain confidential or legally privileged
>> information. If you are not the intended recipient you are hereby notified
>> that any disclosure, copying, distribution or taking any action in reliance
>> on the contents of this information is strictly prohibited and may be
>> unlawful. If you have received this communication in error, please notify us
>> immediately by responding to this email and then delete it from your system.
>> The firm is neither liable for the proper and complete transmission of the
>> information contained in this communication nor for any delay in its
>> receipt.
>> .
>>
>>
>>
>
> The information contained in this communication is intended solely for the
> use of the individual or entity to whom it is addressed and others
> authorized to receive it. It may contain confidential or legally privileged
> information. If you are not the intended recipient you are hereby notified
> that any disclosure, copying, distribution or taking any action in reliance
> on the contents of this information is strictly prohibited and may be
> unlawful. If you have received this communication in error, please notify us
> immediately by responding to this email and then delete it from your system.
> The firm is neither liable for the proper and complete transmission of the
> information contained in this communication nor for any delay in its
> receipt.
>
Re: Embedded Pig + MQO + Logical Plan
Posted by Rohan Rai <ro...@inmobi.com>.
In addition
Even
org.apache.pig.tools.pigscript.parser.PigScriptParser (.jj)
seems to tell that its not running in batch mode .
Is the interpretation incorrect
Regards
Rohan
Rohan Rai wrote:
> Thats what makes it confusing
> If you see the the parameter getting passed is true which is sameBatch
> on which it should ideally not call setBatchOn
>
> if (!mInteractive && !sameBatch) {
> setBatchOn();
> }
>
>
>
> Dmitriy Ryaboy wrote:
>
>> Looks like it's on automatically.
>>
>> Code below is from trunk, but I don't think this changed recently. I got rid
>> of exception handling for conciseness.
>>
>> In PigServer:
>>
>> public void registerScript(String fileName) throws IOException {
>> GruntParser grunt = new GruntParser(new FileReader(new
>> File(fileName)));
>> * grunt.setInteractive(false);*
>> grunt.setParams(this);
>> grunt.parseStopOnError(true);
>> }
>>
>>
>> In GruntParser:
>>
>> public int[] parseStopOnError(boolean sameBatch) throws IOException,
>> ParseException {
>> * if (!mInteractive && !sameBatch) {
>> setBatchOn();
>> }
>> * prompt();
>> mDone = false;
>> while(!mDone) {
>> parse();
>> }
>> if (!sameBatch) {
>> executeBatch();
>> }
>> int [] res = { mNumSucceededJobs, mNumFailedJobs };
>> return res;
>> }
>>
>>
>> On Thu, Mar 4, 2010 at 10:00 AM, Rohan Rai <ro...@inmobi.com> wrote:
>>
>>
>>
>>> Thanks Dmitriy
>>>
>>> Just a question more
>>>
>>> registerScript allows to register a pig script in the embedded mode
>>> So the confusion was does it internally tries to optimize it.
>>> or setBatchOn has to be explicitly called
>>>
>>> Regards
>>> Rohan
>>>
>>>
>>> Dmitriy Ryaboy wrote:
>>>
>>>
>>>
>>>> 1) Automatically, if you call it right. Look for the setBatchOn and
>>>> executeBatch methods (I may be slightly off on the method names, going off
>>>> memory)
>>>>
>>>> 2) The optimizer moves stuff around and may be executing things in a
>>>> slightly different order then what you tell it. This can mean pushing up
>>>> projections, filters, and limits, inserting casts, and doing all kinds of
>>>> other manipulations. The logical plan shows you what's going to happen
>>>> without breaking it down into the MR plan. There are further optimizations
>>>> at the MR level, so both are worth checking. In practice I usually look at
>>>> the logical plan for order-of-operations and general sanity checking, and
>>>> at
>>>> the MR plan for number of jobs and whether things like algebraic and
>>>> accumulative interfaces are kicking in.
>>>>
>>>> 3) Yes. Roughly speaking, one map per block will be generated. The bigger
>>>> the block, the more work per mapper. The smaller the block, the more
>>>> mappers. Depending on the workload, there's an optimal value.
>>>>
>>>> 4) Playing with logical plan -- don't :-). It's exposed so that you can
>>>> look
>>>> at what's going on, and not intended to let you change execution plans.
>>>> Unless you actually want to hack Pig guts. If that's the case, look at the
>>>> optimizer and the MRCompiler classes to see how it's getting modified and
>>>> used.
>>>>
>>>> -D
>>>>
>>>> On Thu, Mar 4, 2010 at 9:14 AM, Rohan Rai <ro...@inmobi.com> wrote:
>>>>
>>>>
>>>> On using embedded Pig Server and registering a pig script for execution
>>>>
>>>>
>>>>> 1) Does Multi Query Optimization happens automatically, or has to
>>>>> explicitly told so.
>>>>>
>>>>> 2) Logical Plan. What one can infer out of it.
>>>>>
>>>>> 3) Does the Block Size (defined in hadoop) has an effect on performance
>>>>> or the number of map job getting selected.
>>>>>
>>>>> Regards
>>>>> Rohan
>>>>>
>>>>> The information contained in this communication is intended solely for
>>>>> the
>>>>> use of the individual or entity to whom it is addressed and others
>>>>> authorized to receive it. It may contain confidential or legally
>>>>> privileged
>>>>> information. If you are not the intended recipient you are hereby
>>>>> notified
>>>>> that any disclosure, copying, distribution or taking any action in
>>>>> reliance
>>>>> on the contents of this information is strictly prohibited and may be
>>>>> unlawful. If you have received this communication in error, please notify
>>>>> us
>>>>> immediately by responding to this email and then delete it from your
>>>>> system.
>>>>> The firm is neither liable for the proper and complete transmission of
>>>>> the
>>>>> information contained in this communication nor for any delay in its
>>>>> receipt.
>>>>>
>>>>>
>>>>> .
>>>>>
>>>>>
>>>>
>>> The information contained in this communication is intended solely for the
>>> use of the individual or entity to whom it is addressed and others
>>> authorized to receive it. It may contain confidential or legally privileged
>>> information. If you are not the intended recipient you are hereby notified
>>> that any disclosure, copying, distribution or taking any action in reliance
>>> on the contents of this information is strictly prohibited and may be
>>> unlawful. If you have received this communication in error, please notify us
>>> immediately by responding to this email and then delete it from your system.
>>> The firm is neither liable for the proper and complete transmission of the
>>> information contained in this communication nor for any delay in its
>>> receipt.
>>>
>>>
>>>
>> .
>>
>>
>>
>
>
> The information contained in this communication is intended solely for the use of the individual or entity to whom it is addressed and others authorized to receive it. It may contain confidential or legally privileged information. If you are not the intended recipient you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by responding to this email and then delete it from your system. The firm is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt.
> .
>
>
The information contained in this communication is intended solely for the use of the individual or entity to whom it is addressed and others authorized to receive it. It may contain confidential or legally privileged information. If you are not the intended recipient you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by responding to this email and then delete it from your system. The firm is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt.
Re: Embedded Pig + MQO + Logical Plan
Posted by Rohan Rai <ro...@inmobi.com>.
Thats what makes it confusing
If you see the the parameter getting passed is true which is sameBatch
on which it should ideally not call setBatchOn
if (!mInteractive && !sameBatch) {
setBatchOn();
}
Dmitriy Ryaboy wrote:
> Looks like it's on automatically.
>
> Code below is from trunk, but I don't think this changed recently. I got rid
> of exception handling for conciseness.
>
> In PigServer:
>
> public void registerScript(String fileName) throws IOException {
> GruntParser grunt = new GruntParser(new FileReader(new
> File(fileName)));
> * grunt.setInteractive(false);*
> grunt.setParams(this);
> grunt.parseStopOnError(true);
> }
>
>
> In GruntParser:
>
> public int[] parseStopOnError(boolean sameBatch) throws IOException,
> ParseException {
> * if (!mInteractive && !sameBatch) {
> setBatchOn();
> }
> * prompt();
> mDone = false;
> while(!mDone) {
> parse();
> }
> if (!sameBatch) {
> executeBatch();
> }
> int [] res = { mNumSucceededJobs, mNumFailedJobs };
> return res;
> }
>
>
> On Thu, Mar 4, 2010 at 10:00 AM, Rohan Rai <ro...@inmobi.com> wrote:
>
>
>> Thanks Dmitriy
>>
>> Just a question more
>>
>> registerScript allows to register a pig script in the embedded mode
>> So the confusion was does it internally tries to optimize it.
>> or setBatchOn has to be explicitly called
>>
>> Regards
>> Rohan
>>
>>
>> Dmitriy Ryaboy wrote:
>>
>>
>>> 1) Automatically, if you call it right. Look for the setBatchOn and
>>> executeBatch methods (I may be slightly off on the method names, going off
>>> memory)
>>>
>>> 2) The optimizer moves stuff around and may be executing things in a
>>> slightly different order then what you tell it. This can mean pushing up
>>> projections, filters, and limits, inserting casts, and doing all kinds of
>>> other manipulations. The logical plan shows you what's going to happen
>>> without breaking it down into the MR plan. There are further optimizations
>>> at the MR level, so both are worth checking. In practice I usually look at
>>> the logical plan for order-of-operations and general sanity checking, and
>>> at
>>> the MR plan for number of jobs and whether things like algebraic and
>>> accumulative interfaces are kicking in.
>>>
>>> 3) Yes. Roughly speaking, one map per block will be generated. The bigger
>>> the block, the more work per mapper. The smaller the block, the more
>>> mappers. Depending on the workload, there's an optimal value.
>>>
>>> 4) Playing with logical plan -- don't :-). It's exposed so that you can
>>> look
>>> at what's going on, and not intended to let you change execution plans.
>>> Unless you actually want to hack Pig guts. If that's the case, look at the
>>> optimizer and the MRCompiler classes to see how it's getting modified and
>>> used.
>>>
>>> -D
>>>
>>> On Thu, Mar 4, 2010 at 9:14 AM, Rohan Rai <ro...@inmobi.com> wrote:
>>>
>>>
>>> On using embedded Pig Server and registering a pig script for execution
>>>
>>>> 1) Does Multi Query Optimization happens automatically, or has to
>>>> explicitly told so.
>>>>
>>>> 2) Logical Plan. What one can infer out of it.
>>>>
>>>> 3) Does the Block Size (defined in hadoop) has an effect on performance
>>>> or the number of map job getting selected.
>>>>
>>>> Regards
>>>> Rohan
>>>>
>>>> The information contained in this communication is intended solely for
>>>> the
>>>> use of the individual or entity to whom it is addressed and others
>>>> authorized to receive it. It may contain confidential or legally
>>>> privileged
>>>> information. If you are not the intended recipient you are hereby
>>>> notified
>>>> that any disclosure, copying, distribution or taking any action in
>>>> reliance
>>>> on the contents of this information is strictly prohibited and may be
>>>> unlawful. If you have received this communication in error, please notify
>>>> us
>>>> immediately by responding to this email and then delete it from your
>>>> system.
>>>> The firm is neither liable for the proper and complete transmission of
>>>> the
>>>> information contained in this communication nor for any delay in its
>>>> receipt.
>>>>
>>>>
>>>> .
>>>>
>>>
>>>
>> The information contained in this communication is intended solely for the
>> use of the individual or entity to whom it is addressed and others
>> authorized to receive it. It may contain confidential or legally privileged
>> information. If you are not the intended recipient you are hereby notified
>> that any disclosure, copying, distribution or taking any action in reliance
>> on the contents of this information is strictly prohibited and may be
>> unlawful. If you have received this communication in error, please notify us
>> immediately by responding to this email and then delete it from your system.
>> The firm is neither liable for the proper and complete transmission of the
>> information contained in this communication nor for any delay in its
>> receipt.
>>
>>
> .
>
>
The information contained in this communication is intended solely for the use of the individual or entity to whom it is addressed and others authorized to receive it. It may contain confidential or legally privileged information. If you are not the intended recipient you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by responding to this email and then delete it from your system. The firm is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt.
Re: Embedded Pig + MQO + Logical Plan
Posted by Dmitriy Ryaboy <dv...@gmail.com>.
Looks like it's on automatically.
Code below is from trunk, but I don't think this changed recently. I got rid
of exception handling for conciseness.
In PigServer:
public void registerScript(String fileName) throws IOException {
GruntParser grunt = new GruntParser(new FileReader(new
File(fileName)));
* grunt.setInteractive(false);*
grunt.setParams(this);
grunt.parseStopOnError(true);
}
In GruntParser:
public int[] parseStopOnError(boolean sameBatch) throws IOException,
ParseException {
* if (!mInteractive && !sameBatch) {
setBatchOn();
}
* prompt();
mDone = false;
while(!mDone) {
parse();
}
if (!sameBatch) {
executeBatch();
}
int [] res = { mNumSucceededJobs, mNumFailedJobs };
return res;
}
On Thu, Mar 4, 2010 at 10:00 AM, Rohan Rai <ro...@inmobi.com> wrote:
> Thanks Dmitriy
>
> Just a question more
>
> registerScript allows to register a pig script in the embedded mode
> So the confusion was does it internally tries to optimize it.
> or setBatchOn has to be explicitly called
>
> Regards
> Rohan
>
>
> Dmitriy Ryaboy wrote:
>
>> 1) Automatically, if you call it right. Look for the setBatchOn and
>> executeBatch methods (I may be slightly off on the method names, going off
>> memory)
>>
>> 2) The optimizer moves stuff around and may be executing things in a
>> slightly different order then what you tell it. This can mean pushing up
>> projections, filters, and limits, inserting casts, and doing all kinds of
>> other manipulations. The logical plan shows you what's going to happen
>> without breaking it down into the MR plan. There are further optimizations
>> at the MR level, so both are worth checking. In practice I usually look at
>> the logical plan for order-of-operations and general sanity checking, and
>> at
>> the MR plan for number of jobs and whether things like algebraic and
>> accumulative interfaces are kicking in.
>>
>> 3) Yes. Roughly speaking, one map per block will be generated. The bigger
>> the block, the more work per mapper. The smaller the block, the more
>> mappers. Depending on the workload, there's an optimal value.
>>
>> 4) Playing with logical plan -- don't :-). It's exposed so that you can
>> look
>> at what's going on, and not intended to let you change execution plans.
>> Unless you actually want to hack Pig guts. If that's the case, look at the
>> optimizer and the MRCompiler classes to see how it's getting modified and
>> used.
>>
>> -D
>>
>> On Thu, Mar 4, 2010 at 9:14 AM, Rohan Rai <ro...@inmobi.com> wrote:
>>
>>
>> On using embedded Pig Server and registering a pig script for execution
>>>
>>> 1) Does Multi Query Optimization happens automatically, or has to
>>> explicitly told so.
>>>
>>> 2) Logical Plan. What one can infer out of it.
>>>
>>> 3) Does the Block Size (defined in hadoop) has an effect on performance
>>> or the number of map job getting selected.
>>>
>>> Regards
>>> Rohan
>>>
>>> The information contained in this communication is intended solely for
>>> the
>>> use of the individual or entity to whom it is addressed and others
>>> authorized to receive it. It may contain confidential or legally
>>> privileged
>>> information. If you are not the intended recipient you are hereby
>>> notified
>>> that any disclosure, copying, distribution or taking any action in
>>> reliance
>>> on the contents of this information is strictly prohibited and may be
>>> unlawful. If you have received this communication in error, please notify
>>> us
>>> immediately by responding to this email and then delete it from your
>>> system.
>>> The firm is neither liable for the proper and complete transmission of
>>> the
>>> information contained in this communication nor for any delay in its
>>> receipt.
>>>
>>>
>>> .
>>
>>
>>
>
> The information contained in this communication is intended solely for the
> use of the individual or entity to whom it is addressed and others
> authorized to receive it. It may contain confidential or legally privileged
> information. If you are not the intended recipient you are hereby notified
> that any disclosure, copying, distribution or taking any action in reliance
> on the contents of this information is strictly prohibited and may be
> unlawful. If you have received this communication in error, please notify us
> immediately by responding to this email and then delete it from your system.
> The firm is neither liable for the proper and complete transmission of the
> information contained in this communication nor for any delay in its
> receipt.
>
Re: Embedded Pig + MQO + Logical Plan
Posted by Rohan Rai <ro...@inmobi.com>.
Thanks Dmitriy
Just a question more
registerScript allows to register a pig script in the embedded mode
So the confusion was does it internally tries to optimize it.
or setBatchOn has to be explicitly called
Regards
Rohan
Dmitriy Ryaboy wrote:
> 1) Automatically, if you call it right. Look for the setBatchOn and
> executeBatch methods (I may be slightly off on the method names, going off
> memory)
>
> 2) The optimizer moves stuff around and may be executing things in a
> slightly different order then what you tell it. This can mean pushing up
> projections, filters, and limits, inserting casts, and doing all kinds of
> other manipulations. The logical plan shows you what's going to happen
> without breaking it down into the MR plan. There are further optimizations
> at the MR level, so both are worth checking. In practice I usually look at
> the logical plan for order-of-operations and general sanity checking, and at
> the MR plan for number of jobs and whether things like algebraic and
> accumulative interfaces are kicking in.
>
> 3) Yes. Roughly speaking, one map per block will be generated. The bigger
> the block, the more work per mapper. The smaller the block, the more
> mappers. Depending on the workload, there's an optimal value.
>
> 4) Playing with logical plan -- don't :-). It's exposed so that you can look
> at what's going on, and not intended to let you change execution plans.
> Unless you actually want to hack Pig guts. If that's the case, look at the
> optimizer and the MRCompiler classes to see how it's getting modified and
> used.
>
> -D
>
> On Thu, Mar 4, 2010 at 9:14 AM, Rohan Rai <ro...@inmobi.com> wrote:
>
>
>> On using embedded Pig Server and registering a pig script for execution
>>
>> 1) Does Multi Query Optimization happens automatically, or has to
>> explicitly told so.
>>
>> 2) Logical Plan. What one can infer out of it.
>>
>> 3) Does the Block Size (defined in hadoop) has an effect on performance
>> or the number of map job getting selected.
>>
>> Regards
>> Rohan
>>
>> The information contained in this communication is intended solely for the
>> use of the individual or entity to whom it is addressed and others
>> authorized to receive it. It may contain confidential or legally privileged
>> information. If you are not the intended recipient you are hereby notified
>> that any disclosure, copying, distribution or taking any action in reliance
>> on the contents of this information is strictly prohibited and may be
>> unlawful. If you have received this communication in error, please notify us
>> immediately by responding to this email and then delete it from your system.
>> The firm is neither liable for the proper and complete transmission of the
>> information contained in this communication nor for any delay in its
>> receipt.
>>
>>
> .
>
>
The information contained in this communication is intended solely for the use of the individual or entity to whom it is addressed and others authorized to receive it. It may contain confidential or legally privileged information. If you are not the intended recipient you are hereby notified that any disclosure, copying, distribution or taking any action in reliance on the contents of this information is strictly prohibited and may be unlawful. If you have received this communication in error, please notify us immediately by responding to this email and then delete it from your system. The firm is neither liable for the proper and complete transmission of the information contained in this communication nor for any delay in its receipt.
Re: Embedded Pig + MQO + Logical Plan
Posted by Dmitriy Ryaboy <dv...@gmail.com>.
1) Automatically, if you call it right. Look for the setBatchOn and
executeBatch methods (I may be slightly off on the method names, going off
memory)
2) The optimizer moves stuff around and may be executing things in a
slightly different order then what you tell it. This can mean pushing up
projections, filters, and limits, inserting casts, and doing all kinds of
other manipulations. The logical plan shows you what's going to happen
without breaking it down into the MR plan. There are further optimizations
at the MR level, so both are worth checking. In practice I usually look at
the logical plan for order-of-operations and general sanity checking, and at
the MR plan for number of jobs and whether things like algebraic and
accumulative interfaces are kicking in.
3) Yes. Roughly speaking, one map per block will be generated. The bigger
the block, the more work per mapper. The smaller the block, the more
mappers. Depending on the workload, there's an optimal value.
4) Playing with logical plan -- don't :-). It's exposed so that you can look
at what's going on, and not intended to let you change execution plans.
Unless you actually want to hack Pig guts. If that's the case, look at the
optimizer and the MRCompiler classes to see how it's getting modified and
used.
-D
On Thu, Mar 4, 2010 at 9:14 AM, Rohan Rai <ro...@inmobi.com> wrote:
> On using embedded Pig Server and registering a pig script for execution
>
> 1) Does Multi Query Optimization happens automatically, or has to
> explicitly told so.
>
> 2) Logical Plan. What one can infer out of it.
>
> 3) Does the Block Size (defined in hadoop) has an effect on performance
> or the number of map job getting selected.
>
> Regards
> Rohan
>
> The information contained in this communication is intended solely for the
> use of the individual or entity to whom it is addressed and others
> authorized to receive it. It may contain confidential or legally privileged
> information. If you are not the intended recipient you are hereby notified
> that any disclosure, copying, distribution or taking any action in reliance
> on the contents of this information is strictly prohibited and may be
> unlawful. If you have received this communication in error, please notify us
> immediately by responding to this email and then delete it from your system.
> The firm is neither liable for the proper and complete transmission of the
> information contained in this communication nor for any delay in its
> receipt.
>