You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@ambari.apache.org by Jonathan Hurley <jh...@hortonworks.com> on 2017/12/05 18:33:20 UTC
Review Request 64348: Pig service check failed after PU with LzoCodec
CNF
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64348/
-----------------------------------------------------------
Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, and Nate Cole.
Bugs: AMBARI-22598
https://issues.apache.org/jira/browse/AMBARI-22598
Repository: ambari
Description
-------
When Pig (and other hadoop clients) are upgraded by themselves, they need to create jobs with versions of the LZO libraries which match _their_ versions.
*STR*
# Deployed cluster with Ambari version: 2.5.1.0-159 and HDP version: 2.6.1.0-129 (HDFS has lzo enabled)
# Upgrade Ambari to Target Version: 2.6.1.0-84
# Perform full stack upgrade to 2.6.4.0-55
# Perform 4th digit PU of some services in batches (see attached for list)
# Run Pig service check
*Result*
```
Caused by: java.lang.IllegalArgumentException: Compression codec com.hadoop.compression.lzo.LzoCodec not found.
at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:139)
at org.apache.hadoop.io.compress.CompressionCodecFactory.<init>(CompressionCodecFactory.java:180)
at org.apache.hadoop.mapreduce.lib.input.TextInputFormat.isSplitable(TextInputFormat.java:59)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:399)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:265)
... 28 more
Caused by: java.lang.ClassNotFoundException: Class com.hadoop.compression.lzo.LzoCodec not found
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2147)
at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:132)
... 32 more
2017-12-04 12:07:30,322 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2118: Compression codec com.hadoop.compression.lzo.LzoCodec not found.
Details at logfile: /home/ambari-qa/pig_1512389247042.log
2017-12-04 12:07:30,338 [main] INFO org.apache.pig.Main - Pig script completed in 3 seconds and 457 milliseconds (3457 ms)
2017-12-04 12:07:30,338 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Shutting down thread pool
```
Diffs
-----
ambari-server/src/main/resources/common-services/MAHOUT/1.0.0.2.3/package/scripts/mahout.py f2c3c18077
ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/pig.py 83f70488a3
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/setup_spark.py 426509cb0e
ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/setup_spark.py 82b6c63d6d
Diff: https://reviews.apache.org/r/64348/diff/1/
Testing
-------
OK
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:29 min
[INFO] Finished at: 2017-12-05T13:27:57-05:00
[INFO] Final Memory: 21M/619M
[INFO] ------------------------------------------------------------------------
Thanks,
Jonathan Hurley
Re: Review Request 64348: Pig service check failed after PU with
LzoCodec CNF
Posted by Dmytro Grinenko <ha...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64348/#review192898
-----------------------------------------------------------
Ship it!
Ship It!
- Dmytro Grinenko
On Dec. 5, 2017, 6:33 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64348/
> -----------------------------------------------------------
>
> (Updated Dec. 5, 2017, 6:33 p.m.)
>
>
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, and Nate Cole.
>
>
> Bugs: AMBARI-22598
> https://issues.apache.org/jira/browse/AMBARI-22598
>
>
> Repository: ambari
>
>
> Description
> -------
>
> When Pig (and other hadoop clients) are upgraded by themselves, they need to create jobs with versions of the LZO libraries which match _their_ versions.
>
> *STR*
> # Deployed cluster with Ambari version: 2.5.1.0-159 and HDP version: 2.6.1.0-129 (HDFS has lzo enabled)
> # Upgrade Ambari to Target Version: 2.6.1.0-84
> # Perform full stack upgrade to 2.6.4.0-55
> # Perform 4th digit PU of some services in batches (see attached for list)
> # Run Pig service check
>
> *Result*
> ```
> Caused by: java.lang.IllegalArgumentException: Compression codec com.hadoop.compression.lzo.LzoCodec not found.
> at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:139)
> at org.apache.hadoop.io.compress.CompressionCodecFactory.<init>(CompressionCodecFactory.java:180)
> at org.apache.hadoop.mapreduce.lib.input.TextInputFormat.isSplitable(TextInputFormat.java:59)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:399)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:265)
> ... 28 more
> Caused by: java.lang.ClassNotFoundException: Class com.hadoop.compression.lzo.LzoCodec not found
> at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2147)
> at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:132)
> ... 32 more
> 2017-12-04 12:07:30,322 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2118: Compression codec com.hadoop.compression.lzo.LzoCodec not found.
> Details at logfile: /home/ambari-qa/pig_1512389247042.log
> 2017-12-04 12:07:30,338 [main] INFO org.apache.pig.Main - Pig script completed in 3 seconds and 457 milliseconds (3457 ms)
> 2017-12-04 12:07:30,338 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Shutting down thread pool
> ```
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/MAHOUT/1.0.0.2.3/package/scripts/mahout.py f2c3c18077
> ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/pig.py 83f70488a3
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/setup_spark.py 426509cb0e
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/setup_spark.py 82b6c63d6d
>
>
> Diff: https://reviews.apache.org/r/64348/diff/1/
>
>
> Testing
> -------
>
> OK
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 01:29 min
> [INFO] Finished at: 2017-12-05T13:27:57-05:00
> [INFO] Final Memory: 21M/619M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Jonathan Hurley
>
>
Re: Review Request 64348: Pig service check failed after PU with
LzoCodec CNF
Posted by Dmitro Lisnichenko <dl...@hortonworks.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/64348/#review192974
-----------------------------------------------------------
Ship it!
Ship It!
- Dmitro Lisnichenko
On Dec. 5, 2017, 8:33 p.m., Jonathan Hurley wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/64348/
> -----------------------------------------------------------
>
> (Updated Dec. 5, 2017, 8:33 p.m.)
>
>
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, and Nate Cole.
>
>
> Bugs: AMBARI-22598
> https://issues.apache.org/jira/browse/AMBARI-22598
>
>
> Repository: ambari
>
>
> Description
> -------
>
> When Pig (and other hadoop clients) are upgraded by themselves, they need to create jobs with versions of the LZO libraries which match _their_ versions.
>
> *STR*
> # Deployed cluster with Ambari version: 2.5.1.0-159 and HDP version: 2.6.1.0-129 (HDFS has lzo enabled)
> # Upgrade Ambari to Target Version: 2.6.1.0-84
> # Perform full stack upgrade to 2.6.4.0-55
> # Perform 4th digit PU of some services in batches (see attached for list)
> # Run Pig service check
>
> *Result*
> ```
> Caused by: java.lang.IllegalArgumentException: Compression codec com.hadoop.compression.lzo.LzoCodec not found.
> at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:139)
> at org.apache.hadoop.io.compress.CompressionCodecFactory.<init>(CompressionCodecFactory.java:180)
> at org.apache.hadoop.mapreduce.lib.input.TextInputFormat.isSplitable(TextInputFormat.java:59)
> at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:399)
> at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:265)
> ... 28 more
> Caused by: java.lang.ClassNotFoundException: Class com.hadoop.compression.lzo.LzoCodec not found
> at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2147)
> at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:132)
> ... 32 more
> 2017-12-04 12:07:30,322 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2118: Compression codec com.hadoop.compression.lzo.LzoCodec not found.
> Details at logfile: /home/ambari-qa/pig_1512389247042.log
> 2017-12-04 12:07:30,338 [main] INFO org.apache.pig.Main - Pig script completed in 3 seconds and 457 milliseconds (3457 ms)
> 2017-12-04 12:07:30,338 [main] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezLauncher - Shutting down thread pool
> ```
>
>
> Diffs
> -----
>
> ambari-server/src/main/resources/common-services/MAHOUT/1.0.0.2.3/package/scripts/mahout.py f2c3c18077
> ambari-server/src/main/resources/common-services/PIG/0.12.0.2.0/package/scripts/pig.py 83f70488a3
> ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/setup_spark.py 426509cb0e
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/setup_spark.py 82b6c63d6d
>
>
> Diff: https://reviews.apache.org/r/64348/diff/1/
>
>
> Testing
> -------
>
> OK
> [INFO] ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------------------
> [INFO] Total time: 01:29 min
> [INFO] Finished at: 2017-12-05T13:27:57-05:00
> [INFO] Final Memory: 21M/619M
> [INFO] ------------------------------------------------------------------------
>
>
> Thanks,
>
> Jonathan Hurley
>
>