You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@ambari.apache.org by Attila Doroszlai <ad...@hortonworks.com> on 2017/03/13 12:47:58 UTC

Review Request 57557: AMBARI-20417. Spark2 Thrift Server cannot be started after upgrade due to incompletely uploaded tar.gz

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57557/
-----------------------------------------------------------

Review request for Ambari, Bikas Saha, Laszlo Puskas, and Sandor Magyari.


Bugs: AMBARI-20417
    https://issues.apache.org/jira/browse/AMBARI-20417


Repository: ambari


Description
-------

Fix directory permissions even if already present.  Force upload to HDFS.


Diffs
-----

  ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py c8a2b33ff158fe781926bf88c03be471fa9de1ca 


Diff: https://reviews.apache.org/r/57557/diff/1/


Testing
-------

Manual test according to steps to reproduce.


Thanks,

Attila Doroszlai

Re: Review Request 57557: AMBARI-20417. Spark2 Thrift Server cannot be started after upgrade due to incompletely uploaded tar.gz

Posted by Alejandro Fernandez <af...@hortonworks.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57557/#review168913
-----------------------------------------------------------


Ship it!




Ship It!

- Alejandro Fernandez


On March 13, 2017, 12:47 p.m., Attila Doroszlai wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57557/
> -----------------------------------------------------------
> 
> (Updated March 13, 2017, 12:47 p.m.)
> 
> 
> Review request for Ambari, Bikas Saha, Laszlo Puskas, and Sandor Magyari.
> 
> 
> Bugs: AMBARI-20417
>     https://issues.apache.org/jira/browse/AMBARI-20417
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Fix directory permissions even if already present.  Force upload to HDFS.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py c8a2b33ff158fe781926bf88c03be471fa9de1ca 
> 
> 
> Diff: https://reviews.apache.org/r/57557/diff/1/
> 
> 
> Testing
> -------
> 
> Manual test according to steps to reproduce.
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 57557: AMBARI-20417. Spark2 Thrift Server cannot be started after upgrade due to incompletely uploaded tar.gz

Posted by Attila Doroszlai <ad...@hortonworks.com>.


> On March 13, 2017, 6:32 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py
> > Line 66 (original), 66 (patched)
> > <https://reviews.apache.org/r/57557/diff/1/?file=1662688#file1662688line66>
> >
> >     Why does this need to replace the existing tarball?
> >     The tarball comes from a specific version which shouldn't be changing?
> >     I'm concerned this will cause the Spark Service Check to be slower.

This code does not run during service check, only when Spark2 History Server is started.


- Attila


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57557/#review168789
-----------------------------------------------------------


On March 13, 2017, 1:47 p.m., Attila Doroszlai wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57557/
> -----------------------------------------------------------
> 
> (Updated March 13, 2017, 1:47 p.m.)
> 
> 
> Review request for Ambari, Bikas Saha, Laszlo Puskas, and Sandor Magyari.
> 
> 
> Bugs: AMBARI-20417
>     https://issues.apache.org/jira/browse/AMBARI-20417
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Fix directory permissions even if already present.  Force upload to HDFS.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py c8a2b33ff158fe781926bf88c03be471fa9de1ca 
> 
> 
> Diff: https://reviews.apache.org/r/57557/diff/1/
> 
> 
> Testing
> -------
> 
> Manual test according to steps to reproduce.
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 57557: AMBARI-20417. Spark2 Thrift Server cannot be started after upgrade due to incompletely uploaded tar.gz

Posted by Alejandro Fernandez <af...@hortonworks.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57557/#review168789
-----------------------------------------------------------




ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py
Line 66 (original), 66 (patched)
<https://reviews.apache.org/r/57557/#comment241043>

    Why does this need to replace the existing tarball?
    The tarball comes from a specific version which shouldn't be changing?
    I'm concerned this will cause the Spark Service Check to be slower.


- Alejandro Fernandez


On March 13, 2017, 12:47 p.m., Attila Doroszlai wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57557/
> -----------------------------------------------------------
> 
> (Updated March 13, 2017, 12:47 p.m.)
> 
> 
> Review request for Ambari, Bikas Saha, Laszlo Puskas, and Sandor Magyari.
> 
> 
> Bugs: AMBARI-20417
>     https://issues.apache.org/jira/browse/AMBARI-20417
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Fix directory permissions even if already present.  Force upload to HDFS.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py c8a2b33ff158fe781926bf88c03be471fa9de1ca 
> 
> 
> Diff: https://reviews.apache.org/r/57557/diff/1/
> 
> 
> Testing
> -------
> 
> Manual test according to steps to reproduce.
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 57557: AMBARI-20417. Spark2 Thrift Server cannot be started after upgrade due to incompletely uploaded tar.gz

Posted by Attila Doroszlai <ad...@hortonworks.com>.


> On March 13, 2017, 6:43 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py
> > Line 66 (original), 66 (patched)
> > <https://reviews.apache.org/r/57557/diff/1/?file=1662688#file1662688line66>
> >
> >     Ok, question still applies. This will then make starting slower.

`copy_to_hdfs` will skip copying the file if its identical to the new source file.

```
2017-03-14 15:00:52,265 - DFS file /hdp/apps/2.5.3.0-37/spark2/spark2-hdp-yarn-archive.tar.gz is identical to /tmp/spark2/spark2-hdp-yarn-archive.tar.gz, skipping the copying
```


- Attila


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57557/#review168791
-----------------------------------------------------------


On March 13, 2017, 1:47 p.m., Attila Doroszlai wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57557/
> -----------------------------------------------------------
> 
> (Updated March 13, 2017, 1:47 p.m.)
> 
> 
> Review request for Ambari, Bikas Saha, Laszlo Puskas, and Sandor Magyari.
> 
> 
> Bugs: AMBARI-20417
>     https://issues.apache.org/jira/browse/AMBARI-20417
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Fix directory permissions even if already present.  Force upload to HDFS.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py c8a2b33ff158fe781926bf88c03be471fa9de1ca 
> 
> 
> Diff: https://reviews.apache.org/r/57557/diff/1/
> 
> 
> Testing
> -------
> 
> Manual test according to steps to reproduce.
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 57557: AMBARI-20417. Spark2 Thrift Server cannot be started after upgrade due to incompletely uploaded tar.gz

Posted by Alejandro Fernandez <af...@hortonworks.com>.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57557/#review168791
-----------------------------------------------------------




ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py
Line 66 (original), 66 (patched)
<https://reviews.apache.org/r/57557/#comment241045>

    Ok, question still applies. This will then make starting slower.


- Alejandro Fernandez


On March 13, 2017, 12:47 p.m., Attila Doroszlai wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57557/
> -----------------------------------------------------------
> 
> (Updated March 13, 2017, 12:47 p.m.)
> 
> 
> Review request for Ambari, Bikas Saha, Laszlo Puskas, and Sandor Magyari.
> 
> 
> Bugs: AMBARI-20417
>     https://issues.apache.org/jira/browse/AMBARI-20417
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Fix directory permissions even if already present.  Force upload to HDFS.
> 
> 
> Diffs
> -----
> 
>   ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py c8a2b33ff158fe781926bf88c03be471fa9de1ca 
> 
> 
> Diff: https://reviews.apache.org/r/57557/diff/1/
> 
> 
> Testing
> -------
> 
> Manual test according to steps to reproduce.
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>