You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bigtop.apache.org by "Roman Shaposhnik (JIRA)" <ji...@apache.org> on 2012/08/14 18:54:38 UTC
[jira] [Created] (BIGTOP-688) improve hue packaging via making
virtual env relocatable and moving DB files into /var/lib/hue
Roman Shaposhnik created BIGTOP-688:
---------------------------------------
Summary: improve hue packaging via making virtual env relocatable and moving DB files into /var/lib/hue
Key: BIGTOP-688
URL: https://issues.apache.org/jira/browse/BIGTOP-688
Project: Bigtop
Issue Type: Improvement
Components: General
Affects Versions: 0.4.0
Reporter: Roman Shaposhnik
Assignee: Roman Shaposhnik
Fix For: 0.5.0
Currently Hue packaging does unfortunate things with permissions and mutability of things under /usr/lib/hue. We should make sure our virtual env is relocatable and can be used in a R/O fashion and we should move mutable bits (desktop.db and app.reg) under /var/lib/hue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (BIGTOP-688) improve hue packaging via
making virtual env relocatable and moving DB files into /var/lib/hue
Posted by "Anatoli Fomenko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13440574#comment-13440574 ]
Anatoli Fomenko edited comment on BIGTOP-688 at 8/24/12 6:22 AM:
-----------------------------------------------------------------
Some comments on the patch:
# In install_hue.sh there are 2 variables related to logs: LOG_DIR and DESKTOP_LOG_DIR. While they may be required by the environment, it may be cleaner to keep it to one.
# In hue-app-postinstall.tpl, hue-app.prerm.tpl and other app related files kept some variables that are removed in other files, such as ROOT=/usr/lib/hue
# In a few places user name "hue" is used without defining USER=hue
# Build successful on Precise
# Package install successful
Tried a few commands. Met this problem:
{noformat}
/usr/lib/hue/build/env/bin]$ sudo ./hue test
Traceback (most recent call last):
File "./hue", line 9, in <module>
load_entry_point('desktop==2.0.1', 'console_scripts', 'hue')()
File "/usr/lib/hue/desktop/core/src/desktop/manage_entry.py", line
60, in entry
execute_manager(settings)
File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.2.3-py2.7.egg/django/core/management/__init__.py",
line 438, in execute_manager
utility.execute()
File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.2.3-py2.7.egg/django/core/management/__init__.py",
line 379, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.2.3-py2.7.egg/django/core/management/__init__.py",
line 261, in fetch_command
klass = load_command_class(app_name, subcommand)
File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.2.3-py2.7.egg/django/core/management/__init__.py",
line 67, in load_command_class
module = import_module('%s.management.commands.%s' % (app_name, name))
File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.2.3-py2.7.egg/django/utils/importlib.py",
line 35, in import_module
__import__(name)
File "/usr/lib/hue/desktop/core/src/desktop/management/commands/test.py",
line 24, in <module>
from django_nose import nose_runner
File "/usr/lib/hue/build/env/lib/python2.7/site-packages/django_nose-0.5-py2.7.egg/django_nose/nose_runner.py",
line 22, in <module>
import nose
ImportError: No module named nose
{noformat}
After installing python-django-nose:
{noformat}
sudo ./hue test
Traceback (most recent call last):
File "./hue", line 9, in <module>
load_entry_point('desktop==2.0.1', 'console_scripts', 'hue')()
File "/usr/lib/hue/desktop/core/src/desktop/manage_entry.py", line
60, in entry
execute_manager(settings)
File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.2.3-py2.7.egg/django/core/management/__init__.py",
line 438, in execute_manager
utility.execute()
File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.2.3-py2.7.egg/django/core/management/__init__.py",
line 379, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.2.3-py2.7.egg/django/core/management/__init__.py",
line 261, in fetch_command
klass = load_command_class(app_name, subcommand)
File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.2.3-py2.7.egg/django/core/management/__init__.py",
line 67, in load_command_class
module = import_module('%s.management.commands.%s' % (app_name, name))
File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.2.3-py2.7.egg/django/utils/importlib.py",
line 35, in import_module
__import__(name)
File "/usr/lib/hue/desktop/core/src/desktop/management/commands/test.py",
line 32, in <module>
from desktop.management.commands import test_windmill
File "/usr/lib/hue/desktop/core/src/desktop/management/commands/test_windmill.py",
line 27, in <module>
from windmill.authoring import djangotest
ImportError: No module named windmill.authoring
{noformat}
Obviously, test_windmill also fails.
Most other commands run fine.
Off-topic question: why the build system does not detect change in the src files, only `rm -rf <component>/build`?
was (Author: anatoli.fomenko):
Some comments on the patch:
# In install_hue.sh there are 2 variables related to logs: LOG_DIR and DESKTOP_LOG_DIR. While they may be required by the environment, it may be cleaner to keep it to one.
# In hue-app-postinstall.tpl, hue-app.prerm.tpl and other app related files kept some variables that are removed in other files, such as ROOT=/usr/lib/hue
# In a few places user name "hue" is used without defining USER=hue
# Build successful on Precise
# Package install successful
Tried a few commands. Met this problem:
(noformat)
/usr/lib/hue/build/env/bin]$ sudo ./hue test
Traceback (most recent call last):
File "./hue", line 9, in <module>
load_entry_point('desktop==2.0.1', 'console_scripts', 'hue')()
File "/usr/lib/hue/desktop/core/src/desktop/manage_entry.py", line
60, in entry
execute_manager(settings)
File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.2.3-py2.7.egg/django/core/management/__init__.py",
line 438, in execute_manager
utility.execute()
File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.2.3-py2.7.egg/django/core/management/__init__.py",
line 379, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.2.3-py2.7.egg/django/core/management/__init__.py",
line 261, in fetch_command
klass = load_command_class(app_name, subcommand)
File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.2.3-py2.7.egg/django/core/management/__init__.py",
line 67, in load_command_class
module = import_module('%s.management.commands.%s' % (app_name, name))
File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.2.3-py2.7.egg/django/utils/importlib.py",
line 35, in import_module
__import__(name)
File "/usr/lib/hue/desktop/core/src/desktop/management/commands/test.py",
line 24, in <module>
from django_nose import nose_runner
File "/usr/lib/hue/build/env/lib/python2.7/site-packages/django_nose-0.5-py2.7.egg/django_nose/nose_runner.py",
line 22, in <module>
import nose
ImportError: No module named nose
(noformat)
After installing python-django-nose:
(noformat)
sudo ./hue test
Traceback (most recent call last):
File "./hue", line 9, in <module>
load_entry_point('desktop==2.0.1', 'console_scripts', 'hue')()
File "/usr/lib/hue/desktop/core/src/desktop/manage_entry.py", line
60, in entry
execute_manager(settings)
File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.2.3-py2.7.egg/django/core/management/__init__.py",
line 438, in execute_manager
utility.execute()
File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.2.3-py2.7.egg/django/core/management/__init__.py",
line 379, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.2.3-py2.7.egg/django/core/management/__init__.py",
line 261, in fetch_command
klass = load_command_class(app_name, subcommand)
File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.2.3-py2.7.egg/django/core/management/__init__.py",
line 67, in load_command_class
module = import_module('%s.management.commands.%s' % (app_name, name))
File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.2.3-py2.7.egg/django/utils/importlib.py",
line 35, in import_module
__import__(name)
File "/usr/lib/hue/desktop/core/src/desktop/management/commands/test.py",
line 32, in <module>
from desktop.management.commands import test_windmill
File "/usr/lib/hue/desktop/core/src/desktop/management/commands/test_windmill.py",
line 27, in <module>
from windmill.authoring import djangotest
ImportError: No module named windmill.authoring
(noformat)
Obviously, test_windmill also fails.
Most other commands run fine.
Off-topic question: why the build system does not detect change in the src files, only `rm -rf <component>/build`?
> improve hue packaging via making virtual env relocatable and moving DB files into /var/lib/hue
> ----------------------------------------------------------------------------------------------
>
> Key: BIGTOP-688
> URL: https://issues.apache.org/jira/browse/BIGTOP-688
> Project: Bigtop
> Issue Type: Improvement
> Components: General
> Affects Versions: 0.4.0
> Reporter: Roman Shaposhnik
> Assignee: Roman Shaposhnik
> Fix For: 0.5.0
>
> Attachments: BIGTOP-688.patch.txt
>
>
> Currently Hue packaging does unfortunate things with permissions and mutability of things under /usr/lib/hue. We should make sure our virtual env is relocatable and can be used in a R/O fashion and we should move mutable bits (desktop.db and app.reg) under /var/lib/hue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-688) improve hue packaging via making
virtual env relocatable and moving DB files into /var/lib/hue
Posted by "Sean Mackrory (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13456136#comment-13456136 ]
Sean Mackrory commented on BIGTOP-688:
--------------------------------------
+1
> improve hue packaging via making virtual env relocatable and moving DB files into /var/lib/hue
> ----------------------------------------------------------------------------------------------
>
> Key: BIGTOP-688
> URL: https://issues.apache.org/jira/browse/BIGTOP-688
> Project: Bigtop
> Issue Type: Improvement
> Components: General
> Affects Versions: 0.4.0
> Reporter: Roman Shaposhnik
> Assignee: Roman Shaposhnik
> Fix For: 0.5.0
>
> Attachments: BIGTOP-688.patch.txt
>
>
> Currently Hue packaging does unfortunate things with permissions and mutability of things under /usr/lib/hue. We should make sure our virtual env is relocatable and can be used in a R/O fashion and we should move mutable bits (desktop.db and app.reg) under /var/lib/hue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (BIGTOP-688) improve hue packaging via making
virtual env relocatable and moving DB files into /var/lib/hue
Posted by "Roman Shaposhnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Roman Shaposhnik resolved BIGTOP-688.
-------------------------------------
Resolution: Fixed
> improve hue packaging via making virtual env relocatable and moving DB files into /var/lib/hue
> ----------------------------------------------------------------------------------------------
>
> Key: BIGTOP-688
> URL: https://issues.apache.org/jira/browse/BIGTOP-688
> Project: Bigtop
> Issue Type: Improvement
> Components: General
> Affects Versions: 0.4.0
> Reporter: Roman Shaposhnik
> Assignee: Roman Shaposhnik
> Fix For: 0.5.0
>
> Attachments: BIGTOP-688.patch.txt
>
>
> Currently Hue packaging does unfortunate things with permissions and mutability of things under /usr/lib/hue. We should make sure our virtual env is relocatable and can be used in a R/O fashion and we should move mutable bits (desktop.db and app.reg) under /var/lib/hue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-688) improve hue packaging via making
virtual env relocatable and moving DB files into /var/lib/hue
Posted by "Anatoli Fomenko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13440927#comment-13440927 ]
Anatoli Fomenko commented on BIGTOP-688:
----------------------------------------
@Roman
{quote}
In general Bigtop packaging tries to remove test code from packages. Hence any residual code is not expected to work. An argument could be had that we should do a better job of completely removing hue test code, but it looks pretty tightly integrated. Do you want to take a look at it?
{quote}
Good idea. Thanks.
{quote}
??Off-topic question: why the build system does not detect change in the src files, only `rm -rf <component>/build`???
Which build system are you referring to? The Bigtop itself or Hue? Can you, please, be more specific?
{quote}
Bigtop build system. Perhaps I should move this question to a separate thread, for better coverage.
> improve hue packaging via making virtual env relocatable and moving DB files into /var/lib/hue
> ----------------------------------------------------------------------------------------------
>
> Key: BIGTOP-688
> URL: https://issues.apache.org/jira/browse/BIGTOP-688
> Project: Bigtop
> Issue Type: Improvement
> Components: General
> Affects Versions: 0.4.0
> Reporter: Roman Shaposhnik
> Assignee: Roman Shaposhnik
> Fix For: 0.5.0
>
> Attachments: BIGTOP-688.patch.txt
>
>
> Currently Hue packaging does unfortunate things with permissions and mutability of things under /usr/lib/hue. We should make sure our virtual env is relocatable and can be used in a R/O fashion and we should move mutable bits (desktop.db and app.reg) under /var/lib/hue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (BIGTOP-688) improve hue packaging via making
virtual env relocatable and moving DB files into /var/lib/hue
Posted by "Roman Shaposhnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Roman Shaposhnik updated BIGTOP-688:
------------------------------------
Attachment: BIGTOP-688.patch.txt
> improve hue packaging via making virtual env relocatable and moving DB files into /var/lib/hue
> ----------------------------------------------------------------------------------------------
>
> Key: BIGTOP-688
> URL: https://issues.apache.org/jira/browse/BIGTOP-688
> Project: Bigtop
> Issue Type: Improvement
> Components: General
> Affects Versions: 0.4.0
> Reporter: Roman Shaposhnik
> Assignee: Roman Shaposhnik
> Fix For: 0.5.0
>
> Attachments: BIGTOP-688.patch.txt
>
>
> Currently Hue packaging does unfortunate things with permissions and mutability of things under /usr/lib/hue. We should make sure our virtual env is relocatable and can be used in a R/O fashion and we should move mutable bits (desktop.db and app.reg) under /var/lib/hue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-688) improve hue packaging via making
virtual env relocatable and moving DB files into /var/lib/hue
Posted by "Roman Shaposhnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13436981#comment-13436981 ]
Roman Shaposhnik commented on BIGTOP-688:
-----------------------------------------
An additional complication is that even on systems with the default python if I don't do the PATH setting the actual executable that is used remains to be the system python, which clearly defeats the purpose of the relocatable virtual environment to begin with (the 3 PIDs are from the top level supervisor down to the rpc server):
{noformat}
// remove the path setting
# service hue start
# ls -l /proc/19736/exe
lrwxrwxrwx 1 root root 0 Aug 17 12:14 /proc/19736/exe -> /usr/bin/python2.4
# ls -l /proc/19740/exe
lrwxrwxrwx 1 root root 0 Aug 17 12:14 /proc/19740/exe -> /usr/bin/python2.4
# ls -l /proc/19772/exe
lrwxrwxrwx 1 hue hue 0 Aug 17 12:14 /proc/19772/exe -> /usr/bin/python2.4
{noformat}
> improve hue packaging via making virtual env relocatable and moving DB files into /var/lib/hue
> ----------------------------------------------------------------------------------------------
>
> Key: BIGTOP-688
> URL: https://issues.apache.org/jira/browse/BIGTOP-688
> Project: Bigtop
> Issue Type: Improvement
> Components: General
> Affects Versions: 0.4.0
> Reporter: Roman Shaposhnik
> Assignee: Roman Shaposhnik
> Fix For: 0.5.0
>
> Attachments: BIGTOP-688.patch.txt
>
>
> Currently Hue packaging does unfortunate things with permissions and mutability of things under /usr/lib/hue. We should make sure our virtual env is relocatable and can be used in a R/O fashion and we should move mutable bits (desktop.db and app.reg) under /var/lib/hue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-688) improve hue packaging via making
virtual env relocatable and moving DB files into /var/lib/hue
Posted by "bc Wong (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13436968#comment-13436968 ]
bc Wong commented on BIGTOP-688:
--------------------------------
You don't need to change the PATH. The beauty of relocating the virtualenv is that you can run it with system python. Magic.
> improve hue packaging via making virtual env relocatable and moving DB files into /var/lib/hue
> ----------------------------------------------------------------------------------------------
>
> Key: BIGTOP-688
> URL: https://issues.apache.org/jira/browse/BIGTOP-688
> Project: Bigtop
> Issue Type: Improvement
> Components: General
> Affects Versions: 0.4.0
> Reporter: Roman Shaposhnik
> Assignee: Roman Shaposhnik
> Fix For: 0.5.0
>
> Attachments: BIGTOP-688.patch.txt
>
>
> Currently Hue packaging does unfortunate things with permissions and mutability of things under /usr/lib/hue. We should make sure our virtual env is relocatable and can be used in a R/O fashion and we should move mutable bits (desktop.db and app.reg) under /var/lib/hue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-688) improve hue packaging via making
virtual env relocatable and moving DB files into /var/lib/hue
Posted by "Roman Shaposhnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13440736#comment-13440736 ]
Roman Shaposhnik commented on BIGTOP-688:
-----------------------------------------
@[~anatoli.fomenko]
bq. In install_hue.sh there are 2 variables related to logs: LOG_DIR and DESKTOP_LOG_DIR. While they may be required by the environment, it may be cleaner to keep it to one.
That's just a convention that we've got going in our _install.sh scripts. I agree that it is a bit confusing since Hue has the same var, but it is probably best to stay consistent within the _install.sh scripts.
bq. In hue-app-postinstall.tpl, hue-app.prerm.tpl and other app related files kept some variables that are removed in other files, such as ROOT=/usr/lib/hue
That's on purpose. We need this functionality to remain for things that are plugins into Hue. The model of Hue integration in Bigtop is that we prepackage all the core bits together (and thus do not need to register them dynamically) but things like beeswax and future plugins still need to run registration code at package install time.
bq. In a few places user name "hue" is used without defining USER=hue
Could you, please, be a bit more specific? I could only see that in chown lines which could arguably be changed.
bq. Tried a few commands. Met this problem: /usr/lib/hue/build/env/bin]$ sudo ./hue test
In general Bigtop packaging tries to remove test code from packages. Hence any residual code is not expected to work. An argument could be had that we should do a better job of completely removing hue test code, but it looks pretty tightly integrated. Do you want to take a look at it?
bq. Off-topic question: why the build system does not detect change in the src files, only `rm -rf <component>/build`?
Which build system are you referring to? The Bigtop itself or Hue? Can you, please, be more specific?
> improve hue packaging via making virtual env relocatable and moving DB files into /var/lib/hue
> ----------------------------------------------------------------------------------------------
>
> Key: BIGTOP-688
> URL: https://issues.apache.org/jira/browse/BIGTOP-688
> Project: Bigtop
> Issue Type: Improvement
> Components: General
> Affects Versions: 0.4.0
> Reporter: Roman Shaposhnik
> Assignee: Roman Shaposhnik
> Fix For: 0.5.0
>
> Attachments: BIGTOP-688.patch.txt
>
>
> Currently Hue packaging does unfortunate things with permissions and mutability of things under /usr/lib/hue. We should make sure our virtual env is relocatable and can be used in a R/O fashion and we should move mutable bits (desktop.db and app.reg) under /var/lib/hue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-688) improve hue packaging via making
virtual env relocatable and moving DB files into /var/lib/hue
Posted by "Roman Shaposhnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13436912#comment-13436912 ]
Roman Shaposhnik commented on BIGTOP-688:
-----------------------------------------
This is somewhat sizable patch, but what it does is actually pretty simple:
# makes the python virtual env relocatable
# moves the desktop.db out of /usr/lib/hue/desktop into /var/lib/hue
# makes everything under /usr/lib/hue owned by root and immutable
I do have one question for the python experts out there. The way that virtual env becomes relocatable includes, among other things, changing all the shebang lines inside of env/bin/<script> into:
{noformat}
#!/usr/bin/env python2.4
{noformat}
this, of course, makes it necessary to have the current location of the env (/usr/lib/hue/build/env/bin) on the PATH (see changes to the init.d scripts). I am wondering whether doing something like this:
{noformat}
- start_daemon -u $USER $DAEMON $DAEMON_OPTS
+ PATH=/usr/lib/hue/build/env/bin:$PATH start_daemon -u $USER $DAEMON $DAEMON_OPTS
{noformat}
is truly the best way to make sure that the virtual env gets used. Ideas?
> improve hue packaging via making virtual env relocatable and moving DB files into /var/lib/hue
> ----------------------------------------------------------------------------------------------
>
> Key: BIGTOP-688
> URL: https://issues.apache.org/jira/browse/BIGTOP-688
> Project: Bigtop
> Issue Type: Improvement
> Components: General
> Affects Versions: 0.4.0
> Reporter: Roman Shaposhnik
> Assignee: Roman Shaposhnik
> Fix For: 0.5.0
>
> Attachments: BIGTOP-688.patch.txt
>
>
> Currently Hue packaging does unfortunate things with permissions and mutability of things under /usr/lib/hue. We should make sure our virtual env is relocatable and can be used in a R/O fashion and we should move mutable bits (desktop.db and app.reg) under /var/lib/hue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-688) improve hue packaging via making
virtual env relocatable and moving DB files into /var/lib/hue
Posted by "Roman Shaposhnik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13436971#comment-13436971 ]
Roman Shaposhnik commented on BIGTOP-688:
-----------------------------------------
What if there's no python on the system?
> improve hue packaging via making virtual env relocatable and moving DB files into /var/lib/hue
> ----------------------------------------------------------------------------------------------
>
> Key: BIGTOP-688
> URL: https://issues.apache.org/jira/browse/BIGTOP-688
> Project: Bigtop
> Issue Type: Improvement
> Components: General
> Affects Versions: 0.4.0
> Reporter: Roman Shaposhnik
> Assignee: Roman Shaposhnik
> Fix For: 0.5.0
>
> Attachments: BIGTOP-688.patch.txt
>
>
> Currently Hue packaging does unfortunate things with permissions and mutability of things under /usr/lib/hue. We should make sure our virtual env is relocatable and can be used in a R/O fashion and we should move mutable bits (desktop.db and app.reg) under /var/lib/hue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-688) improve hue packaging via making
virtual env relocatable and moving DB files into /var/lib/hue
Posted by "bc Wong (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13436980#comment-13436980 ]
bc Wong commented on BIGTOP-688:
--------------------------------
Can we not make that a prereq? Having Python is a very reasonable thing. Trying to make the Hue distribution self-contained is a slippery slope. What about sasl lib, libxml2, libxslt, etc?
> improve hue packaging via making virtual env relocatable and moving DB files into /var/lib/hue
> ----------------------------------------------------------------------------------------------
>
> Key: BIGTOP-688
> URL: https://issues.apache.org/jira/browse/BIGTOP-688
> Project: Bigtop
> Issue Type: Improvement
> Components: General
> Affects Versions: 0.4.0
> Reporter: Roman Shaposhnik
> Assignee: Roman Shaposhnik
> Fix For: 0.5.0
>
> Attachments: BIGTOP-688.patch.txt
>
>
> Currently Hue packaging does unfortunate things with permissions and mutability of things under /usr/lib/hue. We should make sure our virtual env is relocatable and can be used in a R/O fashion and we should move mutable bits (desktop.db and app.reg) under /var/lib/hue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (BIGTOP-688) improve hue packaging via making
virtual env relocatable and moving DB files into /var/lib/hue
Posted by "Anatoli Fomenko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/BIGTOP-688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13440574#comment-13440574 ]
Anatoli Fomenko commented on BIGTOP-688:
----------------------------------------
Some comments on the patch:
# In install_hue.sh there are 2 variables related to logs: LOG_DIR and DESKTOP_LOG_DIR. While they may be required by the environment, it may be cleaner to keep it to one.
# In hue-app-postinstall.tpl, hue-app.prerm.tpl and other app related files kept some variables that are removed in other files, such as ROOT=/usr/lib/hue
# In a few places user name "hue" is used without defining USER=hue
# Build successful on Precise
# Package install successful
Tried a few commands. Met this problem:
(noformat)
/usr/lib/hue/build/env/bin]$ sudo ./hue test
Traceback (most recent call last):
File "./hue", line 9, in <module>
load_entry_point('desktop==2.0.1', 'console_scripts', 'hue')()
File "/usr/lib/hue/desktop/core/src/desktop/manage_entry.py", line
60, in entry
execute_manager(settings)
File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.2.3-py2.7.egg/django/core/management/__init__.py",
line 438, in execute_manager
utility.execute()
File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.2.3-py2.7.egg/django/core/management/__init__.py",
line 379, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.2.3-py2.7.egg/django/core/management/__init__.py",
line 261, in fetch_command
klass = load_command_class(app_name, subcommand)
File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.2.3-py2.7.egg/django/core/management/__init__.py",
line 67, in load_command_class
module = import_module('%s.management.commands.%s' % (app_name, name))
File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.2.3-py2.7.egg/django/utils/importlib.py",
line 35, in import_module
__import__(name)
File "/usr/lib/hue/desktop/core/src/desktop/management/commands/test.py",
line 24, in <module>
from django_nose import nose_runner
File "/usr/lib/hue/build/env/lib/python2.7/site-packages/django_nose-0.5-py2.7.egg/django_nose/nose_runner.py",
line 22, in <module>
import nose
ImportError: No module named nose
(noformat)
After installing python-django-nose:
(noformat)
sudo ./hue test
Traceback (most recent call last):
File "./hue", line 9, in <module>
load_entry_point('desktop==2.0.1', 'console_scripts', 'hue')()
File "/usr/lib/hue/desktop/core/src/desktop/manage_entry.py", line
60, in entry
execute_manager(settings)
File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.2.3-py2.7.egg/django/core/management/__init__.py",
line 438, in execute_manager
utility.execute()
File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.2.3-py2.7.egg/django/core/management/__init__.py",
line 379, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.2.3-py2.7.egg/django/core/management/__init__.py",
line 261, in fetch_command
klass = load_command_class(app_name, subcommand)
File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.2.3-py2.7.egg/django/core/management/__init__.py",
line 67, in load_command_class
module = import_module('%s.management.commands.%s' % (app_name, name))
File "/usr/lib/hue/build/env/lib/python2.7/site-packages/Django-1.2.3-py2.7.egg/django/utils/importlib.py",
line 35, in import_module
__import__(name)
File "/usr/lib/hue/desktop/core/src/desktop/management/commands/test.py",
line 32, in <module>
from desktop.management.commands import test_windmill
File "/usr/lib/hue/desktop/core/src/desktop/management/commands/test_windmill.py",
line 27, in <module>
from windmill.authoring import djangotest
ImportError: No module named windmill.authoring
(noformat)
Obviously, test_windmill also fails.
Most other commands run fine.
Off-topic question: why the build system does not detect change in the src files, only `rm -rf <component>/build`?
> improve hue packaging via making virtual env relocatable and moving DB files into /var/lib/hue
> ----------------------------------------------------------------------------------------------
>
> Key: BIGTOP-688
> URL: https://issues.apache.org/jira/browse/BIGTOP-688
> Project: Bigtop
> Issue Type: Improvement
> Components: General
> Affects Versions: 0.4.0
> Reporter: Roman Shaposhnik
> Assignee: Roman Shaposhnik
> Fix For: 0.5.0
>
> Attachments: BIGTOP-688.patch.txt
>
>
> Currently Hue packaging does unfortunate things with permissions and mutability of things under /usr/lib/hue. We should make sure our virtual env is relocatable and can be used in a R/O fashion and we should move mutable bits (desktop.db and app.reg) under /var/lib/hue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira