You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mahout.apache.org by "Andrew Bayer (JIRA)" <ji...@apache.org> on 2011/09/14 17:42:09 UTC
[jira] [Created] (MAHOUT-811) Mahout examples try to write to
examples/bin/work, which may not be writeable by current user
Mahout examples try to write to examples/bin/work, which may not be writeable by current user
---------------------------------------------------------------------------------------------
Key: MAHOUT-811
URL: https://issues.apache.org/jira/browse/MAHOUT-811
Project: Mahout
Issue Type: Bug
Components: Examples
Affects Versions: 0.5
Reporter: Andrew Bayer
Attachments: MAHOUT-811.diff.txt
The examples in examples/bin create subdirectories (either work or mahout-work) in that directory and write to those subdirectories. This works fine if the current user has write access to examples/bin, but if not (such as in the case of the package generated by Bigtop, in which the files are installed to /usr/lib/mahout and owned by root), the examples can't run. This is causing BIGTOP-96, but needs to be fixed in Mahout. The patch I'm attaching changes all the references to work, examples/bin/work, and mahout-work to instead use /tmp/mahout-work-${USER}, which will be writeable.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (MAHOUT-811) Mahout examples try to write to
examples/bin/work, which may not be writeable by current user
Posted by "Drew Farris (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Drew Farris reopened MAHOUT-811:
--------------------------------
Assignee: Drew Farris (was: Sean Owen)
This patch introduces another problem, specifically with the following line:
{code}
cd ${WORK_DIR}/reuters-sgm && tar xzf ../reuters21578.tar.gz && cd .. && cd ..
{code}
Here the script is making assumptions as to where the ${WORK_DIR} is located, and cd-ing relative to that to get to the mahout exampless bin directory in order to later execute ../../bin/mahout.
As a result, the script will fail any time it needs to download and untar the reuters data.
One fix would be to make the script a little smarter about where the mahout driver script is located by taking advantage of ${SCRIPT_PATH}
Also, it would be nice if the work dir could be read from an environment variable. If the env variable is not set, it could be set to /tmp/mahout-work-${user}.
> Mahout examples try to write to examples/bin/work, which may not be writeable by current user
> ---------------------------------------------------------------------------------------------
>
> Key: MAHOUT-811
> URL: https://issues.apache.org/jira/browse/MAHOUT-811
> Project: Mahout
> Issue Type: Bug
> Components: Examples
> Affects Versions: 0.5
> Reporter: Andrew Bayer
> Assignee: Drew Farris
> Fix For: 0.6
>
> Attachments: MAHOUT-811.diff.txt
>
>
> The examples in examples/bin create subdirectories (either work or mahout-work) in that directory and write to those subdirectories. This works fine if the current user has write access to examples/bin, but if not (such as in the case of the package generated by Bigtop, in which the files are installed to /usr/lib/mahout and owned by root), the examples can't run. This is causing BIGTOP-96, but needs to be fixed in Mahout. The patch I'm attaching changes all the references to work, examples/bin/work, and mahout-work to instead use /tmp/mahout-work-${USER}, which will be writeable.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAHOUT-811) Mahout examples try to write to
examples/bin/work, which may not be writeable by current user
Posted by "Andrew Bayer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Bayer updated MAHOUT-811:
--------------------------------
Attachment: MAHOUT-811.diff.txt
> Mahout examples try to write to examples/bin/work, which may not be writeable by current user
> ---------------------------------------------------------------------------------------------
>
> Key: MAHOUT-811
> URL: https://issues.apache.org/jira/browse/MAHOUT-811
> Project: Mahout
> Issue Type: Bug
> Components: Examples
> Affects Versions: 0.5
> Reporter: Andrew Bayer
> Attachments: MAHOUT-811.diff.txt
>
>
> The examples in examples/bin create subdirectories (either work or mahout-work) in that directory and write to those subdirectories. This works fine if the current user has write access to examples/bin, but if not (such as in the case of the package generated by Bigtop, in which the files are installed to /usr/lib/mahout and owned by root), the examples can't run. This is causing BIGTOP-96, but needs to be fixed in Mahout. The patch I'm attaching changes all the references to work, examples/bin/work, and mahout-work to instead use /tmp/mahout-work-${USER}, which will be writeable.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAHOUT-811) Mahout examples try to write to
examples/bin/work, which may not be writeable by current user
Posted by "Drew Farris (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13106617#comment-13106617 ]
Drew Farris commented on MAHOUT-811:
------------------------------------
{quote}
Should be easy enough to do this without any cd-ing anywhere:
tar xzf ${WORK_DIR}/reuters21578.tar.gz -C ${WORK_DIR}/reuters-sgm
I'll patch it.
{quote}
Great Sean, thanks.
This script also does rm -rf ${WORK_DIR} when it's done and I don't think it should. It is helpful to leave the work directories around so that you don't have to re-download the reuters tarfile and re-vectorize it should you want to run lda after trying kmeans. It's also helpful also to leave these files around for those curious to inspect the output files.
This behavior wasn't introduced in the patch, but I just noticed it while testing.
> Mahout examples try to write to examples/bin/work, which may not be writeable by current user
> ---------------------------------------------------------------------------------------------
>
> Key: MAHOUT-811
> URL: https://issues.apache.org/jira/browse/MAHOUT-811
> Project: Mahout
> Issue Type: Bug
> Components: Examples
> Affects Versions: 0.5
> Reporter: Andrew Bayer
> Assignee: Drew Farris
> Fix For: 0.6
>
> Attachments: MAHOUT-811.diff.txt
>
>
> The examples in examples/bin create subdirectories (either work or mahout-work) in that directory and write to those subdirectories. This works fine if the current user has write access to examples/bin, but if not (such as in the case of the package generated by Bigtop, in which the files are installed to /usr/lib/mahout and owned by root), the examples can't run. This is causing BIGTOP-96, but needs to be fixed in Mahout. The patch I'm attaching changes all the references to work, examples/bin/work, and mahout-work to instead use /tmp/mahout-work-${USER}, which will be writeable.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAHOUT-811) Mahout examples try to write to
examples/bin/work, which may not be writeable by current user
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13104803#comment-13104803 ]
Hudson commented on MAHOUT-811:
-------------------------------
Integrated in Mahout-Quality #1038 (See [https://builds.apache.org/job/Mahout-Quality/1038/])
MAHOUT-811 move work dir to /tmp
srowen : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1170702
Files :
* /mahout/trunk/examples/bin/build-20news-bayes.sh
* /mahout/trunk/examples/bin/build-cluster-syntheticcontrol.sh
* /mahout/trunk/examples/bin/build-reuters.sh
* /mahout/trunk/examples/bin/factorize-movielens-1M.sh
> Mahout examples try to write to examples/bin/work, which may not be writeable by current user
> ---------------------------------------------------------------------------------------------
>
> Key: MAHOUT-811
> URL: https://issues.apache.org/jira/browse/MAHOUT-811
> Project: Mahout
> Issue Type: Bug
> Components: Examples
> Affects Versions: 0.5
> Reporter: Andrew Bayer
> Assignee: Sean Owen
> Fix For: 0.6
>
> Attachments: MAHOUT-811.diff.txt
>
>
> The examples in examples/bin create subdirectories (either work or mahout-work) in that directory and write to those subdirectories. This works fine if the current user has write access to examples/bin, but if not (such as in the case of the package generated by Bigtop, in which the files are installed to /usr/lib/mahout and owned by root), the examples can't run. This is causing BIGTOP-96, but needs to be fixed in Mahout. The patch I'm attaching changes all the references to work, examples/bin/work, and mahout-work to instead use /tmp/mahout-work-${USER}, which will be writeable.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAHOUT-811) Mahout examples try to write to
examples/bin/work, which may not be writeable by current user
Posted by "Andrew Bayer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13106637#comment-13106637 ]
Andrew Bayer commented on MAHOUT-811:
-------------------------------------
Yeah, I kept the rm -rf for consistency, but changed my mind after submitting the patch. =)
> Mahout examples try to write to examples/bin/work, which may not be writeable by current user
> ---------------------------------------------------------------------------------------------
>
> Key: MAHOUT-811
> URL: https://issues.apache.org/jira/browse/MAHOUT-811
> Project: Mahout
> Issue Type: Bug
> Components: Examples
> Affects Versions: 0.5
> Reporter: Andrew Bayer
> Assignee: Drew Farris
> Fix For: 0.6
>
> Attachments: MAHOUT-811.diff.txt
>
>
> The examples in examples/bin create subdirectories (either work or mahout-work) in that directory and write to those subdirectories. This works fine if the current user has write access to examples/bin, but if not (such as in the case of the package generated by Bigtop, in which the files are installed to /usr/lib/mahout and owned by root), the examples can't run. This is causing BIGTOP-96, but needs to be fixed in Mahout. The patch I'm attaching changes all the references to work, examples/bin/work, and mahout-work to instead use /tmp/mahout-work-${USER}, which will be writeable.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAHOUT-811) Mahout examples try to write to
examples/bin/work, which may not be writeable by current user
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13106811#comment-13106811 ]
Hudson commented on MAHOUT-811:
-------------------------------
Integrated in Mahout-Quality #1043 (See [https://builds.apache.org/job/Mahout-Quality/1043/])
MAHOUT-811 Don't delete work dir
srowen : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1171706
Files :
* /mahout/trunk/examples/bin/build-reuters.sh
> Mahout examples try to write to examples/bin/work, which may not be writeable by current user
> ---------------------------------------------------------------------------------------------
>
> Key: MAHOUT-811
> URL: https://issues.apache.org/jira/browse/MAHOUT-811
> Project: Mahout
> Issue Type: Bug
> Components: Examples
> Affects Versions: 0.5
> Reporter: Andrew Bayer
> Assignee: Drew Farris
> Fix For: 0.6
>
> Attachments: MAHOUT-811.diff.txt
>
>
> The examples in examples/bin create subdirectories (either work or mahout-work) in that directory and write to those subdirectories. This works fine if the current user has write access to examples/bin, but if not (such as in the case of the package generated by Bigtop, in which the files are installed to /usr/lib/mahout and owned by root), the examples can't run. This is causing BIGTOP-96, but needs to be fixed in Mahout. The patch I'm attaching changes all the references to work, examples/bin/work, and mahout-work to instead use /tmp/mahout-work-${USER}, which will be writeable.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAHOUT-811) Mahout examples try to write to
examples/bin/work, which may not be writeable by current user
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13106661#comment-13106661 ]
Hudson commented on MAHOUT-811:
-------------------------------
Integrated in Mahout-Quality #1042 (See [https://builds.apache.org/job/Mahout-Quality/1042/])
MAHOUT-811 fix working directory issue with WORK_DIR and extracting archive
srowen : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1171636
Files :
* /mahout/trunk/examples/bin/build-reuters.sh
> Mahout examples try to write to examples/bin/work, which may not be writeable by current user
> ---------------------------------------------------------------------------------------------
>
> Key: MAHOUT-811
> URL: https://issues.apache.org/jira/browse/MAHOUT-811
> Project: Mahout
> Issue Type: Bug
> Components: Examples
> Affects Versions: 0.5
> Reporter: Andrew Bayer
> Assignee: Drew Farris
> Fix For: 0.6
>
> Attachments: MAHOUT-811.diff.txt
>
>
> The examples in examples/bin create subdirectories (either work or mahout-work) in that directory and write to those subdirectories. This works fine if the current user has write access to examples/bin, but if not (such as in the case of the package generated by Bigtop, in which the files are installed to /usr/lib/mahout and owned by root), the examples can't run. This is causing BIGTOP-96, but needs to be fixed in Mahout. The patch I'm attaching changes all the references to work, examples/bin/work, and mahout-work to instead use /tmp/mahout-work-${USER}, which will be writeable.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAHOUT-811) Mahout examples try to write to
examples/bin/work, which may not be writeable by current user
Posted by "Sean Owen (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen resolved MAHOUT-811.
------------------------------
Resolution: Fixed
> Mahout examples try to write to examples/bin/work, which may not be writeable by current user
> ---------------------------------------------------------------------------------------------
>
> Key: MAHOUT-811
> URL: https://issues.apache.org/jira/browse/MAHOUT-811
> Project: Mahout
> Issue Type: Bug
> Components: Examples
> Affects Versions: 0.5
> Reporter: Andrew Bayer
> Assignee: Drew Farris
> Fix For: 0.6
>
> Attachments: MAHOUT-811.diff.txt
>
>
> The examples in examples/bin create subdirectories (either work or mahout-work) in that directory and write to those subdirectories. This works fine if the current user has write access to examples/bin, but if not (such as in the case of the package generated by Bigtop, in which the files are installed to /usr/lib/mahout and owned by root), the examples can't run. This is causing BIGTOP-96, but needs to be fixed in Mahout. The patch I'm attaching changes all the references to work, examples/bin/work, and mahout-work to instead use /tmp/mahout-work-${USER}, which will be writeable.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAHOUT-811) Mahout examples try to write to
examples/bin/work, which may not be writeable by current user
Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean Owen resolved MAHOUT-811.
------------------------------
Resolution: Fixed
Fix Version/s: 0.6
Assignee: Sean Owen
Sounds great to me, thanks. At the least this centralizes the definition of 'work' dir, and I see no reason it can't live in temp. In fact it sounds like it should.
> Mahout examples try to write to examples/bin/work, which may not be writeable by current user
> ---------------------------------------------------------------------------------------------
>
> Key: MAHOUT-811
> URL: https://issues.apache.org/jira/browse/MAHOUT-811
> Project: Mahout
> Issue Type: Bug
> Components: Examples
> Affects Versions: 0.5
> Reporter: Andrew Bayer
> Assignee: Sean Owen
> Fix For: 0.6
>
> Attachments: MAHOUT-811.diff.txt
>
>
> The examples in examples/bin create subdirectories (either work or mahout-work) in that directory and write to those subdirectories. This works fine if the current user has write access to examples/bin, but if not (such as in the case of the package generated by Bigtop, in which the files are installed to /usr/lib/mahout and owned by root), the examples can't run. This is causing BIGTOP-96, but needs to be fixed in Mahout. The patch I'm attaching changes all the references to work, examples/bin/work, and mahout-work to instead use /tmp/mahout-work-${USER}, which will be writeable.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAHOUT-811) Mahout examples try to write to
examples/bin/work, which may not be writeable by current user
Posted by "Sean Owen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAHOUT-811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13106535#comment-13106535 ]
Sean Owen commented on MAHOUT-811:
----------------------------------
Should be easy enough to do this without any cd-ing anywhere:
tar xzf ${WORK_DIR}/reuters21578.tar.gz -C ${WORK_DIR}/reuters-sgm
I'll patch it.
> Mahout examples try to write to examples/bin/work, which may not be writeable by current user
> ---------------------------------------------------------------------------------------------
>
> Key: MAHOUT-811
> URL: https://issues.apache.org/jira/browse/MAHOUT-811
> Project: Mahout
> Issue Type: Bug
> Components: Examples
> Affects Versions: 0.5
> Reporter: Andrew Bayer
> Assignee: Drew Farris
> Fix For: 0.6
>
> Attachments: MAHOUT-811.diff.txt
>
>
> The examples in examples/bin create subdirectories (either work or mahout-work) in that directory and write to those subdirectories. This works fine if the current user has write access to examples/bin, but if not (such as in the case of the package generated by Bigtop, in which the files are installed to /usr/lib/mahout and owned by root), the examples can't run. This is causing BIGTOP-96, but needs to be fixed in Mahout. The patch I'm attaching changes all the references to work, examples/bin/work, and mahout-work to instead use /tmp/mahout-work-${USER}, which will be writeable.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira