You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Taco Jan Osinga <ta...@lable.org> on 2013/05/22 11:21:12 UTC
Auto created 'target' folder?
Hi all,
Quite a newby here.
I'm creating an application for internal use, which deploys several demo-sites of our HBase application. These demos should contain a blank state (fixtures) with some data. Therefor I have created export files which needed to be imported (using the MapReduce-way of importing). This all works when I run my script as root. So this process works.
However, it doesn't work running the script as user tomcat7 (permission denied). Even adding tomcat7 to supergroup didn't fix the problem.
In the process I noticed there's a directory structure created, named "target/test-dir" which contains hadoop**.jar files. If I chmod -R 777 this target folder, I suddenly am able to import being tomcat7!
My question:
- What are these target folders (they appear on the working path)?
- How can I make sure the user(s) in supergroup have write permissions (is it even wise to do that)?
- It seems like some kind of temp folder: Shouldn't this folder be removed after the process is finished (it keeps growing)?
Regards from The Netherlands,
Taco Jan Osinga
Re: Auto created 'target' folder?
Posted by Chris Nauroth <cn...@hortonworks.com>.
Can you provide additional information about the exact commands that you
are trying to run?
"target/test-dir" is something that gets created during the Hadoop
codebase's Maven build process. Are you running Maven commands? If so,
are you running Maven commands as a user different from tomcat7? This
would result in a target directory owned by a different user, and the
tomcat 7 user might not have permission to access it.
If you're not actually running Maven commands, are you otherwise
referencing target/test-dir in your process, such as on a classpath? If
so, then the same problems would apply: if your process launches as a
different user from the owner of the target/test-dir directory, then you
might have a permission problem.
So far, this sounds like a local file system permission issue rather than a
Hadoop-specific issue.
Hope this helps,
Chris Nauroth
Hortonworks
http://hortonworks.com/
On Wed, May 22, 2013 at 2:21 AM, Taco Jan Osinga <ta...@lable.org> wrote:
> Hi all,
>
> Quite a newby here.
>
> I'm creating an application for internal use, which deploys several
> demo-sites of our HBase application. These demos should contain a blank
> state (fixtures) with some data. Therefor I have created export files which
> needed to be imported (using the MapReduce-way of importing). This all
> works when I run my script as root. So this process works.
>
> However, it doesn't work running the script as user tomcat7 (permission
> denied). Even adding tomcat7 to supergroup didn't fix the problem.
>
> In the process I noticed there's a directory structure created, named
> "target/test-dir" which contains hadoop**.jar files. If I chmod -R 777 this
> target folder, I suddenly am able to import being tomcat7!
>
> My question:
> - What are these target folders (they appear on the working path)?
> - How can I make sure the user(s) in supergroup have write permissions (is
> it even wise to do that)?
> - It seems like some kind of temp folder: Shouldn't this folder be removed
> after the process is finished (it keeps growing)?
>
> Regards from The Netherlands,
> Taco Jan Osinga
Re: Auto created 'target' folder?
Posted by Chris Nauroth <cn...@hortonworks.com>.
Can you provide additional information about the exact commands that you
are trying to run?
"target/test-dir" is something that gets created during the Hadoop
codebase's Maven build process. Are you running Maven commands? If so,
are you running Maven commands as a user different from tomcat7? This
would result in a target directory owned by a different user, and the
tomcat 7 user might not have permission to access it.
If you're not actually running Maven commands, are you otherwise
referencing target/test-dir in your process, such as on a classpath? If
so, then the same problems would apply: if your process launches as a
different user from the owner of the target/test-dir directory, then you
might have a permission problem.
So far, this sounds like a local file system permission issue rather than a
Hadoop-specific issue.
Hope this helps,
Chris Nauroth
Hortonworks
http://hortonworks.com/
On Wed, May 22, 2013 at 2:21 AM, Taco Jan Osinga <ta...@lable.org> wrote:
> Hi all,
>
> Quite a newby here.
>
> I'm creating an application for internal use, which deploys several
> demo-sites of our HBase application. These demos should contain a blank
> state (fixtures) with some data. Therefor I have created export files which
> needed to be imported (using the MapReduce-way of importing). This all
> works when I run my script as root. So this process works.
>
> However, it doesn't work running the script as user tomcat7 (permission
> denied). Even adding tomcat7 to supergroup didn't fix the problem.
>
> In the process I noticed there's a directory structure created, named
> "target/test-dir" which contains hadoop**.jar files. If I chmod -R 777 this
> target folder, I suddenly am able to import being tomcat7!
>
> My question:
> - What are these target folders (they appear on the working path)?
> - How can I make sure the user(s) in supergroup have write permissions (is
> it even wise to do that)?
> - It seems like some kind of temp folder: Shouldn't this folder be removed
> after the process is finished (it keeps growing)?
>
> Regards from The Netherlands,
> Taco Jan Osinga
Re: Auto created 'target' folder?
Posted by Chris Nauroth <cn...@hortonworks.com>.
Can you provide additional information about the exact commands that you
are trying to run?
"target/test-dir" is something that gets created during the Hadoop
codebase's Maven build process. Are you running Maven commands? If so,
are you running Maven commands as a user different from tomcat7? This
would result in a target directory owned by a different user, and the
tomcat 7 user might not have permission to access it.
If you're not actually running Maven commands, are you otherwise
referencing target/test-dir in your process, such as on a classpath? If
so, then the same problems would apply: if your process launches as a
different user from the owner of the target/test-dir directory, then you
might have a permission problem.
So far, this sounds like a local file system permission issue rather than a
Hadoop-specific issue.
Hope this helps,
Chris Nauroth
Hortonworks
http://hortonworks.com/
On Wed, May 22, 2013 at 2:21 AM, Taco Jan Osinga <ta...@lable.org> wrote:
> Hi all,
>
> Quite a newby here.
>
> I'm creating an application for internal use, which deploys several
> demo-sites of our HBase application. These demos should contain a blank
> state (fixtures) with some data. Therefor I have created export files which
> needed to be imported (using the MapReduce-way of importing). This all
> works when I run my script as root. So this process works.
>
> However, it doesn't work running the script as user tomcat7 (permission
> denied). Even adding tomcat7 to supergroup didn't fix the problem.
>
> In the process I noticed there's a directory structure created, named
> "target/test-dir" which contains hadoop**.jar files. If I chmod -R 777 this
> target folder, I suddenly am able to import being tomcat7!
>
> My question:
> - What are these target folders (they appear on the working path)?
> - How can I make sure the user(s) in supergroup have write permissions (is
> it even wise to do that)?
> - It seems like some kind of temp folder: Shouldn't this folder be removed
> after the process is finished (it keeps growing)?
>
> Regards from The Netherlands,
> Taco Jan Osinga
Re: Auto created 'target' folder?
Posted by Chris Nauroth <cn...@hortonworks.com>.
Can you provide additional information about the exact commands that you
are trying to run?
"target/test-dir" is something that gets created during the Hadoop
codebase's Maven build process. Are you running Maven commands? If so,
are you running Maven commands as a user different from tomcat7? This
would result in a target directory owned by a different user, and the
tomcat 7 user might not have permission to access it.
If you're not actually running Maven commands, are you otherwise
referencing target/test-dir in your process, such as on a classpath? If
so, then the same problems would apply: if your process launches as a
different user from the owner of the target/test-dir directory, then you
might have a permission problem.
So far, this sounds like a local file system permission issue rather than a
Hadoop-specific issue.
Hope this helps,
Chris Nauroth
Hortonworks
http://hortonworks.com/
On Wed, May 22, 2013 at 2:21 AM, Taco Jan Osinga <ta...@lable.org> wrote:
> Hi all,
>
> Quite a newby here.
>
> I'm creating an application for internal use, which deploys several
> demo-sites of our HBase application. These demos should contain a blank
> state (fixtures) with some data. Therefor I have created export files which
> needed to be imported (using the MapReduce-way of importing). This all
> works when I run my script as root. So this process works.
>
> However, it doesn't work running the script as user tomcat7 (permission
> denied). Even adding tomcat7 to supergroup didn't fix the problem.
>
> In the process I noticed there's a directory structure created, named
> "target/test-dir" which contains hadoop**.jar files. If I chmod -R 777 this
> target folder, I suddenly am able to import being tomcat7!
>
> My question:
> - What are these target folders (they appear on the working path)?
> - How can I make sure the user(s) in supergroup have write permissions (is
> it even wise to do that)?
> - It seems like some kind of temp folder: Shouldn't this folder be removed
> after the process is finished (it keeps growing)?
>
> Regards from The Netherlands,
> Taco Jan Osinga