You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@sqoop.apache.org by Artem Ervits <ar...@nyp.org> on 2012/11/28 16:25:15 UTC

password field in job configuration

Hello all,

I am not sure whether this is correct mailing list but I'm seeing this issue happening where when I review the configuration of a sqoop import job.xml, I can see my password in plain text under "mapreduce.jdbc.url". Is there a way to mask this field, again, if this needs to be directed to Mapreduce mailing list, I'll be happy to do so, but if anyone knows the answer, it will be much appreciated.

Thank you.

Artem Ervits
Data Analyst
New York Presbyterian Hospital

--------------------

This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited. If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message. Thank you.

--------------------

Re: Sqoop 2 availability (Was Re: password field in job configuration)

Posted by Chalcy <ch...@gmail.com>.

Great! Jarec.  You guys are awesome.  I will stay tuned to get sqoop
ungrades. Looks like we have to develop our own tool/scripts for now :)

Thanks,
Chalcy

On Wed, Nov 28, 2012 at 11:18 AM, Jarek Jarcec Cecho <ja...@apache.org>wrote:

> Hi Chalcy,
> I'm glad that you've asked. You've raised a very good question - I would
> personally like to do "first cut" of Sqoop 2 this year to give our users
> first testable bits.
>
> However having said that I need to point out that Sqoop 2 is still work in
> progress and that this "first cut" will be far from completion. For example
> with Artem's concern - we're definitely planning to move out all sensitive
> information from the mapreduce job configuration object, but this first cut
> will most likely do exactly this. Similarly for Web UI - it's definitely on
> our plan, but it won't be included in the first cut.
>
> Does that answer your question?
>
> Jarcec
>
> On Wed, Nov 28, 2012 at 11:04:58AM -0500, Chalcy wrote:
> > Hi Jarec,
> >
> > question about sqoop2.  When are you expecting it to be out?  I hear
> sqoop2
> > will have UI to do data imports as well?
> >
> > Thanks,
> > Chalcy
> >
> > On Wed, Nov 28, 2012 at 10:42 AM, Jarek Jarcec Cecho <jarcec@apache.org
> >wrote:
> >
> > > Hi Artem,
> > > don't worry, I believe that this mailing is the best place to ask your
> > > question.
> > >
> > > Unfortunately Sqoop 1 is always storing db credentials into mapreduce
> job
> > > configuration object and you can always read them from there. I'm
> afraid
> > > that there isn't way how to overcome this in Sqoop 1. Luckily we've
> noticed
> > > that this a big issue for a lot of our users, so one of our primary
> goals
> > > when designing Sqoop 2 was to allow not to put sensitive information
> into
> > > the job object, so stay tuned for Sqoop 2!
> > >
> > > Jarcec
> > >
> > > On Wed, Nov 28, 2012 at 03:25:15PM +0000, Artem Ervits wrote:
> > > > Hello all,
> > > >
> > > > I am not sure whether this is correct mailing list but I'm seeing
> this
> > > issue happening where when I review the configuration of a sqoop import
> > > job.xml, I can see my password in plain text under
> "mapreduce.jdbc.url". Is
> > > there a way to mask this field, again, if this needs to be directed to
> > > Mapreduce mailing list, I'll be happy to do so, but if anyone knows the
> > > answer, it will be much appreciated.
> > > >
> > > > Thank you.
> > > >
> > > >
> > > > Artem Ervits
> > > > Data Analyst
> > > > New York Presbyterian Hospital
> > > >
> > > >
> > > >
> > > > --------------------
> > > >
> > > > This electronic message is intended to be for the use only of the
> named
> > > recipient, and may contain information that is confidential or
> privileged.
> > >  If you are not the intended recipient, you are hereby notified that
> any
> > > disclosure, copying, distribution or use of the contents of this
> message is
> > > strictly prohibited.  If you have received this message in error or
> are not
> > > the named recipient, please notify us immediately by contacting the
> sender
> > > at the electronic mail address noted above, and delete and destroy all
> > > copies of this message.  Thank you.
> > > >
> > > >
> > > >
> > > >
> > > > --------------------
> > > >
> > > > This electronic message is intended to be for the use only of the
> named
> > > recipient, and may contain information that is confidential or
> privileged.
> > >  If you are not the intended recipient, you are hereby notified that
> any
> > > disclosure, copying, distribution or use of the contents of this
> message is
> > > strictly prohibited.  If you have received this message in error or
> are not
> > > the named recipient, please notify us immediately by contacting the
> sender
> > > at the electronic mail address noted above, and delete and destroy all
> > > copies of this message.  Thank you.
> > > >
> > > >
> > > >
> > >
>

Sqoop 2 availability (Was Re: password field in job configuration)

Posted by Jarek Jarcec Cecho <ja...@apache.org>.

Hi Chalcy,
I'm glad that you've asked. You've raised a very good question - I would personally like to do "first cut" of Sqoop 2 this year to give our users first testable bits.

However having said that I need to point out that Sqoop 2 is still work in progress and that this "first cut" will be far from completion. For example with Artem's concern - we're definitely planning to move out all sensitive information from the mapreduce job configuration object, but this first cut will most likely do exactly this. Similarly for Web UI - it's definitely on our plan, but it won't be included in the first cut.

Does that answer your question?

Jarcec

On Wed, Nov 28, 2012 at 11:04:58AM -0500, Chalcy wrote:
> Hi Jarec,
> 
> question about sqoop2.  When are you expecting it to be out?  I hear sqoop2
> will have UI to do data imports as well?
> 
> Thanks,
> Chalcy
> 
> On Wed, Nov 28, 2012 at 10:42 AM, Jarek Jarcec Cecho <ja...@apache.org>wrote:
> 
> > Hi Artem,
> > don't worry, I believe that this mailing is the best place to ask your
> > question.
> >
> > Unfortunately Sqoop 1 is always storing db credentials into mapreduce job
> > configuration object and you can always read them from there. I'm afraid
> > that there isn't way how to overcome this in Sqoop 1. Luckily we've noticed
> > that this a big issue for a lot of our users, so one of our primary goals
> > when designing Sqoop 2 was to allow not to put sensitive information into
> > the job object, so stay tuned for Sqoop 2!
> >
> > Jarcec
> >
> > On Wed, Nov 28, 2012 at 03:25:15PM +0000, Artem Ervits wrote:
> > > Hello all,
> > >
> > > I am not sure whether this is correct mailing list but I'm seeing this
> > issue happening where when I review the configuration of a sqoop import
> > job.xml, I can see my password in plain text under "mapreduce.jdbc.url". Is
> > there a way to mask this field, again, if this needs to be directed to
> > Mapreduce mailing list, I'll be happy to do so, but if anyone knows the
> > answer, it will be much appreciated.
> > >
> > > Thank you.
> > >
> > >
> > > Artem Ervits
> > > Data Analyst
> > > New York Presbyterian Hospital
> > >
> > >
> > >
> > > --------------------
> > >
> > > This electronic message is intended to be for the use only of the named
> > recipient, and may contain information that is confidential or privileged.
> >  If you are not the intended recipient, you are hereby notified that any
> > disclosure, copying, distribution or use of the contents of this message is
> > strictly prohibited.  If you have received this message in error or are not
> > the named recipient, please notify us immediately by contacting the sender
> > at the electronic mail address noted above, and delete and destroy all
> > copies of this message.  Thank you.
> > >
> > >
> > >
> > >
> > > --------------------
> > >
> > > This electronic message is intended to be for the use only of the named
> > recipient, and may contain information that is confidential or privileged.
> >  If you are not the intended recipient, you are hereby notified that any
> > disclosure, copying, distribution or use of the contents of this message is
> > strictly prohibited.  If you have received this message in error or are not
> > the named recipient, please notify us immediately by contacting the sender
> > at the electronic mail address noted above, and delete and destroy all
> > copies of this message.  Thank you.
> > >
> > >
> > >
> >

Re: password field in job configuration

Posted by Chalcy <ch...@gmail.com>.

Hi Jarec,

question about sqoop2.  When are you expecting it to be out?  I hear sqoop2
will have UI to do data imports as well?

Thanks,
Chalcy

On Wed, Nov 28, 2012 at 10:42 AM, Jarek Jarcec Cecho <ja...@apache.org>wrote:

> Hi Artem,
> don't worry, I believe that this mailing is the best place to ask your
> question.
>
> Unfortunately Sqoop 1 is always storing db credentials into mapreduce job
> configuration object and you can always read them from there. I'm afraid
> that there isn't way how to overcome this in Sqoop 1. Luckily we've noticed
> that this a big issue for a lot of our users, so one of our primary goals
> when designing Sqoop 2 was to allow not to put sensitive information into
> the job object, so stay tuned for Sqoop 2!
>
> Jarcec
>
> On Wed, Nov 28, 2012 at 03:25:15PM +0000, Artem Ervits wrote:
> > Hello all,
> >
> > I am not sure whether this is correct mailing list but I'm seeing this
> issue happening where when I review the configuration of a sqoop import
> job.xml, I can see my password in plain text under "mapreduce.jdbc.url". Is
> there a way to mask this field, again, if this needs to be directed to
> Mapreduce mailing list, I'll be happy to do so, but if anyone knows the
> answer, it will be much appreciated.
> >
> > Thank you.
> >
> >
> > Artem Ervits
> > Data Analyst
> > New York Presbyterian Hospital
> >
> >
> >
> > --------------------
> >
> > This electronic message is intended to be for the use only of the named
> recipient, and may contain information that is confidential or privileged.
>  If you are not the intended recipient, you are hereby notified that any
> disclosure, copying, distribution or use of the contents of this message is
> strictly prohibited.  If you have received this message in error or are not
> the named recipient, please notify us immediately by contacting the sender
> at the electronic mail address noted above, and delete and destroy all
> copies of this message.  Thank you.
> >
> >
> >
> >
> > --------------------
> >
> > This electronic message is intended to be for the use only of the named
> recipient, and may contain information that is confidential or privileged.
>  If you are not the intended recipient, you are hereby notified that any
> disclosure, copying, distribution or use of the contents of this message is
> strictly prohibited.  If you have received this message in error or are not
> the named recipient, please notify us immediately by contacting the sender
> at the electronic mail address noted above, and delete and destroy all
> copies of this message.  Thank you.
> >
> >
> >
>

RE: password field in job configuration

Posted by Artem Ervits <ar...@nyp.org>.

Thank you!

-----Original Message-----
From: Jarek Jarcec Cecho [mailto:jarcec@apache.org] 
Sent: Wednesday, November 28, 2012 10:43 AM
To: user@sqoop.apache.org
Subject: Re: password field in job configuration

Hi Artem,
don't worry, I believe that this mailing is the best place to ask your question.

Unfortunately Sqoop 1 is always storing db credentials into mapreduce job configuration object and you can always read them from there. I'm afraid that there isn't way how to overcome this in Sqoop 1. Luckily we've noticed that this a big issue for a lot of our users, so one of our primary goals when designing Sqoop 2 was to allow not to put sensitive information into the job object, so stay tuned for Sqoop 2!

Jarcec

On Wed, Nov 28, 2012 at 03:25:15PM +0000, Artem Ervits wrote:
> Hello all,
> 
> I am not sure whether this is correct mailing list but I'm seeing this issue happening where when I review the configuration of a sqoop import job.xml, I can see my password in plain text under "mapreduce.jdbc.url". Is there a way to mask this field, again, if this needs to be directed to Mapreduce mailing list, I'll be happy to do so, but if anyone knows the answer, it will be much appreciated.
> 
> Thank you.
> 
> 
> Artem Ervits
> Data Analyst
> New York Presbyterian Hospital
> 
> 
> 
> --------------------
> 
> This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged.  If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited.  If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message.  Thank you.
> 
> 
> 
> 
> --------------------
> 
> This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged.  If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited.  If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message.  Thank you.
> 
> 
> 

--------------------

This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged.  If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited.  If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message.  Thank you.

--------------------

This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged.  If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited.  If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message.  Thank you.

Re: password field in job configuration

Posted by Jarek Jarcec Cecho <ja...@apache.org>.

Hi Artem,
don't worry, I believe that this mailing is the best place to ask your question.

Unfortunately Sqoop 1 is always storing db credentials into mapreduce job configuration object and you can always read them from there. I'm afraid that there isn't way how to overcome this in Sqoop 1. Luckily we've noticed that this a big issue for a lot of our users, so one of our primary goals when designing Sqoop 2 was to allow not to put sensitive information into the job object, so stay tuned for Sqoop 2!

Jarcec

On Wed, Nov 28, 2012 at 03:25:15PM +0000, Artem Ervits wrote:
> Hello all,
> 
> I am not sure whether this is correct mailing list but I'm seeing this issue happening where when I review the configuration of a sqoop import job.xml, I can see my password in plain text under "mapreduce.jdbc.url". Is there a way to mask this field, again, if this needs to be directed to Mapreduce mailing list, I'll be happy to do so, but if anyone knows the answer, it will be much appreciated.
> 
> Thank you.
> 
> 
> Artem Ervits
> Data Analyst
> New York Presbyterian Hospital
> 
> 
> 
> --------------------
> 
> This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged.  If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited.  If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message.  Thank you.
> 
> 
> 
> 
> --------------------
> 
> This electronic message is intended to be for the use only of the named recipient, and may contain information that is confidential or privileged.  If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution or use of the contents of this message is strictly prohibited.  If you have received this message in error or are not the named recipient, please notify us immediately by contacting the sender at the electronic mail address noted above, and delete and destroy all copies of this message.  Thank you.
> 
> 
>