You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@sqoop.apache.org by Attila Szabó <ma...@apache.org> on 2018/02/14 11:14:20 UTC
Question: Third party docker support file version
Hi all,
A few months ago Szabi created start and stop scripts for docker-compose,
to run 3rd party DBs for the 3rd party tests, which is great ( it's
intention is to make the integration testing of Sqoop much easier ).
Though the version of docker-compose file is 3+, thus it's only usable with
the quite recent docker-compose+docker versions.
And here comes the problem:
The latest version of CentOS (7.4) by default comes a much older
Docker+Docker-compose version, which is incompatible with the compose file
version. So out of the box, this solution won't work on the latest CentOS
version for example (I've discovered it b/c one of my test servers runs
CentOS7.4).
I'm not telling it's impossible to install it from the net (first docker
and then docker compose), but it's just not the standard/convenient/stable
solution which would come from the RPM package.
Here comes my question:
What do you think: is it possible, and would it make sense to rewrite the
docker compose file to version 2, and thus providing an EZ way to use this
feature by the majority of the Linux users? Is there any vital feature in
version 3 what is required for this task?
Keeping it simple:
Would it be possible to provide a much wider usable version of this docker
based testing solution, or would we like to keep and handle it as an
experimental testing feature?
Thanks,
Attila
Re: Question: Third party docker support file version
Posted by Szabó Attila <ma...@inf.elte.hu>.
Hey Szabi,
Thanks for the quick reply!
I'll check SQOOP-3231, and ask around the legal issues, and I'll also ask my docker expert friends, who has already packed together proprietary software, and released publicly.
Will provide my help as soon as I have my results.
Thanks,
Attila
________________________________
From: Szabolcs Vasas <va...@apache.org>
Sent: Thursday, February 15, 2018 1:34 PM
To: dev@sqoop.apache.org
Cc: Attila Szabó
Subject: Re: Question: Third party docker support file version
Hi Attila,
The healthcheck tag is not supported in compose file version 2 but now I
can see that it is supported in 2.1. I am not sure if the default docker
compose installation on CentOS 7.4 works with version 2.1.
The Oracle image was never available on Dockerhub (at least not when I was
looking for it) and I think there might be some legal barriers to make it
publicly available. I was not sure about this so I made the safe choice.
I definitely agree that it would be great if we could make the third party
testing process a smoother experience, for example by making the Oracle
image and the JDBC drivers available out of the box but we need to make
sure that we consider all the legal aspects before it. SQOOP-3231
<https://issues.apache.org/jira/browse/SQOOP-3231> partially covers this,
[SQOOP-3231] Execute Sqoop third party tests in the CI ...<https://issues.apache.org/jira/browse/SQOOP-3231>
issues.apache.org
The task is to enable the third party test execution in the Sqoop CI system using the docker support implemented earlier. I am not sure this is feasible since the ...
it is more about executing the third party tests in a CI job upstream, but
eventually it involves answering these questions.
Your contribution is more than welcome on this front, if you have
improvement ideas feel free to submit a patch.
Szabolcs
On Wed, Feb 14, 2018 at 4:12 PM, Attila Szabó <ma...@apache.org> wrote:
> Hi,
>
> I've also found another strange thing:
> It seems oracle/database:12.2.0.1-ee is no longer available on dockerhub,
> thus the usage of these scripts are very difficult at the beginning, and
> needs lots of workarounds (of course it's feasible to clone github, build
> the oracle image, etc., but it's very far from the original goal) to make
> it work (nonetheless it's very much not working out of the box).
>
> Do we plan to do anything with this? (e.g. creating an official Sqoop id
> for dockerhub, and push their the prebuilt image or so). If yes I'd more
> than happy to provide my help, guidance on this front.
>
> Cheers,
> Attila
>
> <http://www.avg.com/email-signature?utm_medium=email&
> utm_source=link&utm_campaign=sig-email&utm_content=webmail>
> Virus-free.
> www.avg.com<http://www.avg.com>
> <http://www.avg.com/email-signature?utm_medium=email&
> utm_source=link&utm_campaign=sig-email&utm_content=webmail>
> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
>
> On Wed, Feb 14, 2018 at 12:14 PM, Attila Szabó <ma...@apache.org> wrote:
>
> > Hi all,
> >
> > A few months ago Szabi created start and stop scripts for docker-compose,
> > to run 3rd party DBs for the 3rd party tests, which is great ( it's
> > intention is to make the integration testing of Sqoop much easier ).
> >
> > Though the version of docker-compose file is 3+, thus it's only usable
> > with the quite recent docker-compose+docker versions.
> >
> > And here comes the problem:
> > The latest version of CentOS (7.4) by default comes a much older
> > Docker+Docker-compose version, which is incompatible with the compose
> file
> > version. So out of the box, this solution won't work on the latest CentOS
> > version for example (I've discovered it b/c one of my test servers runs
> > CentOS7.4).
> >
> > I'm not telling it's impossible to install it from the net (first docker
> > and then docker compose), but it's just not the
> standard/convenient/stable
> > solution which would come from the RPM package.
> >
> > Here comes my question:
> > What do you think: is it possible, and would it make sense to rewrite the
> > docker compose file to version 2, and thus providing an EZ way to use
> this
> > feature by the majority of the Linux users? Is there any vital feature in
> > version 3 what is required for this task?
> >
> > Keeping it simple:
> > Would it be possible to provide a much wider usable version of this
> docker
> > based testing solution, or would we like to keep and handle it as an
> > experimental testing feature?
> >
> > Thanks,
> > Attila
> >
>
Re: Question: Third party docker support file version
Posted by Szabolcs Vasas <va...@apache.org>.
Hi Attila,
The healthcheck tag is not supported in compose file version 2 but now I
can see that it is supported in 2.1. I am not sure if the default docker
compose installation on CentOS 7.4 works with version 2.1.
The Oracle image was never available on Dockerhub (at least not when I was
looking for it) and I think there might be some legal barriers to make it
publicly available. I was not sure about this so I made the safe choice.
I definitely agree that it would be great if we could make the third party
testing process a smoother experience, for example by making the Oracle
image and the JDBC drivers available out of the box but we need to make
sure that we consider all the legal aspects before it. SQOOP-3231
<https://issues.apache.org/jira/browse/SQOOP-3231> partially covers this,
it is more about executing the third party tests in a CI job upstream, but
eventually it involves answering these questions.
Your contribution is more than welcome on this front, if you have
improvement ideas feel free to submit a patch.
Szabolcs
On Wed, Feb 14, 2018 at 4:12 PM, Attila Szabó <ma...@apache.org> wrote:
> Hi,
>
> I've also found another strange thing:
> It seems oracle/database:12.2.0.1-ee is no longer available on dockerhub,
> thus the usage of these scripts are very difficult at the beginning, and
> needs lots of workarounds (of course it's feasible to clone github, build
> the oracle image, etc., but it's very far from the original goal) to make
> it work (nonetheless it's very much not working out of the box).
>
> Do we plan to do anything with this? (e.g. creating an official Sqoop id
> for dockerhub, and push their the prebuilt image or so). If yes I'd more
> than happy to provide my help, guidance on this front.
>
> Cheers,
> Attila
>
> <http://www.avg.com/email-signature?utm_medium=email&
> utm_source=link&utm_campaign=sig-email&utm_content=webmail>
> Virus-free.
> www.avg.com
> <http://www.avg.com/email-signature?utm_medium=email&
> utm_source=link&utm_campaign=sig-email&utm_content=webmail>
> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
>
> On Wed, Feb 14, 2018 at 12:14 PM, Attila Szabó <ma...@apache.org> wrote:
>
> > Hi all,
> >
> > A few months ago Szabi created start and stop scripts for docker-compose,
> > to run 3rd party DBs for the 3rd party tests, which is great ( it's
> > intention is to make the integration testing of Sqoop much easier ).
> >
> > Though the version of docker-compose file is 3+, thus it's only usable
> > with the quite recent docker-compose+docker versions.
> >
> > And here comes the problem:
> > The latest version of CentOS (7.4) by default comes a much older
> > Docker+Docker-compose version, which is incompatible with the compose
> file
> > version. So out of the box, this solution won't work on the latest CentOS
> > version for example (I've discovered it b/c one of my test servers runs
> > CentOS7.4).
> >
> > I'm not telling it's impossible to install it from the net (first docker
> > and then docker compose), but it's just not the
> standard/convenient/stable
> > solution which would come from the RPM package.
> >
> > Here comes my question:
> > What do you think: is it possible, and would it make sense to rewrite the
> > docker compose file to version 2, and thus providing an EZ way to use
> this
> > feature by the majority of the Linux users? Is there any vital feature in
> > version 3 what is required for this task?
> >
> > Keeping it simple:
> > Would it be possible to provide a much wider usable version of this
> docker
> > based testing solution, or would we like to keep and handle it as an
> > experimental testing feature?
> >
> > Thanks,
> > Attila
> >
>
Re: Question: Third party docker support file version
Posted by Attila Szabó <ma...@apache.org>.
Hi,
I've also found another strange thing:
It seems oracle/database:12.2.0.1-ee is no longer available on dockerhub,
thus the usage of these scripts are very difficult at the beginning, and
needs lots of workarounds (of course it's feasible to clone github, build
the oracle image, etc., but it's very far from the original goal) to make
it work (nonetheless it's very much not working out of the box).
Do we plan to do anything with this? (e.g. creating an official Sqoop id
for dockerhub, and push their the prebuilt image or so). If yes I'd more
than happy to provide my help, guidance on this front.
Cheers,
Attila
<http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
Virus-free.
www.avg.com
<http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
On Wed, Feb 14, 2018 at 12:14 PM, Attila Szabó <ma...@apache.org> wrote:
> Hi all,
>
> A few months ago Szabi created start and stop scripts for docker-compose,
> to run 3rd party DBs for the 3rd party tests, which is great ( it's
> intention is to make the integration testing of Sqoop much easier ).
>
> Though the version of docker-compose file is 3+, thus it's only usable
> with the quite recent docker-compose+docker versions.
>
> And here comes the problem:
> The latest version of CentOS (7.4) by default comes a much older
> Docker+Docker-compose version, which is incompatible with the compose file
> version. So out of the box, this solution won't work on the latest CentOS
> version for example (I've discovered it b/c one of my test servers runs
> CentOS7.4).
>
> I'm not telling it's impossible to install it from the net (first docker
> and then docker compose), but it's just not the standard/convenient/stable
> solution which would come from the RPM package.
>
> Here comes my question:
> What do you think: is it possible, and would it make sense to rewrite the
> docker compose file to version 2, and thus providing an EZ way to use this
> feature by the majority of the Linux users? Is there any vital feature in
> version 3 what is required for this task?
>
> Keeping it simple:
> Would it be possible to provide a much wider usable version of this docker
> based testing solution, or would we like to keep and handle it as an
> experimental testing feature?
>
> Thanks,
> Attila
>