You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@flink.apache.org by HG <ha...@gmail.com> on 2021/12/20 14:38:59 UTC
flink-playground docker/mvn clean install Unknown host repo.maven.apache.org: Name or service not known
Hello
I am trying to the flink-playground examples.
The docker-compose build fails on the mvn clean install command .
I am behind a proxy.
To diagnose this I started a container based on the already created image
docker run -it --name my-maven-project -v "$(pwd)":/usr/src/mymaven -w
/usr/src/mymaven b92c6af9fde8 /bin/bash
Adding packages with apt goes fine
I added the proxy names with their ip addresses to /etc/hosts, added the
proxies to .m2/settings.xml
After that it worked .
But why does it not work without manual intervention?
Regards Hans-Peter
Re: flink-playground docker/mvn clean install Unknown host repo.maven.apache.org: Name or service not known
Posted by David Morávek <dm...@apache.org>.
I'd say that the playground was never meant to be run behind a corporate
proxy. If you feel like the documentation can be improved improved in this
regard, please open a new jira issue describing what's missing. Ideally if
you'd like to fix this for others with the similar setup that may run into
the issue, the contributions to the documentation are always appreciated ;)
Best,
D.
On Tue, Dec 21, 2021 at 12:15 PM HG <ha...@gmail.com> wrote:
> Hello David,
>
> Yes I understand the issues.
> But the problem is that the documentation on
> https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/try-flink/flink-operations-playground/
> says:
>
> Starting the Playground #
> <https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/try-flink/flink-operations-playground/#starting-the-playground>
>
> *The playground environment is set up in just a few steps*. We will walk
> you through the necessary commands and show how to validate that everything
> is running correctly.
>
> Which is in fact not exactly true
>
> The compose command failse.
>
> I think that instructions need to be added to fix this.
>
>
> Regards Hans-Peter
>
> Op di 21 dec. 2021 om 11:51 schreef David Morávek <dm...@apache.org>:
>
>> oh, I've misread your email... the proxies are generally tricky to reason
>> about, as each software needs to implement the proxy support on its own
>>
>> in general, most tools try to use the commonly "agreed upon" environment
>> variables (HTTP_PROXY & HTTPS_PROXY), which is also case for wget
>>
>> JVM uses java system properties instead [1], so this is most likely what
>> the maven proxy settings do set up. Also consider the fact, that even with
>> this system property, some custom java HTTP clients, may simply decide to
>> ignore this... :)
>>
>> does it explain your concerns?
>>
>> [1]
>> https://docs.oracle.com/javase/8/docs/technotes/guides/net/proxies.html
>>
>> D.
>>
>> On Tue, Dec 21, 2021 at 11:17 AM HG <ha...@gmail.com> wrote:
>>
>>> Hi David
>>> When I start a docker container based on the image that is created
>>> shortly before the mvn command fails, it is resolvable.
>>> I can do : apt update
>>> This basically means that the outside world is reachable
>>>
>>> When I install wget the following succeeds without any issue:
>>>
>>> wget
>>> https://repo.maven.apache.org/maven2/org/apache/flink/flink-table-api-java/1.13.1/flink-table-api-java-1.13.1.pom
>>> --2021-12-21 10:06:37--
>>> https://repo.maven.apache.org/maven2/org/apache/flink/flink-table-api-java/1.13.1/flink-table-api-java-1.13.1.pom
>>> Resolving www-proxy...... (www-proxy......)... 11.9.6.2
>>> Connecting to www-proxy............. ... connected.
>>> Proxy request sent, awaiting response... 200 OK
>>> Length: 5394 (5.3K) [text/xml]
>>> Saving to: ‘flink-table-api-java-1.13.1.pom’
>>>
>>> flink-table-api-java-1.13.1.pom
>>> 100%[=====================================================================================================================================>]
>>> 5.27K --.-KB/s in 0.001s
>>>
>>> 2021-12-21 10:06:39 (4.42 MB/s) - ‘flink-table-api-java-1.13.1.pom’
>>> saved [5394/5394]
>>>
>>> Only setting /root/.m2/settings.xml correctly helps
>>> And that is what I don't understand as the proxy configuration will not
>>> be the same for everyone.
>>>
>>> Regards Hans-Peter
>>>
>>>
>>>
>>>
>>> Op di 21 dec. 2021 om 09:51 schreef David Morávek <dm...@apache.org>:
>>>
>>>> Hello Hans,
>>>>
>>>> it's DNS ;) You need to make sure, that "repo.maven.apache.org" can be
>>>> resolved from your docker container (you can use tools such as host, dig,
>>>> nslookup to verify that). This is may be tricky to debug, unless you're
>>>> familiar with networking. A good place to start might be checking the
>>>> /etc/resolv.conf for details about the DNS server being used.
>>>>
>>>> Best,
>>>> D.
>>>>
>>>> On Mon, Dec 20, 2021 at 3:39 PM HG <ha...@gmail.com> wrote:
>>>>
>>>>> Hello
>>>>> I am trying to the flink-playground examples.
>>>>> The docker-compose build fails on the mvn clean install command .
>>>>>
>>>>> I am behind a proxy.
>>>>> To diagnose this I started a container based on the already created
>>>>> image
>>>>> docker run -it --name my-maven-project -v "$(pwd)":/usr/src/mymaven
>>>>> -w /usr/src/mymaven b92c6af9fde8 /bin/bash
>>>>> Adding packages with apt goes fine
>>>>> I added the proxy names with their ip addresses to /etc/hosts, added
>>>>> the proxies to .m2/settings.xml
>>>>>
>>>>> After that it worked .
>>>>>
>>>>> But why does it not work without manual intervention?
>>>>>
>>>>> Regards Hans-Peter
>>>>>
>>>>
Re: flink-playground docker/mvn clean install Unknown host repo.maven.apache.org: Name or service not known
Posted by HG <ha...@gmail.com>.
Hello David,
Yes I understand the issues.
But the problem is that the documentation on
https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/try-flink/flink-operations-playground/
says:
Starting the Playground #
<https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/try-flink/flink-operations-playground/#starting-the-playground>
*The playground environment is set up in just a few steps*. We will walk
you through the necessary commands and show how to validate that everything
is running correctly.
Which is in fact not exactly true
The compose command failse.
I think that instructions need to be added to fix this.
Regards Hans-Peter
Op di 21 dec. 2021 om 11:51 schreef David Morávek <dm...@apache.org>:
> oh, I've misread your email... the proxies are generally tricky to reason
> about, as each software needs to implement the proxy support on its own
>
> in general, most tools try to use the commonly "agreed upon" environment
> variables (HTTP_PROXY & HTTPS_PROXY), which is also case for wget
>
> JVM uses java system properties instead [1], so this is most likely what
> the maven proxy settings do set up. Also consider the fact, that even with
> this system property, some custom java HTTP clients, may simply decide to
> ignore this... :)
>
> does it explain your concerns?
>
> [1]
> https://docs.oracle.com/javase/8/docs/technotes/guides/net/proxies.html
>
> D.
>
> On Tue, Dec 21, 2021 at 11:17 AM HG <ha...@gmail.com> wrote:
>
>> Hi David
>> When I start a docker container based on the image that is created
>> shortly before the mvn command fails, it is resolvable.
>> I can do : apt update
>> This basically means that the outside world is reachable
>>
>> When I install wget the following succeeds without any issue:
>>
>> wget
>> https://repo.maven.apache.org/maven2/org/apache/flink/flink-table-api-java/1.13.1/flink-table-api-java-1.13.1.pom
>> --2021-12-21 10:06:37--
>> https://repo.maven.apache.org/maven2/org/apache/flink/flink-table-api-java/1.13.1/flink-table-api-java-1.13.1.pom
>> Resolving www-proxy...... (www-proxy......)... 11.9.6.2
>> Connecting to www-proxy............. ... connected.
>> Proxy request sent, awaiting response... 200 OK
>> Length: 5394 (5.3K) [text/xml]
>> Saving to: ‘flink-table-api-java-1.13.1.pom’
>>
>> flink-table-api-java-1.13.1.pom
>> 100%[=====================================================================================================================================>]
>> 5.27K --.-KB/s in 0.001s
>>
>> 2021-12-21 10:06:39 (4.42 MB/s) - ‘flink-table-api-java-1.13.1.pom’
>> saved [5394/5394]
>>
>> Only setting /root/.m2/settings.xml correctly helps
>> And that is what I don't understand as the proxy configuration will not
>> be the same for everyone.
>>
>> Regards Hans-Peter
>>
>>
>>
>>
>> Op di 21 dec. 2021 om 09:51 schreef David Morávek <dm...@apache.org>:
>>
>>> Hello Hans,
>>>
>>> it's DNS ;) You need to make sure, that "repo.maven.apache.org" can be
>>> resolved from your docker container (you can use tools such as host, dig,
>>> nslookup to verify that). This is may be tricky to debug, unless you're
>>> familiar with networking. A good place to start might be checking the
>>> /etc/resolv.conf for details about the DNS server being used.
>>>
>>> Best,
>>> D.
>>>
>>> On Mon, Dec 20, 2021 at 3:39 PM HG <ha...@gmail.com> wrote:
>>>
>>>> Hello
>>>> I am trying to the flink-playground examples.
>>>> The docker-compose build fails on the mvn clean install command .
>>>>
>>>> I am behind a proxy.
>>>> To diagnose this I started a container based on the already created
>>>> image
>>>> docker run -it --name my-maven-project -v "$(pwd)":/usr/src/mymaven
>>>> -w /usr/src/mymaven b92c6af9fde8 /bin/bash
>>>> Adding packages with apt goes fine
>>>> I added the proxy names with their ip addresses to /etc/hosts, added
>>>> the proxies to .m2/settings.xml
>>>>
>>>> After that it worked .
>>>>
>>>> But why does it not work without manual intervention?
>>>>
>>>> Regards Hans-Peter
>>>>
>>>
Re: flink-playground docker/mvn clean install Unknown host repo.maven.apache.org: Name or service not known
Posted by David Morávek <dm...@apache.org>.
oh, I've misread your email... the proxies are generally tricky to reason
about, as each software needs to implement the proxy support on its own
in general, most tools try to use the commonly "agreed upon" environment
variables (HTTP_PROXY & HTTPS_PROXY), which is also case for wget
JVM uses java system properties instead [1], so this is most likely what
the maven proxy settings do set up. Also consider the fact, that even with
this system property, some custom java HTTP clients, may simply decide to
ignore this... :)
does it explain your concerns?
[1] https://docs.oracle.com/javase/8/docs/technotes/guides/net/proxies.html
D.
On Tue, Dec 21, 2021 at 11:17 AM HG <ha...@gmail.com> wrote:
> Hi David
> When I start a docker container based on the image that is created shortly
> before the mvn command fails, it is resolvable.
> I can do : apt update
> This basically means that the outside world is reachable
>
> When I install wget the following succeeds without any issue:
>
> wget
> https://repo.maven.apache.org/maven2/org/apache/flink/flink-table-api-java/1.13.1/flink-table-api-java-1.13.1.pom
> --2021-12-21 10:06:37--
> https://repo.maven.apache.org/maven2/org/apache/flink/flink-table-api-java/1.13.1/flink-table-api-java-1.13.1.pom
> Resolving www-proxy...... (www-proxy......)... 11.9.6.2
> Connecting to www-proxy............. ... connected.
> Proxy request sent, awaiting response... 200 OK
> Length: 5394 (5.3K) [text/xml]
> Saving to: ‘flink-table-api-java-1.13.1.pom’
>
> flink-table-api-java-1.13.1.pom
> 100%[=====================================================================================================================================>]
> 5.27K --.-KB/s in 0.001s
>
> 2021-12-21 10:06:39 (4.42 MB/s) - ‘flink-table-api-java-1.13.1.pom’
> saved [5394/5394]
>
> Only setting /root/.m2/settings.xml correctly helps
> And that is what I don't understand as the proxy configuration will not be
> the same for everyone.
>
> Regards Hans-Peter
>
>
>
>
> Op di 21 dec. 2021 om 09:51 schreef David Morávek <dm...@apache.org>:
>
>> Hello Hans,
>>
>> it's DNS ;) You need to make sure, that "repo.maven.apache.org" can be
>> resolved from your docker container (you can use tools such as host, dig,
>> nslookup to verify that). This is may be tricky to debug, unless you're
>> familiar with networking. A good place to start might be checking the
>> /etc/resolv.conf for details about the DNS server being used.
>>
>> Best,
>> D.
>>
>> On Mon, Dec 20, 2021 at 3:39 PM HG <ha...@gmail.com> wrote:
>>
>>> Hello
>>> I am trying to the flink-playground examples.
>>> The docker-compose build fails on the mvn clean install command .
>>>
>>> I am behind a proxy.
>>> To diagnose this I started a container based on the already created image
>>> docker run -it --name my-maven-project -v "$(pwd)":/usr/src/mymaven
>>> -w /usr/src/mymaven b92c6af9fde8 /bin/bash
>>> Adding packages with apt goes fine
>>> I added the proxy names with their ip addresses to /etc/hosts, added
>>> the proxies to .m2/settings.xml
>>>
>>> After that it worked .
>>>
>>> But why does it not work without manual intervention?
>>>
>>> Regards Hans-Peter
>>>
>>
Re: flink-playground docker/mvn clean install Unknown host repo.maven.apache.org: Name or service not known
Posted by HG <ha...@gmail.com>.
Hi David
When I start a docker container based on the image that is created shortly
before the mvn command fails, it is resolvable.
I can do : apt update
This basically means that the outside world is reachable
When I install wget the following succeeds without any issue:
wget
https://repo.maven.apache.org/maven2/org/apache/flink/flink-table-api-java/1.13.1/flink-table-api-java-1.13.1.pom
--2021-12-21 10:06:37--
https://repo.maven.apache.org/maven2/org/apache/flink/flink-table-api-java/1.13.1/flink-table-api-java-1.13.1.pom
Resolving www-proxy...... (www-proxy......)... 11.9.6.2
Connecting to www-proxy............. ... connected.
Proxy request sent, awaiting response... 200 OK
Length: 5394 (5.3K) [text/xml]
Saving to: ‘flink-table-api-java-1.13.1.pom’
flink-table-api-java-1.13.1.pom
100%[=====================================================================================================================================>]
5.27K --.-KB/s in 0.001s
2021-12-21 10:06:39 (4.42 MB/s) - ‘flink-table-api-java-1.13.1.pom’
saved [5394/5394]
Only setting /root/.m2/settings.xml correctly helps
And that is what I don't understand as the proxy configuration will not be
the same for everyone.
Regards Hans-Peter
Op di 21 dec. 2021 om 09:51 schreef David Morávek <dm...@apache.org>:
> Hello Hans,
>
> it's DNS ;) You need to make sure, that "repo.maven.apache.org" can be
> resolved from your docker container (you can use tools such as host, dig,
> nslookup to verify that). This is may be tricky to debug, unless you're
> familiar with networking. A good place to start might be checking the
> /etc/resolv.conf for details about the DNS server being used.
>
> Best,
> D.
>
> On Mon, Dec 20, 2021 at 3:39 PM HG <ha...@gmail.com> wrote:
>
>> Hello
>> I am trying to the flink-playground examples.
>> The docker-compose build fails on the mvn clean install command .
>>
>> I am behind a proxy.
>> To diagnose this I started a container based on the already created image
>> docker run -it --name my-maven-project -v "$(pwd)":/usr/src/mymaven -w
>> /usr/src/mymaven b92c6af9fde8 /bin/bash
>> Adding packages with apt goes fine
>> I added the proxy names with their ip addresses to /etc/hosts, added the
>> proxies to .m2/settings.xml
>>
>> After that it worked .
>>
>> But why does it not work without manual intervention?
>>
>> Regards Hans-Peter
>>
>
Re: flink-playground docker/mvn clean install Unknown host repo.maven.apache.org: Name or service not known
Posted by David Morávek <dm...@apache.org>.
Hello Hans,
it's DNS ;) You need to make sure, that "repo.maven.apache.org" can be
resolved from your docker container (you can use tools such as host, dig,
nslookup to verify that). This is may be tricky to debug, unless you're
familiar with networking. A good place to start might be checking the
/etc/resolv.conf for details about the DNS server being used.
Best,
D.
On Mon, Dec 20, 2021 at 3:39 PM HG <ha...@gmail.com> wrote:
> Hello
> I am trying to the flink-playground examples.
> The docker-compose build fails on the mvn clean install command .
>
> I am behind a proxy.
> To diagnose this I started a container based on the already created image
> docker run -it --name my-maven-project -v "$(pwd)":/usr/src/mymaven -w
> /usr/src/mymaven b92c6af9fde8 /bin/bash
> Adding packages with apt goes fine
> I added the proxy names with their ip addresses to /etc/hosts, added the
> proxies to .m2/settings.xml
>
> After that it worked .
>
> But why does it not work without manual intervention?
>
> Regards Hans-Peter
>