You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by Clausen, Jörn <jo...@uni-bielefeld.de> on 2021/03/10 13:21:26 UTC

[users@httpd] mod_md and DNS challenge

Hi!

Please let me know if there is a more specific list/forum where I can 
ask this question.

I am trying to use mod_md with challenge type "DNS". I have

MDCAChallenges dns-01
MDChallengeDns01 /data/acme/mod_md_worker.sh

in my config. In the error log I see, that Let's Encrypt tries to access 
the correct TXT record

DNS problem: NXDOMAIN looking up TXT for _acme-challenge....

The problem is, that the script "mod_md_worker.sh" does not seem to get 
executed at all (I have debug code in the script, and I checked using 
"strace"). I see no indication in any log, that httpd has trouble 
executing the file, it seems to ignore it completely.

I am using httpd 2.4.37 and mod_md 1.15.7.

Any idea, what I am missing here?

-- 
Jörn Clausen
BITS - Bielefelder IT-Servicezentrum
https://www.uni-bielefeld.de/bits

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


Re: [users@httpd] mod_md and DNS challenge

Posted by Stefan Eissing <st...@greenbytes.de>.

> Am 10.03.2021 um 14:21 schrieb Clausen, Jörn <jo...@uni-bielefeld.de>:
> 
> Hi!
> 
> Please let me know if there is a more specific list/forum where I can ask this question.
> 
> I am trying to use mod_md with challenge type "DNS". I have
> 
> MDCAChallenges dns-01
> MDChallengeDns01 /data/acme/mod_md_worker.sh
> 
> in my config. In the error log I see, that Let's Encrypt tries to access the correct TXT record
> 
> DNS problem: NXDOMAIN looking up TXT for _acme-challenge....
> 
> The problem is, that the script "mod_md_worker.sh" does not seem to get executed at all (I have debug code in the script, and I checked using "strace"). I see no indication in any log, that httpd has trouble executing the file, it seems to ignore it completely.
> 
> I am using httpd 2.4.37 and mod_md 1.15.7.

1.15.7 looks more like a mod_http2 version. Could you check again which mod_md version you have?

In a "new enough" version, there will be a file `job.json` in the domain and/or staging folder where you can see details of the renewal attempts for that specific MDomain. Do you see anything there?

Best Regards,

Stefan

> Any idea, what I am missing here?
> 
> -- 
> Jörn Clausen
> BITS - Bielefelder IT-Servicezentrum
> https://www.uni-bielefeld.de/bits
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


Re: [users@httpd] mod_md and DNS challenge

Posted by Stefan Eissing <st...@greenbytes.de>.

> Am 11.03.2021 um 09:41 schrieb Clausen, Jörn <jo...@uni-bielefeld.de>:
> 
>        "detail": "Starting challenges for domains"
>        "detail": "Setting up challenge 'dns-01' for domain foo"
>        "detail": "Setting up challenge 'dns-01' for domain bar"
>        "detail": "Monitoring challenge status for foo"
>        "detail": "Monitoring challenge status for foo: domain authorization for foo is valid"
>        "detail": "Monitoring challenge status for foo: domain authorization for bar failed with state 3"
>        "detail": "domain authorization for bar failed with state 3"

This says that the dns-01 setup worked for domain 'foo', but Let's Encrypt could not verify the dns challenge for 'bar'. This tells me:
a) your script was run for domain 'foo' and did the right thing, LE saw the DNS entry and was satisfied.
b) your script, when called immediately afterwards 'foo' for 'bar', did not achieve the same

In mod_md, when executing the MDChallengeDns01 command, two things will be logged at level
ERROR: when the script was unable to be executed, which will probably not be resolved by retrying
INFO: when the script returned != 0 and the setup failed, which is retried as a failure might depend on external services that can be unavailable temporarily

That nothing was logged indicates to me that the script run and returned exit code 0.

I would advise the following:
- configure 'LogLevel md:trace2' to see all the details the module does until you have analyzed it
- remove the "staging/foo+bar" folder with the failed attempt
- reload your server

mod_md will pick up that foo+bar needs renewal and you will see in the log when your script is called and what it returned.

Hope this helps,

Stefan



---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


Re: [users@httpd] mod_md and DNS challenge

Posted by Clausen, Jörn <jo...@uni-bielefeld.de>.
Hello!

> I would advise the following:
> - configure 'LogLevel md:trace2' to see all the details the module does until you have analyzed it
> - remove the "staging/foo+bar" folder with the failed attempt
> - reload your server
> 
> mod_md will pick up that foo+bar needs renewal and you will see in the log when your script is called and what it returned.

Whoa, that helped a lot :)

Okay, let's take this one step at a time:

- Indeed, the shell script is called and actually does the right thing. 
I wasn't aware that httpd has it's private /tmp and /var/tmp, so I 
couldn't find the expected files.

- And indeed, on very rare occasions, the process finished and I 
received a valid certificate. At the moment, I have problems that Let's 
Encrypt does not register the update of the DNS record. As soon as I 
stop one request and start a new one, Let's Encrypt sees the DNS entries 
of the previous try. This might be a problem with TTLs on our side, I'll 
have to investigate more.

- In the rare case, where registering the certificate succeeded, I could 
see in the logs something about "teardown", but I couldn't find any hint 
that the script was actually called to remove the TXT entry. I.e. I find 
entries like

dns-01 setup command: /data/acme/mod_md_worker.sh setup ...

but entries like

order teardown setup dns-01:...

where not followed by an actual call to the script.

Thanks so far for the help!

Joern

-- 
Jörn Clausen
BITS - Bielefelder IT-Servicezentrum
https://www.uni-bielefeld.de/bits

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org


Re: [users@httpd] mod_md and DNS challenge

Posted by Clausen, Jörn <jo...@uni-bielefeld.de>.
Hi!
>> The problem is, that the script "mod_md_worker.sh" does not seem to get executed at all (I have debug code in the script, and I checked using "strace"). I see no indication in any log, that httpd has trouble executing the file, it seems to ignore it completely.
>>
>> I am using httpd 2.4.37 and mod_md 1.15.7.
> 
> 1.15.7 looks more like a mod_http2 version. Could you check again which mod_md version you have?

Yes, indeed, that was the version for mod_http2. The mod_md package is 
"1:2.0.8-8.module+el8.3.0+6814+67d1e611" (from RHEL8 appstream).

> In a "new enough" version, there will be a file `job.json` in the domain and/or staging folder where you can see details of the renewal attempts for that specific MDomain. Do you see anything there?

Yes, that file is populated. I see (in chronological order, and the two 
hostnames replaced by "foo" and "bar"):

         "detail": "Checking staging area"
         "detail": "Resetting staging area"
         "detail": "Assessing current status"
         "detail": "Contacting ACME server for foo at 
https://acme-v02.api.letsencrypt.org/directory"
         "detail": "Resetting staging for foo"
         "detail": "Driving ACME protocol for renewal of foo"
         "detail": "Selecting account to use for foo"
         "detail": "Creating new ACME account for foo"
         "detail": "Creating new order"
         "detail": "Starting challenges for domains"
         "detail": "Setting up challenge 'dns-01' for domain foo"
         "detail": "Setting up challenge 'dns-01' for domain bar"
         "detail": "Monitoring challenge status for foo"
         "detail": "Monitoring challenge status for foo: domain 
authorization for foo is valid"
         "detail": "Monitoring challenge status for foo: domain 
authorization for bar failed with state 3"
         "detail": "domain authorization for bar failed with state 3"
         "detail": "Monitoring challenge status for foo"
         "detail": "Checking staging area"
         "detail": "Assessing current status"
         "detail": "Contacting ACME server for foo at 
https://acme-v02.api.letsencrypt.org/directory"
         "detail": "Driving ACME protocol for renewal of foo"
         "detail": "Selecting account to use for foo"
         "detail": "Loaded order from staging"
         "detail": "Starting challenges for domains"
         "detail": "Starting challenges for domains: unexpected AUTHZ 
state 3 for domain bar"
         "detail": "unexpected AUTHZ state 3 for domain bar"
         "detail": "Starting challenges for domains"
         "detail": "Checking staging area"
         "detail": "Assessing current status"
         "detail": "Contacting ACME server for foo at 
https://acme-v02.api.letsencrypt.org/directory"
         "detail": "Driving ACME protocol for renewal of foo"
         "detail": "Selecting account to use for foo"
         "detail": "Loaded order from staging"
         "detail": "Starting challenges for domains"
         "detail": "Starting challenges for domains: unexpected AUTHZ 
state 3 for domain bar"
         "detail": "unexpected AUTHZ state 3 for domain bar"

I guess "Setting up challenge 'dns-01' for ..." is the part where the 
configured script should be executed.

I checked that user "apache" can access and run this script (using "sudo 
-u apache ...", this will create the expected debug output from the 
script), so I think I can rule out any permission problems.

-- 
Jörn Clausen
BITS - Bielefelder IT-Servicezentrum
https://www.uni-bielefeld.de/bits

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org