You are viewing a plain text version of this content. The canonical link for it is here.
Posted to sysadmins@spamassassin.apache.org by Dave Jones <da...@apache.org> on 2017/05/13 18:15:42 UTC
Rsync config on sa-vm1.apache.org
Kevin,
I am nearly complete setting up the rsyncd again. I have firewalled off
port 873 to a single test IP of my masscheck server at ENA. My
automasscheck-minimal.sh is working properly now and able to upload
result logs like normal.
My question to you is where can we get a current mirror copy of the
[updates] rsync module to put back on the source of
sa-vm1:/var/www/buildbot.spamassassin.org/updates
which will also be http://updates.spamassassin.org once populated,
verified ready, and DNS is updated to point to the new VM.
It looks like you have a mirror at http://sa-update.pccc.com that we
could transfer back to the source directory.
--
Dave
Re: Rsync config on sa-vm1.apache.org
Posted by Dave Jones <da...@apache.org>.
On 05/13/2017 02:00 PM, Dave Jones wrote:
>
>
> On 05/13/2017 01:21 PM, Dave Jones wrote:
>>
>>
>> On 05/13/2017 01:15 PM, Dave Jones wrote:
>>> Kevin,
>>> I am nearly complete setting up the rsyncd again. I have firewalled
>>> off port 873 to a single test IP of my masscheck server at ENA. My
>>> automasscheck-minimal.sh is working properly now and able to upload
>>> result logs like normal.
>>>
>>> My question to you is where can we get a current mirror copy of the
>>> [updates] rsync module to put back on the source of
>>>
>>> sa-vm1:/var/www/buildbot.spamassassin.org/updates
>>>
>>> which will also be http://updates.spamassassin.org once populated,
>>> verified ready, and DNS is updated to point to the new VM.
>>
>> Correction. updates.spamassassin.org is not in DNS so that reference
>> in the /etc/rsyncd.conf to being updates.spamassassin.org is not
>> accurate.
>>
>> Is this [updates] rsync module supposed to be available via HTTP? If
>> so, then maybe it is really updatedist.spamassassin.org that I see in
>> DNS.
>>
>
> Nevermind. I found my answer. The Apache HTTPD config boiled down to
> this for the updates area:
>
> <VirtualHost *:80>
> ServerName updatedist.spamassassin.org
> ServerAlias bbmass.spamassassin.org
> ServerAlias buildbot.spamassassin.org
> ServerAlias bbmass-trunk.spamassassin.org
> <Directory /var/www/updatedist.spamassassin.org/updates>
> options indexes SymLinksIfOwnerMatch
> AllowOverride indexes
> </Directory>
> DocumentRoot /var/www/updatedist.spamassassin.org/updates
> ServerAdmin webmaster@spamassassin.org
> # userdir disabled
> </VirtualHost>
>
Sorry, for the updates but I found a discrepancy in the updatedist path
so I have this back in two different virtualhosts again like it was
before. I thought they could have been combined like above but nope.
>>>
>>> It looks like you have a mirror at http://sa-update.pccc.com that we
>>> could transfer back to the source directory.
>>>
>
> I am putting what came out out the backup in there but I don't think
> it's going to be current. The DNS TXT records show 1786853 as the
> latest version and the backup area doesn't have any files that start
> with that.
Re: Rsync config on sa-vm1.apache.org
Posted by "Kevin A. McGrail" <ke...@mcgrail.com>.
On 5/13/2017 4:47 PM, Dave Jones wrote:
> It's working now. Seeing a lot of clients syncing now in rsyncd.log.
>
> BTW, I have setup a redundant mirror of sa-update.ena.com to add to
> the MIRRORED.BY. I have to get some firewall ACL help on Monday to
> allow port 80 to the servers. After that, you can check them out and
> add them to the list.
Excellent. It will be good to have more mirrors!
I created a MIRROR.CHECK file on the master rsync and confirmed it's
being updated.
Saw your other email and I am writing a lot of detail onto that.
NOTE: I deleted the tar file I uploaded that was in /var/www to save space.
Regards,
KAM
Re: Rsync config on sa-vm1.apache.org
Posted by Dave Jones <da...@apache.org>.
It's working now. Seeing a lot of clients syncing now in rsyncd.log.
BTW, I have setup a redundant mirror of sa-update.ena.com to add to the
MIRRORED.BY. I have to get some firewall ACL help on Monday to allow
port 80 to the servers. After that, you can check them out and add them
to the list.
Dave
On 05/13/2017 03:06 PM, Kevin A. McGrail wrote:
> Sure. We have a tar file if we need to resync.
> Regards,
> KAM
>
> On May 13, 2017 4:02:26 PM EDT, Dave Jones <da...@apache.org> wrote:
>> I have it working now and was able to rsync to my server that is
>> allowed
>> through the firewall. Is this a good enough test/check/validation? If
>>
>> so, I am ready to open up port 873 to any source and then update DNS to
>>
>> point all of the CNAMES to sa-vm1.apache.org.
>>
>> Dave
>>
>> On 05/13/2017 02:29 PM, Kevin A. McGrail wrote:
>>> On 5/13/2017 3:16 PM, Dave Jones wrote:
>>>> Are you going to pop on the box and scp your tar over? Another
>>>> option is to use https://filedrop.ena.com and send it to me
>>>> (djones@ena.com) that way.
>>>>
>>>> The /var/www/bbmass.spamassassin.org dir is going to have most of
>> the
>>>> files. It's a lot of files in there so it would be nice/speedy if
>> you
>>>> could rsync the delta from your mirror.
>>>
>>> So if you move the file sa-update-html.tar.gz from kmcgrail homedir
>> on
>>> sa-vm1, that's a tar of all the files. You can use it as a
>> definitive
>>> source of truth and remove the copy you have.
>>>
>>> Here's the script I run as a mirror. NOTE: the box I'm running the
>>> mirror on is a new IP but I don't think we had any restrictions.
>>>
>>> #!/bin/sh
>>> /usr/bin/rsync --timeout=280 -T /tmp -ta --delete
>>> --address=69.171.29.39 rsync.spamassassin.org::updates
>>> /htdocs/sa-update.pccc.com/html > /dev/null 2>&1
>>>
>>> Regards,
>>>
>>> KAM
>>>
>>>
>
Re: Rsync config on sa-vm1.apache.org
Posted by "Kevin A. McGrail" <ke...@mcgrail.com>.
Sure. We have a tar file if we need to resync.
Regards,
KAM
On May 13, 2017 4:02:26 PM EDT, Dave Jones <da...@apache.org> wrote:
>I have it working now and was able to rsync to my server that is
>allowed
>through the firewall. Is this a good enough test/check/validation? If
>
>so, I am ready to open up port 873 to any source and then update DNS to
>
>point all of the CNAMES to sa-vm1.apache.org.
>
>Dave
>
>On 05/13/2017 02:29 PM, Kevin A. McGrail wrote:
>> On 5/13/2017 3:16 PM, Dave Jones wrote:
>>> Are you going to pop on the box and scp your tar over? Another
>>> option is to use https://filedrop.ena.com and send it to me
>>> (djones@ena.com) that way.
>>>
>>> The /var/www/bbmass.spamassassin.org dir is going to have most of
>the
>>> files. It's a lot of files in there so it would be nice/speedy if
>you
>>> could rsync the delta from your mirror.
>>
>> So if you move the file sa-update-html.tar.gz from kmcgrail homedir
>on
>> sa-vm1, that's a tar of all the files. You can use it as a
>definitive
>> source of truth and remove the copy you have.
>>
>> Here's the script I run as a mirror. NOTE: the box I'm running the
>> mirror on is a new IP but I don't think we had any restrictions.
>>
>> #!/bin/sh
>> /usr/bin/rsync --timeout=280 -T /tmp -ta --delete
>> --address=69.171.29.39 rsync.spamassassin.org::updates
>> /htdocs/sa-update.pccc.com/html > /dev/null 2>&1
>>
>> Regards,
>>
>> KAM
>>
>>
Re: Rsync config on sa-vm1.apache.org
Posted by Dave Jones <da...@apache.org>.
I have it working now and was able to rsync to my server that is allowed
through the firewall. Is this a good enough test/check/validation? If
so, I am ready to open up port 873 to any source and then update DNS to
point all of the CNAMES to sa-vm1.apache.org.
Dave
On 05/13/2017 02:29 PM, Kevin A. McGrail wrote:
> On 5/13/2017 3:16 PM, Dave Jones wrote:
>> Are you going to pop on the box and scp your tar over? Another
>> option is to use https://filedrop.ena.com and send it to me
>> (djones@ena.com) that way.
>>
>> The /var/www/bbmass.spamassassin.org dir is going to have most of the
>> files. It's a lot of files in there so it would be nice/speedy if you
>> could rsync the delta from your mirror.
>
> So if you move the file sa-update-html.tar.gz from kmcgrail homedir on
> sa-vm1, that's a tar of all the files. You can use it as a definitive
> source of truth and remove the copy you have.
>
> Here's the script I run as a mirror. NOTE: the box I'm running the
> mirror on is a new IP but I don't think we had any restrictions.
>
> #!/bin/sh
> /usr/bin/rsync --timeout=280 -T /tmp -ta --delete
> --address=69.171.29.39 rsync.spamassassin.org::updates
> /htdocs/sa-update.pccc.com/html > /dev/null 2>&1
>
> Regards,
>
> KAM
>
>
Re: Rsync config on sa-vm1.apache.org
Posted by "Kevin A. McGrail" <ke...@mcgrail.com>.
On 5/13/2017 3:16 PM, Dave Jones wrote:
> Are you going to pop on the box and scp your tar over? Another
> option is to use https://filedrop.ena.com and send it to me
> (djones@ena.com) that way.
>
> The /var/www/bbmass.spamassassin.org dir is going to have most of the
> files. It's a lot of files in there so it would be nice/speedy if you
> could rsync the delta from your mirror.
So if you move the file sa-update-html.tar.gz from kmcgrail homedir on
sa-vm1, that's a tar of all the files. You can use it as a definitive
source of truth and remove the copy you have.
Here's the script I run as a mirror. NOTE: the box I'm running the
mirror on is a new IP but I don't think we had any restrictions.
#!/bin/sh
/usr/bin/rsync --timeout=280 -T /tmp -ta --delete
--address=69.171.29.39 rsync.spamassassin.org::updates
/htdocs/sa-update.pccc.com/html > /dev/null 2>&1
Regards,
KAM
Re: Rsync config on sa-vm1.apache.org
Posted by Dave Jones <da...@apache.org>.
Are you going to pop on the box and scp your tar over? Another
option is to use https://filedrop.ena.com and send it to me
(djones@ena.com) that way.
The /var/www/bbmass.spamassassin.org dir is going to have most of the
files. It's a lot of files in there so it would be nice/speedy if you
could rsync the delta from your mirror.
Dave
On 05/13/2017 02:11 PM, Kevin A. McGrail wrote:
> Don't turn it on yet. I run a mirror and will tar it up.
> Regards,
> KAM
>
> On May 13, 2017 3:00:54 PM EDT, Dave Jones <da...@apache.org> wrote:
>
>
>
> On 05/13/2017 01:21 PM, Dave Jones wrote:
>
>
>
> On 05/13/2017 01:15 PM, Dave Jones wrote:
>
> Kevin,
> I am nearly complete setting up the rsyncd again. I have
> firewalled
> off port 873 to a single test IP of my masscheck server at
> ENA. My
> automasscheck-minimal.sh <http://automasscheck-minimal.sh>
> is working properly now and able to upload
> result logs like normal.
>
> My question to you is where can we get a current mirror copy
> of the
> [updates] rsync module to put back on the source of
>
> sa-vm1:/var/www/buildbot.spamassassin.org/updates
> <http://buildbot.spamassassin.org/updates>
>
> which will also be http://updates.spamassassin.org once
> populated,
> verified ready, and DNS is updated to point to the new VM.
>
>
> Correction. updates.spamassassin.org
> <http://updates.spamassassin.org> is not in DNS so that
> reference in
> the /etc/rsyncd.conf to being updates.spamassassin.org
> <http://updates.spamassassin.org> is not accurate.
>
> Is this [updates] rsync module supposed to be available via
> HTTP? If
> so, then maybe it is really updatedist.spamassassin.org
> <http://updatedist.spamassassin.org> that I see in DNS.
>
>
> Nevermind. I found my answer. The Apache HTTPD config boiled down to
> this for the updates area:
>
> <VirtualHost *:80>
> ServerNameupdatedist.spamassassin.org <http://updatedist.spamassassin.org>
> ServerAliasbbmass.spamassassin.org <http://bbmass.spamassassin.org>
> ServerAliasbuildbot.spamassassin.org <http://buildbot.spamassassin.org>
> ServerAliasbbmass-trunk.spamassassin.org <http://bbmass-trunk.spamassassin.org>
> <Directory /var/www/updatedist.spamassassin.org/updates
> <http://updatedist.spamassassin.org/updates>>
> options indexes SymLinksIfOwnerMatch
> AllowOverride indexes
> </Directory>
> DocumentRoot /var/www/updatedist.spamassassin.org/updates
> <http://updatedist.spamassassin.org/updates>
> ServerAdmin webmaster@spamassassin.org
> # userdir disabled
> </VirtualHost>
>
>
> It looks like you have a mirror at http://sa-update.pccc.com
> that we
> could transfer back to the source directory.
>
>
>
> I am putting what came out out the backup in there but I don't think
> it's going to be current. The DNS TXT records show 1786853 as the
> latest version and the backup area doesn't have any files that start
> with that.
>
Re: Rsync config on sa-vm1.apache.org
Posted by "Kevin A. McGrail" <ke...@mcgrail.com>.
Don't turn it on yet. I run a mirror and will tar it up.
Regards,
KAM
On May 13, 2017 3:00:54 PM EDT, Dave Jones <da...@apache.org> wrote:
>
>
>On 05/13/2017 01:21 PM, Dave Jones wrote:
>>
>>
>> On 05/13/2017 01:15 PM, Dave Jones wrote:
>>> Kevin,
>>> I am nearly complete setting up the rsyncd again. I have firewalled
>
>>> off port 873 to a single test IP of my masscheck server at ENA. My
>>> automasscheck-minimal.sh is working properly now and able to upload
>>> result logs like normal.
>>>
>>> My question to you is where can we get a current mirror copy of the
>>> [updates] rsync module to put back on the source of
>>>
>>> sa-vm1:/var/www/buildbot.spamassassin.org/updates
>>>
>>> which will also be http://updates.spamassassin.org once populated,
>>> verified ready, and DNS is updated to point to the new VM.
>>
>> Correction. updates.spamassassin.org is not in DNS so that reference
>in
>> the /etc/rsyncd.conf to being updates.spamassassin.org is not
>accurate.
>>
>> Is this [updates] rsync module supposed to be available via HTTP? If
>
>> so, then maybe it is really updatedist.spamassassin.org that I see in
>DNS.
>>
>
>Nevermind. I found my answer. The Apache HTTPD config boiled down to
>this for the updates area:
>
><VirtualHost *:80>
> ServerName updatedist.spamassassin.org
> ServerAlias bbmass.spamassassin.org
> ServerAlias buildbot.spamassassin.org
> ServerAlias bbmass-trunk.spamassassin.org
> <Directory /var/www/updatedist.spamassassin.org/updates>
> options indexes SymLinksIfOwnerMatch
> AllowOverride indexes
> </Directory>
> DocumentRoot /var/www/updatedist.spamassassin.org/updates
> ServerAdmin webmaster@spamassassin.org
> # userdir disabled
></VirtualHost>
>
>>>
>>> It looks like you have a mirror at http://sa-update.pccc.com that we
>
>>> could transfer back to the source directory.
>>>
>
>I am putting what came out out the backup in there but I don't think
>it's going to be current. The DNS TXT records show 1786853 as the
>latest version and the backup area doesn't have any files that start
>with that.
Re: Rsync config on sa-vm1.apache.org
Posted by Dave Jones <da...@apache.org>.
On 05/13/2017 01:21 PM, Dave Jones wrote:
>
>
> On 05/13/2017 01:15 PM, Dave Jones wrote:
>> Kevin,
>> I am nearly complete setting up the rsyncd again. I have firewalled
>> off port 873 to a single test IP of my masscheck server at ENA. My
>> automasscheck-minimal.sh is working properly now and able to upload
>> result logs like normal.
>>
>> My question to you is where can we get a current mirror copy of the
>> [updates] rsync module to put back on the source of
>>
>> sa-vm1:/var/www/buildbot.spamassassin.org/updates
>>
>> which will also be http://updates.spamassassin.org once populated,
>> verified ready, and DNS is updated to point to the new VM.
>
> Correction. updates.spamassassin.org is not in DNS so that reference in
> the /etc/rsyncd.conf to being updates.spamassassin.org is not accurate.
>
> Is this [updates] rsync module supposed to be available via HTTP? If
> so, then maybe it is really updatedist.spamassassin.org that I see in DNS.
>
Nevermind. I found my answer. The Apache HTTPD config boiled down to
this for the updates area:
<VirtualHost *:80>
ServerName updatedist.spamassassin.org
ServerAlias bbmass.spamassassin.org
ServerAlias buildbot.spamassassin.org
ServerAlias bbmass-trunk.spamassassin.org
<Directory /var/www/updatedist.spamassassin.org/updates>
options indexes SymLinksIfOwnerMatch
AllowOverride indexes
</Directory>
DocumentRoot /var/www/updatedist.spamassassin.org/updates
ServerAdmin webmaster@spamassassin.org
# userdir disabled
</VirtualHost>
>>
>> It looks like you have a mirror at http://sa-update.pccc.com that we
>> could transfer back to the source directory.
>>
I am putting what came out out the backup in there but I don't think
it's going to be current. The DNS TXT records show 1786853 as the
latest version and the backup area doesn't have any files that start
with that.
Re: Rsync config on sa-vm1.apache.org
Posted by Dave Jones <da...@apache.org>.
On 05/13/2017 01:15 PM, Dave Jones wrote:
> Kevin,
> I am nearly complete setting up the rsyncd again. I have firewalled off
> port 873 to a single test IP of my masscheck server at ENA. My
> automasscheck-minimal.sh is working properly now and able to upload
> result logs like normal.
>
> My question to you is where can we get a current mirror copy of the
> [updates] rsync module to put back on the source of
>
> sa-vm1:/var/www/buildbot.spamassassin.org/updates
>
> which will also be http://updates.spamassassin.org once populated,
> verified ready, and DNS is updated to point to the new VM.
Correction. updates.spamassassin.org is not in DNS so that reference in
the /etc/rsyncd.conf to being updates.spamassassin.org is not accurate.
Is this [updates] rsync module supposed to be available via HTTP? If
so, then maybe it is really updatedist.spamassassin.org that I see in DNS.
>
> It looks like you have a mirror at http://sa-update.pccc.com that we
> could transfer back to the source directory.
>