You are viewing a plain text version of this content. The canonical link for it is here.
Posted to sysadmins@spamassassin.apache.org by Dave Jones <da...@apache.org> on 2017/05/03 22:53:39 UTC
Next priority to get running on sa-vm1.apache.org
Kevin,
I am sorting through the backups in /x1 which is a lot. What do you
want me to target next while we finish off the DNS hidden master
details? Is it the Masscheck?
--
Dave
Re: Next priority to get running on sa-vm1.apache.org
Posted by "Kevin A. McGrail" <km...@apache.org>.
On 5/12/2017 12:34 PM, Bryan Vest wrote:
> I see the step I missed. I'll get it fixed up.
Excellent. When you have done that, then Dave or I need to open a Jira
ticket.
--
Kevin A. McGrail
Asst. Treasurer, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
Re: Next priority to get running on sa-vm1.apache.org
Posted by Bryan Vest <mu...@gmail.com>.
I see the step I missed. I'll get it fixed up.
On May 12, 2017 9:13 AM, "Kevin A. McGrail" <km...@apache.org> wrote:
> Looking at this second email made me realize I never saw a request for
> Bryan to have access to the machine. Sorry Bryan, I think there are a few
> steps being skipped.
>
> My LONG email a few days ago* had the onboard process
>
> * I am hoping one of you two will integrate this into the wiki so that it
> has another pair of eyes on it.
>
> Repeating the relevant item below so Bryan can test out the process and we
> can identify missing steps, please!
>
> Regards,
> KAM
>
>
> How to Onboard someone as a SysAdmin:
>
> - A PMC Member nominates a new SASA member as a committer since we store
> items in SVN for configs
> NOTE: If they later produce code, they should request that permission from
> the PMC.
>
> - If the vote is successful, they then follow all the normal committer
> guidelines to get them an Apache ID including an appropriate committer
> license: https://www.apache.org/dev/new-committers-guide.html
>
> - Once they have an Apache ID, they should:
>
> - SASA Member signs up for an Infra Jira account at
> https://issues.apache.org/jira/secure/Signup!default.jspa?
> - SASA Member adds an SSH public key to id.apache.org
> - Add your PGP public key. http://people.apache.org/~kmcgrail/
> - Create an account on our Wiki
> - Email sysadmins-subscribe@spamassassin.apache.org
> - Email sysadmins@spamassassin.apache.org and ask for karma to access
> sa-vm1 with sudo access
> - Email sysadmins@spamassassin.apache.org and ask for your account to
> be added to https://wiki.apache.org/spamassassin/ContributorsGroup and
> https://wiki.apache.org/spamassassin/AdminGroup
> - Start looking at https://wiki.apache.org/spamassassin/DevelopmentStuff
> under infrastruction
>
> - Someone with Karma needs to:
> - Approve request to sysadmins mailing list
> - Add them to ContributorsGroup and AdminGroup on Wikki
> - Open a JIRA ticket at issues.apache.org similar to INFRA-14045 to get
> them access to our box
>
> On 5/12/2017 6:59 AM, Bryan Vest wrote:
>
> Having a problem getting setup. I went through the process can I can sftp
> to home.apache.org but sa-vm1.apache.org gives me LDAP authorisation check
> failed.
>
> --Bryan Vest<http://sa-vm1.apache.org> <http://sa-vm1.apache.org>
>
> On Wed, May 10, 2017 at 11:08 AM, Dave Jones <da...@apache.org> <da...@apache.org> wrote:
>
>
> Brian,
>
> I am still working on getting /usr/local/spamassassin setup properly as
> our primary directory with all of the components under it. Let me get a
> little further on this over the next couple of days and then I think we
> will be at a place where 2 sysadmins can "be in the kitchen" without having
> "too many cooks in the kitchen."
>
> At that point we can split up the components -- automc (auto masscheck),
> bbmass (buildbot mass check), corpus, etc. -- and work on these in parallel.
>
> In the meantime:
>
> 1. Setup your apache.org email address and the forwarding to yourgmail.com address at https://id.apache.org.
>
> 2. Setup your SSH key, PGP key, etc. in https://id.apache.org.
>
> 3. Setup your OPIE password so you can SSH into sa-vm1.apache.org:
> https://reference.apache.org/committer/opie
>
> 4. Make sure you have the ability to "sudo su -" to become root onsa-vm1.apache.org by using OPIE. Here is the OPIE client I am using:
> https://reference.apache.org/committer/otp-md5
>
> Dave
>
>
> On 05/08/2017 10:15 AM, Kevin A. McGrail wrote:
>
>
> On 5/7/2017 3:43 PM, Dave Jones wrote:
>
>
> Do you know what the long-term plan is for the /x1 mount? Was it setup
> as a temporary place to restore old backups?
>
> I am trying to organize everything under /usr/local/spamassassin and the
> root FS doesn't have enough space. If the plan was to eventually unmount
> and give back the /x1 storage, we can do that. If we are going able to keep
> it, then I would like to remount it under /usr/local/spamassassin if that
> is OK.
>
> root@sa-vm1:/usr/local/spamassassin# df -H
> Filesystem Size Used Avail Use% Mounted on
> /dev/sda1 34G 8.9G 24G 28% /
> /dev/sdb1 1.2T 426G 678G 39% /x1
>
>
>
> If we need the space, use it and remount how you would like!
>
> Regards,
>
> KAM
>
>
>
>
>
>
> --
> Kevin A. McGrail
> Asst. Treasurer, Apache Software Foundation
> Chair Emeritus Apache SpamAssassin Project
>
>
Re: Next priority to get running on sa-vm1.apache.org
Posted by "Kevin A. McGrail" <km...@apache.org>.
Looking at this second email made me realize I never saw a request for
Bryan to have access to the machine. Sorry Bryan, I think there are a
few steps being skipped.
My LONG email a few days ago* had the onboard process
* I am hoping one of you two will integrate this into the wiki so that
it has another pair of eyes on it.
Repeating the relevant item below so Bryan can test out the process and
we can identify missing steps, please!
Regards,
KAM
How to Onboard someone as a SysAdmin:
- A PMC Member nominates a new SASA member as a committer since we store
items in SVN for configs
NOTE: If they later produce code, they should request that permission
from the PMC.
- If the vote is successful, they then follow all the normal committer
guidelines to get them an Apache ID including an appropriate committer
license: https://www.apache.org/dev/new-committers-guide.html
- Once they have an Apache ID, they should:
- SASA Member signs up for an Infra Jira account at
https://issues.apache.org/jira/secure/Signup!default.jspa?
- SASA Member adds an SSH public key to id.apache.org
- Add your PGP public key. http://people.apache.org/~kmcgrail/
- Create an account on our Wiki
- Email sysadmins-subscribe@spamassassin.apache.org
- Email sysadmins@spamassassin.apache.org and ask for karma to access
sa-vm1 with sudo access
- Email sysadmins@spamassassin.apache.org and ask for your account to
be added to https://wiki.apache.org/spamassassin/ContributorsGroup and
https://wiki.apache.org/spamassassin/AdminGroup
- Start looking at
https://wiki.apache.org/spamassassin/DevelopmentStuff under infrastruction
- Someone with Karma needs to:
- Approve request to sysadmins mailing list
- Add them to ContributorsGroup and AdminGroup on Wikki
- Open a JIRA ticket at issues.apache.org similar to INFRA-14045 to
get them access to our box
On 5/12/2017 6:59 AM, Bryan Vest wrote:
> Having a problem getting setup. I went through the process can I can sftp
> to home.apache.org but sa-vm1.apache.org gives me LDAP authorisation check
> failed.
>
> --Bryan Vest
> <http://sa-vm1.apache.org>
>
> On Wed, May 10, 2017 at 11:08 AM, Dave Jones <da...@apache.org> wrote:
>
>> Brian,
>>
>> I am still working on getting /usr/local/spamassassin setup properly as
>> our primary directory with all of the components under it. Let me get a
>> little further on this over the next couple of days and then I think we
>> will be at a place where 2 sysadmins can "be in the kitchen" without having
>> "too many cooks in the kitchen."
>>
>> At that point we can split up the components -- automc (auto masscheck),
>> bbmass (buildbot mass check), corpus, etc. -- and work on these in parallel.
>>
>> In the meantime:
>>
>> 1. Setup your apache.org email address and the forwarding to your
>> gmail.com address at https://id.apache.org.
>>
>> 2. Setup your SSH key, PGP key, etc. in https://id.apache.org.
>>
>> 3. Setup your OPIE password so you can SSH into sa-vm1.apache.org:
>>
>> https://reference.apache.org/committer/opie
>>
>> 4. Make sure you have the ability to "sudo su -" to become root on
>> sa-vm1.apache.org by using OPIE. Here is the OPIE client I am using:
>>
>> https://reference.apache.org/committer/otp-md5
>>
>> Dave
>>
>>
>> On 05/08/2017 10:15 AM, Kevin A. McGrail wrote:
>>
>>> On 5/7/2017 3:43 PM, Dave Jones wrote:
>>>
>>>> Do you know what the long-term plan is for the /x1 mount? Was it setup
>>>> as a temporary place to restore old backups?
>>>>
>>>> I am trying to organize everything under /usr/local/spamassassin and the
>>>> root FS doesn't have enough space. If the plan was to eventually unmount
>>>> and give back the /x1 storage, we can do that. If we are going able to keep
>>>> it, then I would like to remount it under /usr/local/spamassassin if that
>>>> is OK.
>>>>
>>>> root@sa-vm1:/usr/local/spamassassin# df -H
>>>> Filesystem Size Used Avail Use% Mounted on
>>>> /dev/sda1 34G 8.9G 24G 28% /
>>>> /dev/sdb1 1.2T 426G 678G 39% /x1
>>>>
>>> If we need the space, use it and remount how you would like!
>>>
>>> Regards,
>>>
>>> KAM
>>>
>>>
>>>
--
Kevin A. McGrail
Asst. Treasurer, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
Re: Next priority to get running on sa-vm1.apache.org
Posted by Bryan Vest <mu...@gmail.com>.
Having a problem getting setup. I went through the process can I can sftp
to home.apache.org but sa-vm1.apache.org gives me LDAP authorisation check
failed.
--Bryan Vest
<http://sa-vm1.apache.org>
On Wed, May 10, 2017 at 11:08 AM, Dave Jones <da...@apache.org> wrote:
> Brian,
>
> I am still working on getting /usr/local/spamassassin setup properly as
> our primary directory with all of the components under it. Let me get a
> little further on this over the next couple of days and then I think we
> will be at a place where 2 sysadmins can "be in the kitchen" without having
> "too many cooks in the kitchen."
>
> At that point we can split up the components -- automc (auto masscheck),
> bbmass (buildbot mass check), corpus, etc. -- and work on these in parallel.
>
> In the meantime:
>
> 1. Setup your apache.org email address and the forwarding to your
> gmail.com address at https://id.apache.org.
>
> 2. Setup your SSH key, PGP key, etc. in https://id.apache.org.
>
> 3. Setup your OPIE password so you can SSH into sa-vm1.apache.org:
>
> https://reference.apache.org/committer/opie
>
> 4. Make sure you have the ability to "sudo su -" to become root on
> sa-vm1.apache.org by using OPIE. Here is the OPIE client I am using:
>
> https://reference.apache.org/committer/otp-md5
>
> Dave
>
>
> On 05/08/2017 10:15 AM, Kevin A. McGrail wrote:
>
>> On 5/7/2017 3:43 PM, Dave Jones wrote:
>>
>>> Do you know what the long-term plan is for the /x1 mount? Was it setup
>>> as a temporary place to restore old backups?
>>>
>>> I am trying to organize everything under /usr/local/spamassassin and the
>>> root FS doesn't have enough space. If the plan was to eventually unmount
>>> and give back the /x1 storage, we can do that. If we are going able to keep
>>> it, then I would like to remount it under /usr/local/spamassassin if that
>>> is OK.
>>>
>>> root@sa-vm1:/usr/local/spamassassin# df -H
>>> Filesystem Size Used Avail Use% Mounted on
>>> /dev/sda1 34G 8.9G 24G 28% /
>>> /dev/sdb1 1.2T 426G 678G 39% /x1
>>>
>>
>> If we need the space, use it and remount how you would like!
>>
>> Regards,
>>
>> KAM
>>
>>
>>
>
Re: Next priority to get running on sa-vm1.apache.org
Posted by Dave Jones <da...@apache.org>.
Brian,
I am still working on getting /usr/local/spamassassin setup properly as
our primary directory with all of the components under it. Let me get a
little further on this over the next couple of days and then I think we
will be at a place where 2 sysadmins can "be in the kitchen" without
having "too many cooks in the kitchen."
At that point we can split up the components -- automc (auto masscheck),
bbmass (buildbot mass check), corpus, etc. -- and work on these in parallel.
In the meantime:
1. Setup your apache.org email address and the forwarding to your
gmail.com address at https://id.apache.org.
2. Setup your SSH key, PGP key, etc. in https://id.apache.org.
3. Setup your OPIE password so you can SSH into sa-vm1.apache.org:
https://reference.apache.org/committer/opie
4. Make sure you have the ability to "sudo su -" to become root on
sa-vm1.apache.org by using OPIE. Here is the OPIE client I am using:
https://reference.apache.org/committer/otp-md5
Dave
On 05/08/2017 10:15 AM, Kevin A. McGrail wrote:
> On 5/7/2017 3:43 PM, Dave Jones wrote:
>> Do you know what the long-term plan is for the /x1 mount? Was it
>> setup as a temporary place to restore old backups?
>>
>> I am trying to organize everything under /usr/local/spamassassin and
>> the root FS doesn't have enough space. If the plan was to eventually
>> unmount and give back the /x1 storage, we can do that. If we are
>> going able to keep it, then I would like to remount it under
>> /usr/local/spamassassin if that is OK.
>>
>> root@sa-vm1:/usr/local/spamassassin# df -H
>> Filesystem Size Used Avail Use% Mounted on
>> /dev/sda1 34G 8.9G 24G 28% /
>> /dev/sdb1 1.2T 426G 678G 39% /x1
>
> If we need the space, use it and remount how you would like!
>
> Regards,
>
> KAM
>
>
Re: Next priority to get running on sa-vm1.apache.org
Posted by "Kevin A. McGrail" <km...@apache.org>.
On 5/7/2017 3:43 PM, Dave Jones wrote:
> Do you know what the long-term plan is for the /x1 mount? Was it
> setup as a temporary place to restore old backups?
>
> I am trying to organize everything under /usr/local/spamassassin and
> the root FS doesn't have enough space. If the plan was to eventually
> unmount and give back the /x1 storage, we can do that. If we are going
> able to keep it, then I would like to remount it under
> /usr/local/spamassassin if that is OK.
>
> root@sa-vm1:/usr/local/spamassassin# df -H
> Filesystem Size Used Avail Use% Mounted on
> /dev/sda1 34G 8.9G 24G 28% /
> /dev/sdb1 1.2T 426G 678G 39% /x1
If we need the space, use it and remount how you would like!
Regards,
KAM
--
Kevin A. McGrail
Asst. Treasurer, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
Re: Next priority to get running on sa-vm1.apache.org
Posted by Dave Jones <da...@apache.org>.
On 05/07/2017 09:37 AM, Kevin A. McGrail wrote:
> Rip and shred away. I can also search other backups I likely have.
> Regards,
> KAM
>
> On May 7, 2017 9:54:01 AM EDT, Dave Jones <da...@apache.org> wrote:
>> On 05/04/2017 10:18 PM, Kevin A. McGrail wrote:
>>> On 5/3/2017 6:53 PM, Dave Jones wrote:
>>>> Kevin,
>>>> I am sorting through the backups in /x1 which is a lot. What do you
>>
>>>> want me to target next while we finish off the DNS hidden master
>>>> details? Is it the Masscheck?
>>>>
>>> OK, so the next thing I would work on is look for the rsync
>>> configuration / password file / locations and get that running.
>>>
>>> IMPORTANT: don't let the sync's happen until we have the content they
>>
>>> sync. I don't want the sa-update mirrors to suddenly go blank.
>>>
>>> My key goal after DNS is to get it so the process for building a
>>> release can complete. That's the build/README file in the 3.4
>> branch.
>>>
>>> That's failing at this point.
>>>
>>> /usr/bin/perl -e 'print "CPAN_VERSION=3.004002\n"' >> version.env
>>> --2017-05-04 23:16:19--
>>> http://buildbot.spamassassin.org/updatestage//1786853.tar.gz
>>> Resolving buildbot.spamassassin.org (buildbot.spamassassin.org)...
>>> 140.211.11.80
>>> Connecting to buildbot.spamassassin.org
>>> (buildbot.spamassassin.org)|140.211.11.80|:80...
>>>
>>> Regards,
>>> KAM
>>
>> Kevin,
>>
>> I have been searching through all of the vm backup sorting everything
>> out. It's kinda a mess due to all of the various paths that have been
>> in place and then carried forward with symlinks. I would like to sort
>> out everything into logically-name directories under /home/sa or
>> /usr/local/sa if that is OK. I know this will take a little more time
>> but I feel this "spring cleaning" is long overdue and needed for
>> improved support going forward.
>>
>> I will be working most of today on this still organizing my notes so
>> let
>> me know your thoughts or recommendations before I get too far.
>>
>> --
>> Dave
>
Kevin,
Do you know what the long-term plan is for the /x1 mount? Was it setup
as a temporary place to restore old backups?
I am trying to organize everything under /usr/local/spamassassin and the
root FS doesn't have enough space. If the plan was to eventually
unmount and give back the /x1 storage, we can do that. If we are going
able to keep it, then I would like to remount it under
/usr/local/spamassassin if that is OK.
root@sa-vm1:/usr/local/spamassassin# df -H
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 34G 8.9G 24G 28% /
/dev/sdb1 1.2T 426G 678G 39% /x1
Dave
Re: Next priority to get running on sa-vm1.apache.org
Posted by "Kevin A. McGrail" <ke...@mcgrail.com>.
Rip and shred away. I can also search other backups I likely have.
Regards,
KAM
On May 7, 2017 9:54:01 AM EDT, Dave Jones <da...@apache.org> wrote:
>On 05/04/2017 10:18 PM, Kevin A. McGrail wrote:
>> On 5/3/2017 6:53 PM, Dave Jones wrote:
>>> Kevin,
>>> I am sorting through the backups in /x1 which is a lot. What do you
>
>>> want me to target next while we finish off the DNS hidden master
>>> details? Is it the Masscheck?
>>>
>> OK, so the next thing I would work on is look for the rsync
>> configuration / password file / locations and get that running.
>>
>> IMPORTANT: don't let the sync's happen until we have the content they
>
>> sync. I don't want the sa-update mirrors to suddenly go blank.
>>
>> My key goal after DNS is to get it so the process for building a
>> release can complete. That's the build/README file in the 3.4
>branch.
>>
>> That's failing at this point.
>>
>> /usr/bin/perl -e 'print "CPAN_VERSION=3.004002\n"' >> version.env
>> --2017-05-04 23:16:19--
>> http://buildbot.spamassassin.org/updatestage//1786853.tar.gz
>> Resolving buildbot.spamassassin.org (buildbot.spamassassin.org)...
>> 140.211.11.80
>> Connecting to buildbot.spamassassin.org
>> (buildbot.spamassassin.org)|140.211.11.80|:80...
>>
>> Regards,
>> KAM
>
>Kevin,
>
>I have been searching through all of the vm backup sorting everything
>out. It's kinda a mess due to all of the various paths that have been
>in place and then carried forward with symlinks. I would like to sort
>out everything into logically-name directories under /home/sa or
>/usr/local/sa if that is OK. I know this will take a little more time
>but I feel this "spring cleaning" is long overdue and needed for
>improved support going forward.
>
>I will be working most of today on this still organizing my notes so
>let
>me know your thoughts or recommendations before I get too far.
>
>--
>Dave
Re: Next priority to get running on sa-vm1.apache.org
Posted by Dave Jones <da...@apache.org>.
On 05/04/2017 10:18 PM, Kevin A. McGrail wrote:
> On 5/3/2017 6:53 PM, Dave Jones wrote:
>> Kevin,
>> I am sorting through the backups in /x1 which is a lot. What do you
>> want me to target next while we finish off the DNS hidden master
>> details? Is it the Masscheck?
>>
> OK, so the next thing I would work on is look for the rsync
> configuration / password file / locations and get that running.
>
> IMPORTANT: don't let the sync's happen until we have the content they
> sync. I don't want the sa-update mirrors to suddenly go blank.
>
> My key goal after DNS is to get it so the process for building a
> release can complete. That's the build/README file in the 3.4 branch.
>
> That's failing at this point.
>
> /usr/bin/perl -e 'print "CPAN_VERSION=3.004002\n"' >> version.env
> --2017-05-04 23:16:19--
> http://buildbot.spamassassin.org/updatestage//1786853.tar.gz
> Resolving buildbot.spamassassin.org (buildbot.spamassassin.org)...
> 140.211.11.80
> Connecting to buildbot.spamassassin.org
> (buildbot.spamassassin.org)|140.211.11.80|:80...
>
> Regards,
> KAM
Kevin,
I have been searching through all of the vm backup sorting everything
out. It's kinda a mess due to all of the various paths that have been
in place and then carried forward with symlinks. I would like to sort
out everything into logically-name directories under /home/sa or
/usr/local/sa if that is OK. I know this will take a little more time
but I feel this "spring cleaning" is long overdue and needed for
improved support going forward.
I will be working most of today on this still organizing my notes so let
me know your thoughts or recommendations before I get too far.
--
Dave
Re: SysAdmin Tasklist was Re: Next priority to get running on sa-vm1.apache.org
Posted by "Kevin A. McGrail" <ke...@mcgrail.com>.
Makes sense and thanks.
Regards,
KAM
On May 15, 2017 9:09:05 AM EDT, Dave Jones <da...@apache.org> wrote:
>
>
>On 05/14/2017 09:25 AM, Kevin A. McGrail wrote:
>> On 5/14/2017 10:11 AM, Dave Jones wrote:
>>>
>>> Do we want to subscribe root like this? It doesn't need to receive
>>> any of these emails that will just fill up the root mailbox or
>>> possibly create a mail loop. I was thinking about allowing it as a
>>> non-member poster. I am more familiar with Mailman that allows a
>list
>>> of addresses that can post without being a member. Do you know if
>>> there is something like this available on the ASF lists? If so, I
>>> assumed this would be a Jira task for someone with admin rights to
>the
>>> listserv.
>>>
>>
>> Good point. Can you open an infra ticket and ask? In the meantime
>just
>> subscribe root.
>
>I am not able to subscribe root@sa-vm1.apache.org since inbound mail to
>
>sa-vm1.apache.org is not setup currently since Postfix is only
>listening
>on localhost. It don't want to change anything with Postfix just for
>this.
>
>I opened a Jira task to add root@sa-vm1.apache.org to be allowed to
>post
>as a non-subscriber. It seems that ezmlm supports this feature but it
>has to be added manually to a file by the admins.
>
>>
>>> I tweaked the sa-update-mirror-check.sh script and it's now
>>> /usr/local/bin/checkSAupdateMirrors.sh and symlink'd to
>>> /etc/cron.daily to send this list an email with the status of all
>the
>>> mirrors.
>>
>> Good! Can you change it to hourly when you get it tweaked.
>>
>
>Done. It is now symlinked to /etc/cron.hourly.
Re: SysAdmin Tasklist was Re: Next priority to get running on
sa-vm1.apache.org
Posted by Dave Jones <da...@apache.org>.
On 05/14/2017 09:25 AM, Kevin A. McGrail wrote:
> On 5/14/2017 10:11 AM, Dave Jones wrote:
>>
>> Do we want to subscribe root like this? It doesn't need to receive
>> any of these emails that will just fill up the root mailbox or
>> possibly create a mail loop. I was thinking about allowing it as a
>> non-member poster. I am more familiar with Mailman that allows a list
>> of addresses that can post without being a member. Do you know if
>> there is something like this available on the ASF lists? If so, I
>> assumed this would be a Jira task for someone with admin rights to the
>> listserv.
>>
>
> Good point. Can you open an infra ticket and ask? In the meantime just
> subscribe root.
I am not able to subscribe root@sa-vm1.apache.org since inbound mail to
sa-vm1.apache.org is not setup currently since Postfix is only listening
on localhost. It don't want to change anything with Postfix just for this.
I opened a Jira task to add root@sa-vm1.apache.org to be allowed to post
as a non-subscriber. It seems that ezmlm supports this feature but it
has to be added manually to a file by the admins.
>
>> I tweaked the sa-update-mirror-check.sh script and it's now
>> /usr/local/bin/checkSAupdateMirrors.sh and symlink'd to
>> /etc/cron.daily to send this list an email with the status of all the
>> mirrors.
>
> Good! Can you change it to hourly when you get it tweaked.
>
Done. It is now symlinked to /etc/cron.hourly.
Re: SysAdmin Tasklist was Re: Next priority to get running on
sa-vm1.apache.org
Posted by "Kevin A. McGrail" <ke...@mcgrail.com>.
On 5/14/2017 10:11 AM, Dave Jones wrote:
>
> Do we want to subscribe root like this? It doesn't need to receive
> any of these emails that will just fill up the root mailbox or
> possibly create a mail loop. I was thinking about allowing it as a
> non-member poster. I am more familiar with Mailman that allows a list
> of addresses that can post without being a member. Do you know if
> there is something like this available on the ASF lists? If so, I
> assumed this would be a Jira task for someone with admin rights to the
> listserv.
>
Good point. Can you open an infra ticket and ask? In the meantime just
subscribe root.
> I tweaked the sa-update-mirror-check.sh script and it's now
> /usr/local/bin/checkSAupdateMirrors.sh and symlink'd to
> /etc/cron.daily to send this list an email with the status of all the
> mirrors.
Good! Can you change it to hourly when you get it tweaked.
Re: SysAdmin Tasklist was Re: Next priority to get running on
sa-vm1.apache.org
Posted by Dave Jones <da...@apache.org>.
On 05/14/2017 08:31 AM, Kevin A. McGrail wrote:
> On 5/14/2017 7:38 AM, Dave Jones wrote:
>> Thank goodness! I took a look at that README and my head started to
>> explode. I am not a perl developer so that was going to be very time
>> consuming to get up to speed on that. I will let you have it... :)
> Well, if you want to learn, I'd love to mentor others. I've tried to
> make it more straightforward each time I edit that document. And the
> interesting things is that it's very multi-disciplinary and more
> sysadminny than perly.
>
I will definitely take you up on this later when we have all of these
other things fixed and working smoothly.
>>> To me, the priority would be getting ruleqa/masscheck better
>>> documented and back up and running would be ideal.
>> I will change to working on this as my next priority.
>
> Cool, there is at least one CGI so getting that website working along
> with rsync for submissions would be my first attack. I believe you are
> a masscheck submitter so you'll know when the rsync password file, etc.
> is working.
>
I am so I will work on this next.
> Once the website works again and the rsync is working, work on what's
> needed to get the scripts firing again.
>
> *IMPORTANT* I've captured a week + a few days of the cron output before
> the server was turned off. This is in /home/kmcgrail/SACron.gz.
>
> It's an mbox format to let us sleuth into things like cron jobs for
> masscheck and rules update that run differently once per week. It also
> will let us look at any errors the system might have been throwing
> before the server switch over so we don't chase down strawmen.
>
> This should be one of the last sources of information that I have that
> isn't shared with the rest of the group well. Going forward, the
> centralized cron logging to the sysadmin mailing list will give us some
> archives of this data. Plus I figure any sysadmin can filter out the
> noise.
>
>> This is referenced in DNS by mirrors.updates.spamassasin.org TXT
>> record pointing to
>> http://spamassassin.apache.org/updates/MIRRORED.BY. Do you have the
>> details on how this file gets updated on that apache.org server so we
>> can add this to the wiki?
> I'd document it in the wiki at least referring to the file as I'm trying
> to make the wiki a single source of truth. The fact that it doesn't
> show up at all is worrisome.
>
> To my knowledge, MIRRORED.BY is hand edited on the server doing the
> hosting itself.
>
> I'm working on instructions to setup your own channel which will nicely
> intersect with this work.
>
>>
>> root email will deliver to this list now. Need someone to setup the
>> root@sa-vm1.apache.org to be allowed without moderation. See test
>> email I recently sent from root to the list.
> OK, see https://www.apache.org/foundation/mailinglists.html where you
> should be a moderator of this list and subscribe away. Assuming
> incoming mail works to root, you can then confirm the subscription. Ask
> if you need help.
Do we want to subscribe root like this? It doesn't need to receive any
of these emails that will just fill up the root mailbox or possibly
create a mail loop. I was thinking about allowing it as a non-member
poster. I am more familiar with Mailman that allows a list of addresses
that can post without being a member. Do you know if there is something
like this available on the ASF lists? If so, I assumed this would be a
Jira task for someone with admin rights to the listserv.
>
> Regards,
> KAM
>
I tweaked the sa-update-mirror-check.sh script and it's now
/usr/local/bin/checkSAupdateMirrors.sh and symlink'd to /etc/cron.daily
to send this list an email with the status of all the mirrors.
Dave
Re: SysAdmin Tasklist was Re: Next priority to get running on
sa-vm1.apache.org
Posted by "Kevin A. McGrail" <km...@apache.org>.
On 5/14/2017 7:38 AM, Dave Jones wrote:
> Thank goodness! I took a look at that README and my head started to
> explode. I am not a perl developer so that was going to be very time
> consuming to get up to speed on that. I will let you have it... :)
Well, if you want to learn, I'd love to mentor others. I've tried to
make it more straightforward each time I edit that document. And the
interesting things is that it's very multi-disciplinary and more
sysadminny than perly.
>> To me, the priority would be getting ruleqa/masscheck better
>> documented and back up and running would be ideal.
> I will change to working on this as my next priority.
Cool, there is at least one CGI so getting that website working along
with rsync for submissions would be my first attack. I believe you are
a masscheck submitter so you'll know when the rsync password file, etc.
is working.
Once the website works again and the rsync is working, work on what's
needed to get the scripts firing again.
*IMPORTANT* I've captured a week + a few days of the cron output before
the server was turned off. This is in /home/kmcgrail/SACron.gz.
It's an mbox format to let us sleuth into things like cron jobs for
masscheck and rules update that run differently once per week. It also
will let us look at any errors the system might have been throwing
before the server switch over so we don't chase down strawmen.
This should be one of the last sources of information that I have that
isn't shared with the rest of the group well. Going forward, the
centralized cron logging to the sysadmin mailing list will give us some
archives of this data. Plus I figure any sysadmin can filter out the noise.
> This is referenced in DNS by mirrors.updates.spamassasin.org TXT
> record pointing to
> http://spamassassin.apache.org/updates/MIRRORED.BY. Do you have the
> details on how this file gets updated on that apache.org server so we
> can add this to the wiki?
I'd document it in the wiki at least referring to the file as I'm trying
to make the wiki a single source of truth. The fact that it doesn't
show up at all is worrisome.
To my knowledge, MIRRORED.BY is hand edited on the server doing the
hosting itself.
I'm working on instructions to setup your own channel which will nicely
intersect with this work.
>
> root email will deliver to this list now. Need someone to setup the
> root@sa-vm1.apache.org to be allowed without moderation. See test
> email I recently sent from root to the list.
OK, see https://www.apache.org/foundation/mailinglists.html where you
should be a moderator of this list and subscribe away. Assuming
incoming mail works to root, you can then confirm the subscription. Ask
if you need help.
Regards,
KAM
--
Kevin A. McGrail
Asst. Treasurer, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
Re: SysAdmin Tasklist was Re: Next priority to get running on
sa-vm1.apache.org
Posted by Dave Jones <da...@apache.org>.
On 05/13/2017 08:02 PM, Kevin A. McGrail wrote:
> RESENDING: Scripts were blocked for security reasons
>
> On 5/13/2017 4:56 PM, Dave Jones wrote:
>> What's the next priority now that the rsync and httpd configs are active?
>>
>> I will work on the build next using this:
>>
>> https://svn.apache.org/repos/asf/spamassassin/trunk/build/README
>
> PREFACE: I'm working on the build. If you would like to help, we need
> to coordinate first.
>
Thank goodness! I took a look at that README and my head started to
explode. I am not a perl developer so that was going to be very time
consuming to get up to speed on that. I will let you have it... :)
> Here is the promised list of items I've identified.
>
> To me, the priority would be getting ruleqa/masscheck better documented
> and back up and running would be ideal.
>
I will change to working on this as my next priority.
> If we can get that system running smoother with a shorter lag to
> publishing rules, I'd like to help more with it.
>
>
> DONE - Touch a file called MIRROR.CHECK in
> /var/www/bbmass.spamassassin.org/updates on SA-VM1 and test if it is
> synced to the Mirrors. NOTE: I sync every 10 mins
>
> - Document on the wiki that MIRRORED.BY contains the sa update mirror
> contact names.
>
This is referenced in DNS by mirrors.updates.spamassasin.org TXT record
pointing to http://spamassassin.apache.org/updates/MIRRORED.BY. Do you
have the details on how this file gets updated on that apache.org server
so we can add this to the wiki?
> - Get the various files for running the sa-update aka bbmass website
> into SVN. This would NOT be the update files but likely everything else
> including the httpd.conf, MIRRORED.BY, etc.
>
> - Get the email to root from sa-vm1 to go to sysadmins@ without
> moderation so we have cron logs, etc. archived.
>
root email will deliver to this list now. Need someone to setup the
root@sa-vm1.apache.org to be allowed without moderation. See test email
I recently sent from root to the list.
> - KAM to Get the passwords for crashplan for SA into sysadmins repo
> encrypted so we have multiple people who have access.
>
> - Get the sa-update-mirror-check script (attached) running on SA-VM1 and
> emailing sysadmins@ without moderation
>
> - Get Darxus' rule update check script (attached) running on SA-VM1 and
> emailing sysadmins@ without moderation. See SA Dev list example: Rule
> updates are too old - 2017-05-08
>
> - Get Darxus' check script updated for 3.4.2 and 3.3.2.
>
> - Perhaps update the sa-update-mirror-check to use the MIRROR.CHECK with
> a timestamp to confirm it's within a reasonable period of time.
>
> - Find out who wrote the sa-update-mirror-check (likely on the list
> archives), check the licensing on the post and hopefully ask who wrote
> it to public domain or Apache license. Then add
> attribution/license/copyright and add it to the sysadmins repo.
>
> - Ask Darxus' if we can repo his script as well with
> attribution/license/copyright as above
>
> - Ask Darxus' to turn off his script that runs on his infrastructure
>
> - Identify what we used to provide on the old servers. Some things KAM
> believes we had that need to be verified and likely expanded on:
>
> o Masscheck RSYNC for people to send us their Masscheck Logs
> o An email system for people to email and it would send the results
> of checking that email
> o Masscheck Corpora RSYNC or perhaps SSH for people to send us their
> corpora for us to run our own Masscheck server. NOTE: This is the most
> sensitive data we would have I believe since it is other people's real
> mail.
> o For the above, I think I myself have this setup. I'd like to
> identify where and extend it / improve it / make sure it's working, etc.
> o Look at the rsync MOTD[1]
> o Masscheck stuff:
> https://wiki.apache.org/spamassassin/NightlyMassCheck - KAM sent notes a
> few days ago about how he got this running on spamassassin-vm. if that
> doesn't suffice, please let me know.
>
Will check this out today and try to get this working on sa-vm1.
> - Identify what jm was using talon1.pccc.com to provide so I can mimic
> it. His cron jobs were disabled last January but I think they were
> running items related to masscheck.
>
> - Get the RuleQA Website running again.
>
Will check this out today.
> - Identify what the incoming.spamassassin.org server did/does/can do for
> us. NOTE: It might be the the same as below.
>
> - Talk to Grant Kellar with Sonic about the traps they have in place and
> where they are sent to make sure we are utilizing them.
>
> - Clean up and remove unnecessary backup data on sa-vm1 - NO NEED TO BE
> HASTY ON THIS, I'M JUST WRITING A COMPLETE LIST.
>
> - Identify how much data we need if Infra can shrink the data storage
> allocated for sa-vm1
>
> - Talk to AXB about SOUGHT and SOUGHT2
>
> - Update the documentation for InfraNotes2017 with another pass of
> updates about machines, etc.
>
>
> [1]
> corpus
> nightly mass-check result upload area. It is password protected.
> If you would like a password, please send a request to
> pmc@spamassassin.apache.org and request a "nightly" username and password.
>
> submit
> Score generation mass-check result upload area. It is password
> protected. If you would like a password, please send a request to
> pmc@spamassassin.apache.org and request a "score generation" username
> and password. Generally these are only granted after a mass-check
> announcement has been made on the spamassassin developer mailing list.
>
> anoncorpus
> mass-check result download area, available via anonymous access.
>
>
> --
> Kevin A. McGrail
> Asst. Treasurer, Apache Software Foundation
> Chair Emeritus Apache SpamAssassin Project
>
Re: SysAdmin Tasklist was Re: Next priority to get running on
sa-vm1.apache.org
Posted by "Kevin A. McGrail" <ke...@mcgrail.com>.
RESENDING: Scripts were blocked for security reasons
On 5/13/2017 4:56 PM, Dave Jones wrote:
> What's the next priority now that the rsync and httpd configs are active?
>
> I will work on the build next using this:
>
> https://svn.apache.org/repos/asf/spamassassin/trunk/build/README
PREFACE: I'm working on the build. If you would like to help, we need
to coordinate first.
Here is the promised list of items I've identified.
To me, the priority would be getting ruleqa/masscheck better documented
and back up and running would be ideal.
If we can get that system running smoother with a shorter lag to
publishing rules, I'd like to help more with it.
DONE - Touch a file called MIRROR.CHECK in
/var/www/bbmass.spamassassin.org/updates on SA-VM1 and test if it is
synced to the Mirrors. NOTE: I sync every 10 mins
- Document on the wiki that MIRRORED.BY contains the sa update mirror
contact names.
- Get the various files for running the sa-update aka bbmass website
into SVN. This would NOT be the update files but likely everything else
including the httpd.conf, MIRRORED.BY, etc.
- Get the email to root from sa-vm1 to go to sysadmins@ without
moderation so we have cron logs, etc. archived.
- KAM to Get the passwords for crashplan for SA into sysadmins repo
encrypted so we have multiple people who have access.
- Get the sa-update-mirror-check script (attached) running on SA-VM1 and
emailing sysadmins@ without moderation
- Get Darxus' rule update check script (attached) running on SA-VM1 and
emailing sysadmins@ without moderation. See SA Dev list example: Rule
updates are too old - 2017-05-08
- Get Darxus' check script updated for 3.4.2 and 3.3.2.
- Perhaps update the sa-update-mirror-check to use the MIRROR.CHECK with
a timestamp to confirm it's within a reasonable period of time.
- Find out who wrote the sa-update-mirror-check (likely on the list
archives), check the licensing on the post and hopefully ask who wrote
it to public domain or Apache license. Then add
attribution/license/copyright and add it to the sysadmins repo.
- Ask Darxus' if we can repo his script as well with
attribution/license/copyright as above
- Ask Darxus' to turn off his script that runs on his infrastructure
- Identify what we used to provide on the old servers. Some things KAM
believes we had that need to be verified and likely expanded on:
o Masscheck RSYNC for people to send us their Masscheck Logs
o An email system for people to email and it would send the results
of checking that email
o Masscheck Corpora RSYNC or perhaps SSH for people to send us their
corpora for us to run our own Masscheck server. NOTE: This is the most
sensitive data we would have I believe since it is other people's real
mail.
o For the above, I think I myself have this setup. I'd like to
identify where and extend it / improve it / make sure it's working, etc.
o Look at the rsync MOTD[1]
o Masscheck stuff:
https://wiki.apache.org/spamassassin/NightlyMassCheck - KAM sent notes a
few days ago about how he got this running on spamassassin-vm. if that
doesn't suffice, please let me know.
- Identify what jm was using talon1.pccc.com to provide so I can mimic
it. His cron jobs were disabled last January but I think they were
running items related to masscheck.
- Get the RuleQA Website running again.
- Identify what the incoming.spamassassin.org server did/does/can do for
us. NOTE: It might be the the same as below.
- Talk to Grant Kellar with Sonic about the traps they have in place and
where they are sent to make sure we are utilizing them.
- Clean up and remove unnecessary backup data on sa-vm1 - NO NEED TO BE
HASTY ON THIS, I'M JUST WRITING A COMPLETE LIST.
- Identify how much data we need if Infra can shrink the data storage
allocated for sa-vm1
- Talk to AXB about SOUGHT and SOUGHT2
- Update the documentation for InfraNotes2017 with another pass of
updates about machines, etc.
[1]
corpus
nightly mass-check result upload area. It is password protected.
If you would like a password, please send a request to
pmc@spamassassin.apache.org and request a "nightly" username and password.
submit
Score generation mass-check result upload area. It is password
protected. If you would like a password, please send a request to
pmc@spamassassin.apache.org and request a "score generation" username
and password. Generally these are only granted after a mass-check
announcement has been made on the spamassassin developer mailing list.
anoncorpus
mass-check result download area, available via anonymous access.
--
Kevin A. McGrail
Asst. Treasurer, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
SysAdmin Tasklist was Re: Next priority to get running on
sa-vm1.apache.org
Posted by "Kevin A. McGrail" <km...@apache.org>.
On 5/13/2017 4:56 PM, Dave Jones wrote:
> What's the next priority now that the rsync and httpd configs are active?
>
> I will work on the build next using this:
>
> https://svn.apache.org/repos/asf/spamassassin/trunk/build/README
PREFACE: I'm working on the build. If you would like to help, we need
to coordinate first.
Here is the promised list of items I've identified.
To me, the priority would be getting ruleqa/masscheck better documented
and back up and running would be ideal.
If we can get that system running smoother with a shorter lag to
publishing rules, I'd like to help more with it.
DONE - Touch a file called MIRROR.CHECK in
/var/www/bbmass.spamassassin.org/updates on SA-VM1 and test if it is
synced to the Mirrors. NOTE: I sync every 10 mins
- Document on the wiki that MIRRORED.BY contains the sa update mirror
contact names.
- Get the various files for running the sa-update aka bbmass website
into SVN. This would NOT be the update files but likely everything else
including the httpd.conf, MIRRORED.BY, etc.
- Get the email to root from sa-vm1 to go to sysadmins@ without
moderation so we have cron logs, etc. archived.
- KAM to Get the passwords for crashplan for SA into sysadmins repo
encrypted so we have multiple people who have access.
- Get the sa-update-mirror-check script (attached) running on SA-VM1 and
emailing sysadmins@ without moderation
- Get Darxus' rule update check script (attached) running on SA-VM1 and
emailing sysadmins@ without moderation. See SA Dev list example: Rule
updates are too old - 2017-05-08
- Get Darxus' check script updated for 3.4.2 and 3.3.2.
- Perhaps update the sa-update-mirror-check to use the MIRROR.CHECK with
a timestamp to confirm it's within a reasonable period of time.
- Find out who wrote the sa-update-mirror-check (likely on the list
archives), check the licensing on the post and hopefully ask who wrote
it to public domain or Apache license. Then add
attribution/license/copyright and add it to the sysadmins repo.
- Ask Darxus' if we can repo his script as well with
attribution/license/copyright as above
- Ask Darxus' to turn off his script that runs on his infrastructure
- Identify what we used to provide on the old servers. Some things KAM
believes we had that need to be verified and likely expanded on:
o Masscheck RSYNC for people to send us their Masscheck Logs
o An email system for people to email and it would send the results
of checking that email
o Masscheck Corpora RSYNC or perhaps SSH for people to send us their
corpora for us to run our own Masscheck server. NOTE: This is the most
sensitive data we would have I believe since it is other people's real
mail.
o For the above, I think I myself have this setup. I'd like to
identify where and extend it / improve it / make sure it's working, etc.
o Look at the rsync MOTD[1]
o Masscheck stuff:
https://wiki.apache.org/spamassassin/NightlyMassCheck - KAM sent notes a
few days ago about how he got this running on spamassassin-vm. if that
doesn't suffice, please let me know.
- Identify what jm was using talon1.pccc.com to provide so I can mimic
it. His cron jobs were disabled last January but I think they were
running items related to masscheck.
- Get the RuleQA Website running again.
- Identify what the incoming.spamassassin.org server did/does/can do for
us. NOTE: It might be the the same as below.
- Talk to Grant Kellar with Sonic about the traps they have in place and
where they are sent to make sure we are utilizing them.
- Clean up and remove unnecessary backup data on sa-vm1 - NO NEED TO BE
HASTY ON THIS, I'M JUST WRITING A COMPLETE LIST.
- Identify how much data we need if Infra can shrink the data storage
allocated for sa-vm1
- Talk to AXB about SOUGHT and SOUGHT2
- Update the documentation for InfraNotes2017 with another pass of
updates about machines, etc.
[1]
corpus
nightly mass-check result upload area. It is password protected.
If you would like a password, please send a request to
pmc@spamassassin.apache.org and request a "nightly" username and password.
submit
Score generation mass-check result upload area. It is password
protected. If you would like a password, please send a request to
pmc@spamassassin.apache.org and request a "score generation" username
and password. Generally these are only granted after a mass-check
announcement has been made on the spamassassin developer mailing list.
anoncorpus
mass-check result download area, available via anonymous access.
--
Kevin A. McGrail
Asst. Treasurer, Apache Software Foundation
Chair Emeritus Apache SpamAssassin Project
Re: Next priority to get running on sa-vm1.apache.org
Posted by Dave Jones <da...@apache.org>.
What's the next priority now that the rsync and httpd configs are active?
I will work on the build next using this:
https://svn.apache.org/repos/asf/spamassassin/trunk/build/README
Dave
On 05/04/2017 10:18 PM, Kevin A. McGrail wrote:
> On 5/3/2017 6:53 PM, Dave Jones wrote:
>> Kevin,
>> I am sorting through the backups in /x1 which is a lot. What do you
>> want me to target next while we finish off the DNS hidden master
>> details? Is it the Masscheck?
>>
> OK, so the next thing I would work on is look for the rsync
> configuration / password file / locations and get that running.
>
> IMPORTANT: don't let the sync's happen until we have the content they
> sync. I don't want the sa-update mirrors to suddenly go blank.
>
> My key goal after DNS is to get it so the process for building a release
> can complete. That's the build/README file in the 3.4 branch.
>
> That's failing at this point.
>
> /usr/bin/perl -e 'print "CPAN_VERSION=3.004002\n"' >> version.env
> --2017-05-04 23:16:19--
> http://buildbot.spamassassin.org/updatestage//1786853.tar.gz
> Resolving buildbot.spamassassin.org (buildbot.spamassassin.org)...
> 140.211.11.80
> Connecting to buildbot.spamassassin.org
> (buildbot.spamassassin.org)|140.211.11.80|:80...
>
> Regards,
> KAM
Re: Next priority to get running on sa-vm1.apache.org
Posted by "Kevin A. McGrail" <ke...@mcgrail.com>.
On 5/3/2017 6:53 PM, Dave Jones wrote:
> Kevin,
> I am sorting through the backups in /x1 which is a lot. What do you
> want me to target next while we finish off the DNS hidden master
> details? Is it the Masscheck?
>
OK, so the next thing I would work on is look for the rsync
configuration / password file / locations and get that running.
IMPORTANT: don't let the sync's happen until we have the content they
sync. I don't want the sa-update mirrors to suddenly go blank.
My key goal after DNS is to get it so the process for building a release
can complete. That's the build/README file in the 3.4 branch.
That's failing at this point.
/usr/bin/perl -e 'print "CPAN_VERSION=3.004002\n"' >> version.env
--2017-05-04 23:16:19--
http://buildbot.spamassassin.org/updatestage//1786853.tar.gz
Resolving buildbot.spamassassin.org (buildbot.spamassassin.org)...
140.211.11.80
Connecting to buildbot.spamassassin.org
(buildbot.spamassassin.org)|140.211.11.80|:80...
Regards,
KAM