You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by Mitchell Smith <Mi...@cwc.com> on 2016/03/11 20:45:57 UTC
[users@httpd] No Space Left on Device and Couldn't perform atomic initialization
Errors
Hi
I have a strange issue with Apache (2.2.23), fronting a svn repository
Sometimes the system runs perfect for months between issues, others it's a matter of days. Could anyone shed any light on whats could be causing this issue. I've found many posts online where people raise the issue, but none where its actually tracked down to its root cause and resoloution. Most seem to just result is 'Restarted it and it worked'
When the disk has over 100GB spare on the SVN storage disk, and 10GB tmpfs /tmp spare.... I get the following error logged like:
[Tue Mar 08 18:31:31 2016] [error] [client 172.21.73.66] A failure occurred while driving the update report editor [500, #28]
[Tue Mar 08 18:31:31 2016] [error] [client 172.21.73.66] Can't set position pointer in file '/tmp/svn-5t9C5Y': No space left on device [500, #28]
[Tue Mar 08 18:31:31 2016] [error] [client 172.21.73.66] Can't close file '/tmp/svn-5t9C5Y': No space left on device [500, #28]
[Tue Mar 08 18:36:50 2016] [error] [client 192.168.228.33] A failure occurred while driving the update report editor [500, #28]
[Tue Mar 08 18:36:50 2016] [error] [client 192.168.228.33] Can't set position pointer in file '/tmp/svn-JAMh9d': No space left on device [500, #28]
[Tue Mar 08 18:36:50 2016] [error] [client 192.168.228.33] Can't close file '/tmp/svn-JAMh9d': No space left on device [500, #28]
[Tue Mar 08 18:37:01 2016] [error] [client 192.168.228.33] A failure occurred while driving the update report editor [500, #28]
[Tue Mar 08 18:37:01 2016] [error] [client 192.168.228.33] Can't set position pointer in file '/tmp/svn-VmFoHr': No space left on device [500, #28]
[Tue Mar 08 18:37:01 2016] [error] [client 192.168.228.33] Can't close file '/tmp/svn-VmFoHr': No space left on device [500, #28]
[Tue Mar 08 18:38:30 2016] [error] [client 192.168.228.33] A failure occurred while driving the update report editor [500, #28]
[Tue Mar 08 18:38:30 2016] [error] [client 192.168.228.33] Can't set position pointer in file '/tmp/svn-VvPNxp': No space left on device [500, #28]
[Tue Mar 08 18:38:30 2016] [error] [client 192.168.228.33] Can't close file '/tmp/svn-VvPNxp': No space left on device [500, #28]
[Tue Mar 08 18:53:58 2016] [error] [client 192.168.228.33] A failure occurred while driving the update report editor [500, #28]
[Tue Mar 08 18:53:58 2016] [error] [client 192.168.228.33] Can't set position pointer in file '/tmp/svn-6dZ1ca': No space left on device [500, #28]
[Tue Mar 08 18:53:58 2016] [error] [client 192.168.228.33] Can't close file '/tmp/svn-6dZ1ca': No space left on device [500, #28]
[Tue Mar 08 18:58:23 2016] [error] [client 192.168.228.33] A failure occurred while driving the update report editor [500, #28]
[Tue Mar 08 18:58:23 2016] [error] [client 192.168.228.33] Can't set position pointer in file '/tmp/svn-eKJfBS': No space left on device [500, #28]
[Tue Mar 08 18:58:23 2016] [error] [client 192.168.228.33] Can't close file '/tmp/svn-eKJfBS': No space left on device [500, #28]
[Tue Mar 08 18:59:30 2016] [error] [client 192.168.228.33] A failure occurred while driving the update report editor [500, #28]
[Tue Mar 08 18:59:30 2016] [error] [client 192.168.228.33] Can't set position pointer in file '/tmp/svn-121CkB': No space left on device [500, #28]
[Tue Mar 08 18:59:30 2016] [error] [client 192.168.228.33] Can't close file '/tmp/svn-121CkB': No space left on device [500, #28]
[Tue Mar 08 19:02:03 2016] [error] [client 192.168.228.33] A failure occurred while driving the update report editor [500, #28]
[Tue Mar 08 19:02:03 2016] [error] [client 192.168.228.33] Can't set position pointer in file '/tmp/svn-taCjge': No space left on device [500, #28]
[Tue Mar 08 19:02:03 2016] [error] [client 192.168.228.33] Can't close file '/tmp/svn-taCjge': No space left on device [500, #28]
[Tue Mar 08 20:31:23 2016] [error] [client 172.21.73.66] A failure occurred while driving the update report editor [500, #28]
[Tue Mar 08 20:31:23 2016] [error] [client 172.21.73.66] Can't set position pointer in file '/tmp/svn-1CNqAP': No space left on device [500, #28]
[Tue Mar 08 20:31:23 2016] [error] [client 172.21.73.66] Can't close file '/tmp/svn-1CNqAP': No space left on device [500, #28]
[Tue Mar 08 20:40:45 2016] [error] [client 192.168.228.33] A failure occurred while driving the update report editor [500, #28]
[Tue Mar 08 20:40:45 2016] [error] [client 192.168.228.33] Can't set position pointer in file '/tmp/svn-x9g0wN': No space left on device [500, #28]
Which after some time appears to progress to the following error
[Wed Mar 09 18:31:30 2016] [error] [client 172.21.73.66] The state report gatherer could not be created. [500, #200029]
[Wed Mar 09 18:31:30 2016] [error] [client 172.21.73.66] Couldn't perform atomic initialization [500, #200029]
Any help would be greatly appreciated
Thanks
Mitchell
[users@httpd] RE: No Space Left on Device and Couldn't perform atomic
initialization Errors
Posted by Gary Smith <Ga...@holdstead.com>.
Mitchell,
How many indoes do you have free? I ran into this issue a lifetime ago when I had a lot of small small small files that ran me out of inodes without even touching to overall disk space.
[JBDEV01 ~]$ df -i
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/sda1 1905008 41221 1863787 3% /
devtmpfs 83438 313 83125 1% /dev
tmpfs 85655 1 85654 1% /dev/shm
tmpfs 85655 368 85287 1% /run
tmpfs 85655 13 85642 1% /sys/fs/cgroup
/dev/sdb1 1310720 12 1310708 1% /mnt/resource
tmpfs 85655 1 85654 1% /run/user/1000
[jbsam@JBDEV01 ~]$ df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda1 29822848 6241336 22043516 23% /
devtmpfs 333752 0 333752 0% /dev
tmpfs 342620 0 342620 0% /dev/shm
tmpfs 342620 34600 308020 11% /run
tmpfs 342620 0 342620 0% /sys/fs/cgroup
/dev/sdb1 20509264 45084 19399324 1% /mnt/resource
tmpfs 68524 0 68524 0% /run/user/1000
[JBDEV01 ~]$
________________________________
From: Mitchell Smith [Mitchell.Smith@cwc.com]
Sent: Friday, March 11, 2016 11:45 AM
To: users@httpd.apache.org
Subject: [users@httpd] No Space Left on Device and Couldn't perform atomic initialization Errors
Hi
I have a strange issue with Apache (2.2.23), fronting a svn repository
Sometimes the system runs perfect for months between issues, others it’s a matter of days. Could anyone shed any light on whats could be causing this issue. I’ve found many posts online where people raise the issue, but none where its actually tracked down to its root cause and resoloution. Most seem to just result is ‘Restarted it and it worked’
When the disk has over 100GB spare on the SVN storage disk, and 10GB tmpfs /tmp spare…. I get the following error logged like:
[Tue Mar 08 18:31:31 2016] [error] [client 172.21.73.66] A failure occurred while driving the update report editor [500, #28]
[Tue Mar 08 18:31:31 2016] [error] [client 172.21.73.66] Can't set position pointer in file '/tmp/svn-5t9C5Y': No space left on device [500, #28]
[Tue Mar 08 18:31:31 2016] [error] [client 172.21.73.66] Can't close file '/tmp/svn-5t9C5Y': No space left on device [500, #28]
[Tue Mar 08 18:36:50 2016] [error] [client 192.168.228.33] A failure occurred while driving the update report editor [500, #28]
[Tue Mar 08 18:36:50 2016] [error] [client 192.168.228.33] Can't set position pointer in file '/tmp/svn-JAMh9d': No space left on device [500, #28]
[Tue Mar 08 18:36:50 2016] [error] [client 192.168.228.33] Can't close file '/tmp/svn-JAMh9d': No space left on device [500, #28]
[Tue Mar 08 18:37:01 2016] [error] [client 192.168.228.33] A failure occurred while driving the update report editor [500, #28]
[Tue Mar 08 18:37:01 2016] [error] [client 192.168.228.33] Can't set position pointer in file '/tmp/svn-VmFoHr': No space left on device [500, #28]
[Tue Mar 08 18:37:01 2016] [error] [client 192.168.228.33] Can't close file '/tmp/svn-VmFoHr': No space left on device [500, #28]
[Tue Mar 08 18:38:30 2016] [error] [client 192.168.228.33] A failure occurred while driving the update report editor [500, #28]
[Tue Mar 08 18:38:30 2016] [error] [client 192.168.228.33] Can't set position pointer in file '/tmp/svn-VvPNxp': No space left on device [500, #28]
[Tue Mar 08 18:38:30 2016] [error] [client 192.168.228.33] Can't close file '/tmp/svn-VvPNxp': No space left on device [500, #28]
[Tue Mar 08 18:53:58 2016] [error] [client 192.168.228.33] A failure occurred while driving the update report editor [500, #28]
[Tue Mar 08 18:53:58 2016] [error] [client 192.168.228.33] Can't set position pointer in file '/tmp/svn-6dZ1ca': No space left on device [500, #28]
[Tue Mar 08 18:53:58 2016] [error] [client 192.168.228.33] Can't close file '/tmp/svn-6dZ1ca': No space left on device [500, #28]
[Tue Mar 08 18:58:23 2016] [error] [client 192.168.228.33] A failure occurred while driving the update report editor [500, #28]
[Tue Mar 08 18:58:23 2016] [error] [client 192.168.228.33] Can't set position pointer in file '/tmp/svn-eKJfBS': No space left on device [500, #28]
[Tue Mar 08 18:58:23 2016] [error] [client 192.168.228.33] Can't close file '/tmp/svn-eKJfBS': No space left on device [500, #28]
[Tue Mar 08 18:59:30 2016] [error] [client 192.168.228.33] A failure occurred while driving the update report editor [500, #28]
[Tue Mar 08 18:59:30 2016] [error] [client 192.168.228.33] Can't set position pointer in file '/tmp/svn-121CkB': No space left on device [500, #28]
[Tue Mar 08 18:59:30 2016] [error] [client 192.168.228.33] Can't close file '/tmp/svn-121CkB': No space left on device [500, #28]
[Tue Mar 08 19:02:03 2016] [error] [client 192.168.228.33] A failure occurred while driving the update report editor [500, #28]
[Tue Mar 08 19:02:03 2016] [error] [client 192.168.228.33] Can't set position pointer in file '/tmp/svn-taCjge': No space left on device [500, #28]
[Tue Mar 08 19:02:03 2016] [error] [client 192.168.228.33] Can't close file '/tmp/svn-taCjge': No space left on device [500, #28]
[Tue Mar 08 20:31:23 2016] [error] [client 172.21.73.66] A failure occurred while driving the update report editor [500, #28]
[Tue Mar 08 20:31:23 2016] [error] [client 172.21.73.66] Can't set position pointer in file '/tmp/svn-1CNqAP': No space left on device [500, #28]
[Tue Mar 08 20:31:23 2016] [error] [client 172.21.73.66] Can't close file '/tmp/svn-1CNqAP': No space left on device [500, #28]
[Tue Mar 08 20:40:45 2016] [error] [client 192.168.228.33] A failure occurred while driving the update report editor [500, #28]
[Tue Mar 08 20:40:45 2016] [error] [client 192.168.228.33] Can't set position pointer in file '/tmp/svn-x9g0wN': No space left on device [500, #28]
Which after some time appears to progress to the following error
[Wed Mar 09 18:31:30 2016] [error] [client 172.21.73.66] The state report gatherer could not be created. [500, #200029]
[Wed Mar 09 18:31:30 2016] [error] [client 172.21.73.66] Couldn't perform atomic initialization [500, #200029]
Any help would be greatly appreciated
Thanks
Mitchell
________________________________
No virus found in this message.
Checked by AVG - www.avg.com<http://www.avg.com>
Version: 2016.0.7442 / Virus Database: 4540/11789 - Release Date: 03/10/16
Re: [users@httpd] No Space Left on Device and Couldn't perform atomic initialization Errors
Posted by Jim Jagielski <ji...@jaguNET.com>.
Many, many times this is when you fill up on IPCS
(semaphores and/or shared memory).
Use 'ipcs -a' to see what your current limits and usage is.
> On Mar 11, 2016, at 2:45 PM, Mitchell Smith <Mi...@cwc.com> wrote:
>
> Hi
>
> I have a strange issue with Apache (2.2.23), fronting a svn repository
>
> Sometimes the system runs perfect for months between issues, others it’s a matter of days. Could anyone shed any light on whats could be causing this issue. I’ve found many posts online where people raise the issue, but none where its actually tracked down to its root cause and resoloution. Most seem to just result is ‘Restarted it and it worked’
>
>
> When the disk has over 100GB spare on the SVN storage disk, and 10GB tmpfs /tmp spare…. I get the following error logged like:
>
> [Tue Mar 08 18:31:31 2016] [error] [client 172.21.73.66] A failure occurred while driving the update report editor [500, #28]
> [Tue Mar 08 18:31:31 2016] [error] [client 172.21.73.66] Can't set position pointer in file '/tmp/svn-5t9C5Y': No space left on device [500, #28]
> [Tue Mar 08 18:31:31 2016] [error] [client 172.21.73.66] Can't close file '/tmp/svn-5t9C5Y': No space left on device [500, #28]
> [Tue Mar 08 18:36:50 2016] [error] [client 192.168.228.33] A failure occurred while driving the update report editor [500, #28]
> [Tue Mar 08 18:36:50 2016] [error] [client 192.168.228.33] Can't set position pointer in file '/tmp/svn-JAMh9d': No space left on device [500, #28]
> [Tue Mar 08 18:36:50 2016] [error] [client 192.168.228.33] Can't close file '/tmp/svn-JAMh9d': No space left on device [500, #28]
> [Tue Mar 08 18:37:01 2016] [error] [client 192.168.228.33] A failure occurred while driving the update report editor [500, #28]
> [Tue Mar 08 18:37:01 2016] [error] [client 192.168.228.33] Can't set position pointer in file '/tmp/svn-VmFoHr': No space left on device [500, #28]
> [Tue Mar 08 18:37:01 2016] [error] [client 192.168.228.33] Can't close file '/tmp/svn-VmFoHr': No space left on device [500, #28]
> [Tue Mar 08 18:38:30 2016] [error] [client 192.168.228.33] A failure occurred while driving the update report editor [500, #28]
> [Tue Mar 08 18:38:30 2016] [error] [client 192.168.228.33] Can't set position pointer in file '/tmp/svn-VvPNxp': No space left on device [500, #28]
> [Tue Mar 08 18:38:30 2016] [error] [client 192.168.228.33] Can't close file '/tmp/svn-VvPNxp': No space left on device [500, #28]
> [Tue Mar 08 18:53:58 2016] [error] [client 192.168.228.33] A failure occurred while driving the update report editor [500, #28]
> [Tue Mar 08 18:53:58 2016] [error] [client 192.168.228.33] Can't set position pointer in file '/tmp/svn-6dZ1ca': No space left on device [500, #28]
> [Tue Mar 08 18:53:58 2016] [error] [client 192.168.228.33] Can't close file '/tmp/svn-6dZ1ca': No space left on device [500, #28]
> [Tue Mar 08 18:58:23 2016] [error] [client 192.168.228.33] A failure occurred while driving the update report editor [500, #28]
> [Tue Mar 08 18:58:23 2016] [error] [client 192.168.228.33] Can't set position pointer in file '/tmp/svn-eKJfBS': No space left on device [500, #28]
> [Tue Mar 08 18:58:23 2016] [error] [client 192.168.228.33] Can't close file '/tmp/svn-eKJfBS': No space left on device [500, #28]
> [Tue Mar 08 18:59:30 2016] [error] [client 192.168.228.33] A failure occurred while driving the update report editor [500, #28]
> [Tue Mar 08 18:59:30 2016] [error] [client 192.168.228.33] Can't set position pointer in file '/tmp/svn-121CkB': No space left on device [500, #28]
> [Tue Mar 08 18:59:30 2016] [error] [client 192.168.228.33] Can't close file '/tmp/svn-121CkB': No space left on device [500, #28]
> [Tue Mar 08 19:02:03 2016] [error] [client 192.168.228.33] A failure occurred while driving the update report editor [500, #28]
> [Tue Mar 08 19:02:03 2016] [error] [client 192.168.228.33] Can't set position pointer in file '/tmp/svn-taCjge': No space left on device [500, #28]
> [Tue Mar 08 19:02:03 2016] [error] [client 192.168.228.33] Can't close file '/tmp/svn-taCjge': No space left on device [500, #28]
> [Tue Mar 08 20:31:23 2016] [error] [client 172.21.73.66] A failure occurred while driving the update report editor [500, #28]
> [Tue Mar 08 20:31:23 2016] [error] [client 172.21.73.66] Can't set position pointer in file '/tmp/svn-1CNqAP': No space left on device [500, #28]
> [Tue Mar 08 20:31:23 2016] [error] [client 172.21.73.66] Can't close file '/tmp/svn-1CNqAP': No space left on device [500, #28]
> [Tue Mar 08 20:40:45 2016] [error] [client 192.168.228.33] A failure occurred while driving the update report editor [500, #28]
> [Tue Mar 08 20:40:45 2016] [error] [client 192.168.228.33] Can't set position pointer in file '/tmp/svn-x9g0wN': No space left on device [500, #28]
>
> Which after some time appears to progress to the following error
>
> [Wed Mar 09 18:31:30 2016] [error] [client 172.21.73.66] The state report gatherer could not be created. [500, #200029]
> [Wed Mar 09 18:31:30 2016] [error] [client 172.21.73.66] Couldn't perform atomic initialization [500, #200029]
>
>
>
> Any help would be greatly appreciated
>
> Thanks
> Mitchell
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org