Question: My web server (Linux, Debian Wheezy) has suddenly gone read-only, MySQL have crashed (but Apache didn’t) – giving errors when pressing tab to complete file name as:
?
bash: cannot create temp file for here-document: Read-only file system
Also, when I restart Apache it won’t turn on anymore.
What I tried to do : I’ve replaced the hard disk , moved web files again ; error repeated after 1 week.
Logs shows nothing, one notice that I’ve seen many failed attempts to access the server through ssh.
df output:
Filesystem ???1K-blocks ?Used Available Use% Mounted onrootfs ???249773956 2328024 234758164 ?1% /udev ????10240 ?0 ?10240 ?0% /devtmpfs ????406336 ?200 ?406136 ?1% /run/dev/disk/by-uuid/e45e30eb-efa4-4cd9-aaf9-c6cbe46aa41c 249773956 2328024 234758164 ?1% /tmpfs ????5120 ?0 ?5120 ?0% /run/locktmpfs ????2489760 ?0 ?2489760 ?0% /run/shm/dev/sdb1 ???249773956 2303784 234782404 ?1% /mnt/sdb1
mount -n output:
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)udev on /dev type devtmpfs (rw,relatime,size=10240k,nr_inodes=506431,mode=755)devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)tmpfs on /run type tmpfs (rw,nosuid,noexec,relatime,size=406336k,mode=755)/dev/disk/by-uuid/e45e30eb-efa4-4cd9-aaf9-c6cbe46aa41c on / type ext4 (ro,relatime,errors=remount-ro,user_xattr,barrier=1,data=ordered)tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)tmpfs on /run/shm type tmpfs (rw,nosuid,nodev,noexec,relatime,size=2489760k)rpc_pipefs on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)/dev/sdb1 on /mnt/sdb1 type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered)
Answer: Your root file system is mounted read-only. ?This likely happened on a reboot. ?There are a few options:
- Configure the system to fix errors during reboot. ?On Ubuntu this is controlled by the FSCKFIX option in the file /etc/default/rcS.
- Reboot in recovery mode and run fsck -f /dev/disk/by-uuid/e45e30eb-efa4-4cd9-aaf9-c6cbe46aa41c and reboot again.
- Boot from a recovery disk and run fsck as shown above.
All options will require your system to be down for a bit. ?However, it appears it is effectively down now.
It may be possible to run fsck without rebooting and remount the system rw. ?I would run a test fsck without enabling fixing problems to see how broken the file system