From mjbaysek at cs.cmu.edu Tue Jan 3 15:30:05 2012 From: mjbaysek at cs.cmu.edu (Michael J. Baysek) Date: Tue, 03 Jan 2012 15:30:05 -0500 Subject: [auton-users] AUTON System Downtime tomorrow 5pm to Sunday Message-ID: <4F03654D.8040301@cs.cmu.edu> Hi Lab, This mail is to remind you that the core /auton and /neill systems will be down beginning 5pm tomorrow, and lasting into Sunday. Please make local copies of your important files if you will need access to them during the outage, and contact me with any questions. Happy New Year in 2012, Mike -- Michael J. Baysek Systems Analyst Carnegie Mellon University / Auton Lab 412-268-8939 - mjbaysek at cs.cmu.edu http://www.autonlab.org From mjbaysek at cs.cmu.edu Wed Jan 4 16:19:00 2012 From: mjbaysek at cs.cmu.edu (Michael J. Baysek) Date: Wed, 04 Jan 2012 16:19:00 -0500 Subject: [auton-users] AUTON and NEILL Downtime Reminder: Today 5PM Message-ID: <4F04C244.90903@cs.cmu.edu> This is a reminder that all /auton and /neill resources and compute nodes will be down during the FMS power upgrade beginning at 5pm today. The system will continue to be down through Sunday while I perform various administration tasks to the equipment. -- Michael J. Baysek Systems Analyst Carnegie Mellon University / Auton Lab 412-268-8939 - mjbaysek at cs.cmu.edu http://www.autonlab.org From mjbaysek at cs.cmu.edu Sun Jan 8 14:26:28 2012 From: mjbaysek at cs.cmu.edu (Michael J. Baysek) Date: Sun, 08 Jan 2012 14:26:28 -0500 Subject: [auton-users] Auton Downtime Extended Message-ID: <4F09EDE4.9050800@cs.cmu.edu> ---------------------------- AUTON SYSTEM STATUS UPDATE SUNDAY JAN 8, 2012 ---------------------------- Due to some factors encountered during the system maintenance, the system will continue to be unavailable until tomorrow. The www.autonlab.org website and autonlab.org mailing lists are back in service, but all other services are likely to remain down until tomorrow (Monday). From mjbaysek at cs.cmu.edu Mon Jan 9 11:56:50 2012 From: mjbaysek at cs.cmu.edu (Michael J. Baysek) Date: Mon, 09 Jan 2012 11:56:50 -0500 Subject: [auton-users] Auton Downtime Status Message-ID: <4F0B1C52.7030108@cs.cmu.edu> ------------------------------------- AUTON SYSTEM STATUS UPDATE MONDAY JAN 9, 2012 11:51 AM EST ------------------------------------- A corrupt /auton filesystem was discovered late Saturday, and efforts to restore from backup have been underway since Sunday. This event and the subsequent full restore is responsible for the scheduled downtime being extended. Estimated time of completion of the restoration process is 5:00 PM. System availability will resume shortly thereafter. Look for an email later today announcing system availability. Thank you for your patience. From mjbaysek at cs.cmu.edu Mon Jan 9 18:19:28 2012 From: mjbaysek at cs.cmu.edu (Michael J. Baysek) Date: Mon, 09 Jan 2012 18:19:28 -0500 Subject: [auton-users] Auton System Back Online Message-ID: <4F0B7600.8000006@cs.cmu.edu> -------------------------------- AUTON SYSTEM UPDATE MONDAY JAN 12, 2012 18:08 EST -------------------------------- The system is now online, which a few caveats. It is important that you read this mail in it's ENTIRETY. The primary file system on which /auton was stored went corrupt. A complete restore from backups was necessary. I have been working around the clock since late Saturday night to get everything restored. There were a few gaps in backup coverages in a very small subset of directories, but by and large, everything is intact. It is strongly recommended that you check all files that you have modified since Dec 25. You can easily do this by counting the number of days from December 25th until now, and plugging that number into the find command like so: find /auton/home/yourusername -mtime -15 find /auton/userdirs/yourusername -mtime -15 etc And check these files for up-to-date-ness and consistency by loading them and sanity checking them. Also, the last completed backup on /auton/project/* was on November 30th. If you have modified the contents of these directories since then, please let me know so we can formulate a plan to respond to the displaced files. If you notice anything missing or improperly restored, it is vital that you let me know by the end of this week. Please let me know if you have any questions or concerns. -- Michael J. Baysek Systems Analyst Carnegie Mellon University / Auton Lab 412-268-8939 - mjbaysek at cs.cmu.edu http://www.autonlab.org From mjbaysek at cs.cmu.edu Mon Jan 9 18:22:00 2012 From: mjbaysek at cs.cmu.edu (Michael J. Baysek) Date: Mon, 09 Jan 2012 18:22:00 -0500 Subject: [auton-users] Auton System Back Online Message-ID: <4F0B7698.4070608@cs.cmu.edu> [Date was wrong. It felt like the 12th to me...] -------------------------------- AUTON SYSTEM UPDATE MONDAY JAN 09, 2012 18:08 EST -------------------------------- The system is now online, which a few caveats. It is important that you read this mail in it's ENTIRETY. The primary file system on which /auton was stored went corrupt. A complete restore from backups was necessary. I have been working around the clock since late Saturday night to get everything restored. There were a few gaps in backup coverages in a very small subset of directories, but by and large, everything is intact. It is strongly recommended that you check all files that you have modified since Dec 25. You can easily do this by counting the number of days from December 25th until now, and plugging that number into the find command like so: find /auton/home/yourusername -mtime -15 find /auton/userdirs/yourusername -mtime -15 etc And check these files for up-to-date-ness and consistency by loading them and sanity checking them. Also, the last completed backup on /auton/project/* was on November 30th. If you have modified the contents of these directories since then, please let me know so we can formulate a plan to respond to the displaced files. If you notice anything missing or improperly restored, it is vital that you let me know by the end of this week. Please let me know if you have any questions or concerns. -- Michael J. Baysek Systems Analyst Carnegie Mellon University / Auton Lab 412-268-8939 - mjbaysek at cs.cmu.edu http://www.autonlab.org From mjbaysek at cs.cmu.edu Tue Jan 17 14:09:13 2012 From: mjbaysek at cs.cmu.edu (Michael J. Baysek) Date: Tue, 17 Jan 2012 14:09:13 -0500 Subject: [auton-users] REMINDER: WEH Network Outage Scheduled, January 18th 2012 In-Reply-To: <000a01ccd543$f3783bd0$da68b370$@cs.cmu.edu> References: <000a01ccd543$f3783bd0$da68b370$@cs.cmu.edu> Message-ID: <4F15C759.7040601@cs.cmu.edu> Hi Lab. Please be advised that this SCS network outage may disrupt any connections you have to our central computing resources between 6 and 8 tomorrow AM. Best, Mike -------- Original Message -------- Subject: REMINDER: WEH Network Outage Scheduled, January 18th 2012 Date: Tue, 17 Jan 2012 13:15:10 -0500 From: Help Desk To: Help Desk Day/Date: Wednesday, January 18, 2012 Time: 6:00AM - 8:00AM Service Affected: Wean Hall Network Details: On Wednesday, January 18, 2012 SCS Computing Facilities staff will perform routing changes on the network equipment that provides network connectivity for Wean Hall. There will be a 5-10 minute network outage during the scheduled maintenance period in Wean Hall when the routing changes are implemented. Please contact the SCS Help Desk at x8-4231 or send mail to help+ at cs.cmu.edu with any questions or concerns regarding this maintenance period. Thank you for your attention, SCS Help Desk -------------- next part -------------- An HTML attachment was scrubbed... URL: