From nichol at cmu.edu Sun Feb 2 14:00:25 2003 From: nichol at cmu.edu (nichol) Date: Sun, 2 Feb 2003 14:00:25 -0500 (EST) Subject: [auton-users] Backup Policy In-Reply-To: <200301291915.50092.jmjoseph@andrew.cmu.edu> Message-ID: Jacob, This is great. However, the files I want to backup are always the ones Im working on just now (or the last few weeks). Therefore, I'd have to re-make my backup list every day. That ain't going to happen! Could you add a feature that backs-up all files upto 10GB, starting with the latest? If this was automatic, that would be awesome! Cheers Bob *Hi. *I'd like to announce that we've implemented nightly backups of all users' home *directories. In short, we have the space to back up 10GB per user and store *it for 3 months. * *Here's how you can be backed up: ** If your home directory is under 10GB in size, it will be backed up *automatically. You can check with "du -sh ~". * ** If it is over 10GB, your data WILL NOT BE BACKED UP. You must create a list *of files to be backed up. It should be in the root of your home directory, *called "backup.lst". Each line in "backup.lst" is relative to your home *directory and can specify either a directory or file. Lines beginning with *"#" are comments. For example, here's my backup.lst *(/mnt/BigPapa/home/jacob/backup.lst): *------------------------------------------------------------------------------- *# Jacob's backup.lst * *# some files *backup.lst *idl_linux.alpha.tar.gz * *# some directories *# Note they may be specified with or without a / *backup *.ssh/ *----------------------------------------------------------------------------- * ** Be sure that your backup.lst does not specify more than 10GB of files, or it *will nothing will be backed up. You can check with: du -sc $(cat *backup.lst|grep -v "#") * ** Backups begin at 1 am every night, in alphabetical order of users * ** Full backups are stored on the first of every month. Incrementals occur *every sunday for the current month, and for every day in the current week. * * *While this is primarily in case of catestrophic loss/corruption of our *file-server, we will of course be able to restore your files if you *accidentally delete them, for example. Send such requests to: *admin at autonlab.org . I recommend everyone check the size of their home *directory as soon as possible so as to be included. I will send e-mail to *over-size users. * *If you have any questions, don't hesitate to let me know. * *-Jacob Joseph * -- Bob Nichol 5000 Forbes Ave. Carnegie Mellon 412-268-8068 nichol at cmu.edu From dpelleg+ at cs.cmu.edu Mon Feb 3 17:38:59 2003 From: dpelleg+ at cs.cmu.edu (Dan Pelleg) Date: Mon, 3 Feb 2003 17:38:59 -0500 Subject: [auton-users] software on limey Message-ID: <15934.61315.962053.674784@gs166.sp.cs.cmu.edu> I'm having trouble running stuff on limey: 1. "perl" seems to be version 5.8.0, which breaks some of my code. How can I run an earlier version? There seems to be a v5.6 lying around. 2. Where is gcc? ~/h/kmeans/ >gcc bash: gcc: command not found My PATH is: :/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:/usr/X11R6/bin From jmjoseph at andrew.cmu.edu Mon Feb 3 20:42:13 2003 From: jmjoseph at andrew.cmu.edu (Jacob Joseph) Date: Mon, 3 Feb 2003 20:42:13 -0500 Subject: [auton-users] software on limey In-Reply-To: <15934.61315.962053.674784@gs166.sp.cs.cmu.edu> References: <15934.61315.962053.674784@gs166.sp.cs.cmu.edu> Message-ID: <200302032042.13589.jmjoseph@andrew.cmu.edu> > 1. "perl" seems to be version 5.8.0, which breaks some of my code. How can > I run an earlier version? There seems to be a v5.6 lying around. I think I'd rather not go back to 5.6 simply out of the fear that it will mess up other parts of the system. For example, every time we do an update, it will grab the latest perl unless we explicitly tell it not to. Furthermore, any new systems running redhat, such as the coming athlons, are unlikely to have 5.6 at all. Would it be better for you to upgrade all of the machines to 5.8 and fix the code to run on 5.8 instead? > 2. Where is gcc? /usr/alpha-unknown-linux-gnu/gcc-bin/3.2/gcc We can add a symlink from /usr/bin/gcc, which is where it is on the other alphas. Will this work? ccc is at /usr/bin/ccc FYI. -Jacob From dpelleg+ at cs.cmu.edu Mon Feb 3 21:09:03 2003 From: dpelleg+ at cs.cmu.edu (Dan Pelleg) Date: Mon, 3 Feb 2003 21:09:03 -0500 Subject: [auton-users] software on limey In-Reply-To: <200302032042.13589.jmjoseph@andrew.cmu.edu> References: <15934.61315.962053.674784@gs166.sp.cs.cmu.edu> <200302032042.13589.jmjoseph@andrew.cmu.edu> Message-ID: <15935.8383.419856.341190@toto.wburn> Jacob Joseph writes: > > 2. Where is gcc? > /usr/alpha-unknown-linux-gnu/gcc-bin/3.2/gcc > > We can add a symlink from /usr/bin/gcc, which is where it is on the other > alphas. Will this work? > I tried to symlink it from my own ~/bin directory (which is in my path), and that seems to work. So we can try that. I'm not sure how gcc decides which libraries to use, and I'm sure I don't want to know. But we might want to keep this in mind in case we have more gcc versions installed and the symlinks mess things up. [The failure mode I'm alluding to is getting one gcc's libraries linked against code compiled with another] But for now it seems using a symlink should work. At least I'm not aware of anyone running gcc not through the makefiles or via $PATH. > ccc is at /usr/bin/ccc FYI. > Yes, that works well (at least choosing c=ccc in our makefiles does), thanks. From komarek at cmu.edu Mon Feb 3 21:15:06 2003 From: komarek at cmu.edu (Paul Komarek) Date: 03 Feb 2003 21:15:06 -0500 Subject: [Fwd: Re: [auton-users] software on limey] Message-ID: <1044324906.6380.163.camel@tux.home.fake> Oops, I meant for my reply to Dan to go to everyone, but it just went to Dan. That message (about gcc and perl on limey) is attached. -Paul -------------- next part -------------- An embedded message was scrubbed... From: Paul Komarek Subject: Re: [auton-users] software on limey Date: 03 Feb 2003 21:14:08 -0500 Size: 1726 URL: From komarek at cmu.edu Mon Feb 3 21:19:25 2003 From: komarek at cmu.edu (Paul Komarek) Date: 03 Feb 2003 21:19:25 -0500 Subject: [auton-users] software on limey In-Reply-To: <15935.8383.419856.341190@toto.wburn> References: <15934.61315.962053.674784@gs166.sp.cs.cmu.edu> <200302032042.13589.jmjoseph@andrew.cmu.edu> <15935.8383.419856.341190@toto.wburn> Message-ID: <1044325165.20002.167.camel@tux.home.fake> On Mon, 2003-02-03 at 21:09, Dan Pelleg wrote: > Jacob Joseph writes: > > > 2. Where is gcc? > > /usr/alpha-unknown-linux-gnu/gcc-bin/3.2/gcc > > > > We can add a symlink from /usr/bin/gcc, which is where it is on the other > > alphas. Will this work? > > > > I tried to symlink it from my own ~/bin directory (which is in my path), > and that seems to work. So we can try that. I'm not sure how gcc decides > which libraries to use, and I'm sure I don't want to know. But we might > want to keep this in mind in case we have more gcc versions installed and > the symlinks mess things up. [The failure mode I'm alluding to is getting > one gcc's libraries linked against code compiled with another] > gcc chooses it's libs through compiled-in paths and the dynamic linker, I believe. Gentoo's gcc-config program configured /etc/ld.so.conf according to which version of gcc has been selected. It also configures /etc/profile.env or something-or-another that gets sourced by users from the system defaults. If anyone is using tcsh, has not messed with their PATH, and still has trouble with gcc, please let me know. -Paul From pgunn at cs.cmu.edu Tue Feb 4 08:07:09 2003 From: pgunn at cs.cmu.edu (Pat Gunn) Date: Tue, 4 Feb 2003 08:07:09 -0500 (EST) Subject: [auton-users] software on limey In-Reply-To: <200302032042.13589.jmjoseph@andrew.cmu.edu> Message-ID: On Mon, 3 Feb 2003, Jacob Joseph wrote: > > 1. "perl" seems to be version 5.8.0, which breaks some of my code. How can > > I run an earlier version? There seems to be a v5.6 lying around. > > I think I'd rather not go back to 5.6 simply out of the fear that it will mess > up other parts of the system. For example, every time we do an update, it > will grab the latest perl unless we explicitly tell it not to. Furthermore, > any new systems running redhat, such as the coming athlons, are unlikely to > have 5.6 at all. Would it be better for you to upgrade all of the machines > to 5.8 and fix the code to run on 5.8 instead? If you like, just pipe your old perl scripts through me, and I'll give them both a spring (well, winter) cleaning and make them happy on 5.8. I'd love for us to have 5.8 everywhere, and will do whatever I need, even using my spare time at home, to help make that happen. Another side note on limey... limey:~$ uname -a Linux limey.autonlab.org 2.4.21-pre1 #1 SMP Thu Dec 15 03:00:39 Local time zone must be set--see zic alpha EV6 GNU/Linux limey:~$ That's rather interesting, no? :) -- Pat Gunn Research Programmer, Auton Group, CMU From komarek at andrew.cmu.edu Tue Feb 4 15:35:52 2003 From: komarek at andrew.cmu.edu (Paul Komarek) Date: Tue, 4 Feb 2003 15:35:52 -0500 (EST) Subject: [auton-users] funny domains within lab Message-ID: Hi everyone, This is a reminder to use the autonlab.org domain for machines within the AUTON lab. For instance, always refer to limey as limey.autonlab.org, and never as limey.auton.cs.cmu.edu. The cmu.edu name will work from the outside, but it won't work from inside the lab. -Paul Komarek From awm at cs.cmu.edu Mon Feb 24 14:40:20 2003 From: awm at cs.cmu.edu (Andrew W. Moore) Date: Mon, 24 Feb 2003 14:40:20 -0500 Subject: [auton-users] bigpapa connection Message-ID: <20030224194052.8A7DE3F0@mail.autonlab.org> I was running with some of Paul's data in /.automount/lofty/BigPapa/home/komarek/h/pfizer/icml2003_experiments/athlonmp/reuters/mydata/ and all was fine. But then it claimed file not found for a few minutes and now its back again. Is there something like the equivalent of klogging that I should be doing? Thanks, Andrew From dpelleg+ at cs.cmu.edu Mon Feb 24 14:53:48 2003 From: dpelleg+ at cs.cmu.edu (Dan Pelleg) Date: Mon, 24 Feb 2003 14:53:48 -0500 Subject: [auton-users] bigpapa connection In-Reply-To: <20030224194052.8A7DE3F0@mail.autonlab.org> References: <20030224194052.8A7DE3F0@mail.autonlab.org> Message-ID: <15962.30796.507056.563666@gs166.sp.cs.cmu.edu> Andrew W. Moore writes: > I was running with some of Paul's data in > > /.automount/lofty/BigPapa/home/komarek/h/pfizer/icml2003_experiments/athlonmp/reuters/mydata/ > > and all was fine. > > But then it claimed file not found for a few minutes > > and now its back again. > > Is there something like the equivalent of klogging that I should be > doing? Thanks, > > Andrew I got the same thing today. Could be that the automounter unmounted the filesystem after some idle time, and couldn't re-mount it quickly enough when you accessed it again. The re-mount speed is most likely a function of the load on the local network - the fileserver isn't at all loaded. Pat: can we change the automounter defaults so it: 1. waits longer before unmounting an idle connection 2. tries harder and more times to mount a filesystem before deciding it's dead ? As a workaround, try typing "ls /mnt/BigPapa" - for me it convinced the automounter to mount the filesystem again, and then the application was happy. From pgunn at cs.cmu.edu Mon Feb 24 15:16:42 2003 From: pgunn at cs.cmu.edu (Pat Gunn) Date: Mon, 24 Feb 2003 15:16:42 -0500 (EST) Subject: [auton-users] bigpapa connection In-Reply-To: <15962.30796.507056.563666@gs166.sp.cs.cmu.edu> Message-ID: > Pat: can we change the automounter defaults so it: > > 1. waits longer before unmounting an idle connection > > 2. tries harder and more times to mount a filesystem before deciding it's > dead Here's my stab at it, in amd.conf, changed map_options to: map_options = opts:=proto=tcp,nfsvers=3,nfs_retry_interfal=.5,nfs_retransmit_counter=50,cache_duration=3000 If it keeps misbehaving, we can try turning on some of AMD's debug options.. I made this change on liam and lime. -- Pat Gunn Research Programmer, Auton Group, CMU From nichol at cmu.edu Mon Feb 24 15:32:46 2003 From: nichol at cmu.edu (nichol) Date: Mon, 24 Feb 2003 15:32:46 -0500 (EST) Subject: [auton-users] bigpapa connection In-Reply-To: <15962.30796.507056.563666@gs166.sp.cs.cmu.edu> Message-ID: Dont want to scare people, but we have had two RAID array failures up here in Physics. * > I was running with some of Paul's data in * > * > /.automount/lofty/BigPapa/home/komarek/h/pfizer/icml2003_experiments/athlonmp/reuters/mydata/ * > * > and all was fine. * > * > But then it claimed file not found for a few minutes * > * > and now its back again. * > * > Is there something like the equivalent of klogging that I should be * > doing? Thanks, * > * > Andrew * *I got the same thing today. Could be that the automounter unmounted the *filesystem after some idle time, and couldn't re-mount it quickly enough *when you accessed it again. The re-mount speed is most likely a function of *the load on the local network - the fileserver isn't at all loaded. * *Pat: can we change the automounter defaults so it: * * 1. waits longer before unmounting an idle connection * * 2. tries harder and more times to mount a filesystem before deciding it's *dead * *? * * *As a workaround, try typing "ls /mnt/BigPapa" - for me it convinced the *automounter to mount the filesystem again, and then the application was *happy. * -- Bob Nichol 5000 Forbes Ave. Carnegie Mellon 412-268-8068 nichol at cmu.edu From jmjoseph at andrew.cmu.edu Mon Feb 24 18:11:21 2003 From: jmjoseph at andrew.cmu.edu (Jacob Joseph) Date: Mon, 24 Feb 2003 18:11:21 -0500 Subject: [auton-users] bigpapa connection In-Reply-To: References: Message-ID: <200302241811.21500.jmjoseph@andrew.cmu.edu> Bob, can you elaborate? If it was RAID, why was anything lost? Half of the point is to ensure redundancy. -Jacob On Monday 24 February 2003 15:32, nichol wrote: > Dont want to scare people, but we have had two RAID array failures > up here in Physics. > > * > I was running with some of Paul's data in > * > > * > > /.automount/lofty/BigPapa/home/komarek/h/pfizer/icml2003_experiments/athlon >mp/reuters/mydata/ * > > * > and all was fine. > * > > * > But then it claimed file not found for a few minutes > * > > * > and now its back again. > * > > * > Is there something like the equivalent of klogging that I should be > * > doing? Thanks, > * > > * > Andrew > * > *I got the same thing today. Could be that the automounter unmounted the > *filesystem after some idle time, and couldn't re-mount it quickly enough > *when you accessed it again. The re-mount speed is most likely a function > of *the load on the local network - the fileserver isn't at all loaded. * > *Pat: can we change the automounter defaults so it: > * > * 1. waits longer before unmounting an idle connection > * > * 2. tries harder and more times to mount a filesystem before deciding it's > *dead > * > *? > * > * > *As a workaround, try typing "ls /mnt/BigPapa" - for me it convinced the > *automounter to mount the filesystem again, and then the application was > *happy. > * From nichol at cmu.edu Mon Feb 24 19:14:30 2003 From: nichol at cmu.edu (nichol) Date: Mon, 24 Feb 2003 19:14:30 -0500 (EST) Subject: [auton-users] bigpapa connection In-Reply-To: <200302241811.21500.jmjoseph@andrew.cmu.edu> Message-ID: Not when 2 disks go! *Bob, can you elaborate? If it was RAID, why was anything lost? Half of the *point is to ensure redundancy. * *-Jacob * *On Monday 24 February 2003 15:32, nichol wrote: *> Dont want to scare people, but we have had two RAID array failures *> up here in Physics. *> *> * > I was running with some of Paul's data in *> * > *> * > *> /.automount/lofty/BigPapa/home/komarek/h/pfizer/icml2003_experiments/athlon *>mp/reuters/mydata/ * > *> * > and all was fine. *> * > *> * > But then it claimed file not found for a few minutes *> * > *> * > and now its back again. *> * > *> * > Is there something like the equivalent of klogging that I should be *> * > doing? Thanks, *> * > *> * > Andrew *> * *> *I got the same thing today. Could be that the automounter unmounted the *> *filesystem after some idle time, and couldn't re-mount it quickly enough *> *when you accessed it again. The re-mount speed is most likely a function *> of *the load on the local network - the fileserver isn't at all loaded. * *> *Pat: can we change the automounter defaults so it: *> * *> * 1. waits longer before unmounting an idle connection *> * *> * 2. tries harder and more times to mount a filesystem before deciding it's *> *dead *> * *> *? *> * *> * *> *As a workaround, try typing "ls /mnt/BigPapa" - for me it convinced the *> *automounter to mount the filesystem again, and then the application was *> *happy. *> * * -- Bob Nichol 5000 Forbes Ave. Carnegie Mellon 412-268-8068 nichol at cmu.edu