From predragp at andrew.cmu.edu Mon Dec 1 13:23:03 2014 From: predragp at andrew.cmu.edu (predragp at andrew.cmu.edu) Date: Mon, 1 Dec 2014 13:23:03 -0500 Subject: neill-zfs scheduled maintenance Message-ID: Dear Autonians, Neill-zfs is now upgraded and rebooted. Please report ASAP if you notice something unusual. Predrag From predragp at andrew.cmu.edu Wed Dec 10 13:42:04 2014 From: predragp at andrew.cmu.edu (predragp at andrew.cmu.edu) Date: Wed, 10 Dec 2014 13:42:04 -0500 Subject: LOT2 RAID6 rebuilt Message-ID: <0b1b56302fa9233b2ab0db9d5c69bc81.squirrel@webmail.andrew.cmu.edu> Dear Autonians, I was able to get two replacement Seagate HDD at no cost to lab. RAID 6 on LOT2 aka. /xfsdata is rebuilt. I could not repair the RAID 6 because due to the rebuilt process another drive was marked as bad (it is actually good). That means that your /home/sratch directories have been cleaned. Predrag From predragp at andrew.cmu.edu Sat Dec 13 12:31:15 2014 From: predragp at andrew.cmu.edu (predragp at andrew.cmu.edu) Date: Sat, 13 Dec 2014 12:31:15 -0500 Subject: Kerberos tickets and AFS tokens demystified Message-ID: Dear Jeff, First off I would like to apologize for wasting your time with improperly set AFS. After I left yesterday I got mad at myself for not taking the time to truly understand how Kerberos and AFS work so I did some experimenting and reading. I would like to summarize things in few short paragraphs which will hopefully help you and other Auton Lab affiliates to use AFS more efficiently. CMU SCS uses Kerberos, an authentication protocol which works on the basis of 'tickets', to allow access to its infrastructure over hostile network. kinit command (part of krb5-workstation package on Red Hat) obtains the master Kerberos ticket. You should type this command first whenever you want to use AFS. klist shows you all of your tickets and tells you when they will expire. In my understanding CMU Kerberos server will issue tickets up to 30 days. Having a Kerberos ticket is not sufficient to grant you access to your files on AFS! AFS does secure authentication through tokens. You get token after obtaining Kerberos ticket first by typing klog.krb5 @CS.CMU.EDU -c cs.cmu.edu -k CS.CMU.EDU (alternatively you can also use aklog command). The above command is part of openafs-krb5 package. An AFS token is a Kerberos ticket for the AFS service, stored in the kernel file system layer. The > tokens command will show you your current AFS tokens and when they expire. CMU SCS issues tokens for up to 25 hours. That creates problems as you found out when your data stored on AFS is used by computer programs which run longer than 25h. > krenew command renews an existing renewable Kerbers tickets but more importantly it has an important switch -t to run external program like aklog and obtain/renew AFS taken. If you want to run programs which will use AFS for longer than 25 hours you should run krenew command as a demon. Finally the fact that you were able to use klog.krb5 alone on your works station to access CS.CMU.EDU AFS tells me that either their server is misconfigured or that they were automatically assuming that all machines on the domain name cs.cmu.edu are Kerberised. Since our computing nodes are not the part of cs.cmu.edu domain you have to use both kinit and krb5.klog commands. Few other remarks for other Auton Lab members. Only people with CS.CMU.EDU accounts (cost $100 a month) have access to AFS and other network services (like personal web page hosting) provided by the school of computer science. If your LDAP username like in my case is different than your CS account name you will have to get Kerberos ticket as kinit account at CS.CMU.EDU AFS requires kernel module which has to be identical to the kernel I am running. As of yesterday I have OpenAFS kernel modules available for all Red Hat kernels including the latest 2.6.32-504.1.3 which currently runs on LOT2 for example. AFS is unforgiving when it comes to network connection. If service openafs-client start command is issued at the moment when AFS servers (note plural as we are talking about distributed file system here) not available (use many UDP ports in range 7000-7010) AFS daemon afsd will become zombie process. Such a zombie process can be killed only by rebooting the machine. Due to my lack of understanding of Kerberos and AFS I have such zombie processes running on several computing nodes which will have to be rebooted if you guys want to use AFS. Most Kind Regards, Predrag Punosevac From predragp at andrew.cmu.edu Tue Dec 16 16:26:55 2014 From: predragp at andrew.cmu.edu (predragp at andrew.cmu.edu) Date: Tue, 16 Dec 2014 16:26:55 -0500 Subject: LOT2 moving, LOT1 needs reboot Message-ID: <6cde34770b830435aa0926803f179e89.squirrel@webmail.andrew.cmu.edu> Dear Autonians, Due to our contractual agreements LOT2 machine will be relocated from its current location to NREC withing next 24-26h. Please wrap up all the jobs and move your data from scratch directory. I will also have to reboot LOT1 in order to have Andrew File System properly working. Currently LOT2 is only machine which has AFS available. I will give you 2-3 days to wrap up the jobs and reboot on Friday afternoon. Most Kind Regards, Predrag Punosevac From predragp at andrew.cmu.edu Thu Dec 18 12:35:34 2014 From: predragp at andrew.cmu.edu (predragp at andrew.cmu.edu) Date: Thu, 18 Dec 2014 12:35:34 -0500 Subject: LOT2 is moving to NREC Message-ID: Dear Autonians, As announced two days ago LOT2 will be taken off line withing next 35 minutes and moved to NREC. Only designated group of user will have access to the machine. Thank you for your understanding. Predrag From predragp at andrew.cmu.edu Sat Dec 20 12:37:16 2014 From: predragp at andrew.cmu.edu (predragp at andrew.cmu.edu) Date: Sat, 20 Dec 2014 12:37:16 -0500 Subject: LOT1 to be rebooted 12/21/14 Message-ID: <8c10aeffa8e5bc589f8fcbf1030fa031.squirrel@webmail.andrew.cmu.edu> Dear Autonians, I hope everyone is enjoying to holiday season with your family and friends. As noted earlier this week LOT1 has to be rebooted so that I can get AFS working and to be able to pull out eSata controller which is needed for another project. Thank you for your understanding. Predrag From predragp at andrew.cmu.edu Mon Dec 22 11:08:26 2014 From: predragp at andrew.cmu.edu (predragp at andrew.cmu.edu) Date: Mon, 22 Dec 2014 11:08:26 -0500 Subject: Help needed for X2Go In-Reply-To: <52672.79.114.66.155.1419263744.squirrel@webmail.cs.cmu.edu> References: <52672.79.114.66.155.1419263744.squirrel@webmail.cs.cmu.edu> Message-ID: <4b13e87cfad73019fef6edabd9e34641.squirrel@webmail.andrew.cmu.edu> > Hi Predrag, > I hope you don't mind me CC users at autonlab since this is not the first nor the last e-mail I will receive this holiday season with the same question. > Sorry to disturb you during the holidays, but I need some help using X2Go > as I'm working remotely. > > First of all, do I need to set up a desktop environment (Gnome or kde) > before running X2Go or is it already done by default? > You can't neither Gnome nor KDE are installed on any of computing nodes. You can lunch a single application xterm which will in turn allow you to lunch MATLAB for example in GUI mode or RDesktop. The other option is custom desktop using JWM (Joe's own window manager). I think I installed few other window managers but JWM is the easiest to use. > Here's what I tried: > > 1. ran the port forwarding command (see screenshot1) > ssh -L 8080:low1.int.autonlab.org:22 mfiterau at lop1.autonlab.org > Please just use bash.autonlab.org as a proxy to access computing node and forget about port forwarding. bash.autonlab.org might become unavailable when I use it to connect to NREC. I am almost done with NREC so it will be at most couple of hours next couple of days. Predrag > 2. put in localhost and port 8080 in the X2Go client (see screenshot2) > > When I try to connect, I get the following error "Timeout connecting to > localhost" > > Do you know what I may be doing wrong? > Help with this would be highly appreciated and would make my work during > the holidays a lot easier. > > Thank you very much, > > Ina >