From chiragn at cs.cmu.edu Mon Sep 2 14:48:35 2019 From: chiragn at cs.cmu.edu (Chirag Nagpal) Date: Mon, 2 Sep 2019 14:48:35 -0400 Subject: Ice cream in the fridge Message-ID: As part of my relentless pursuit to advance Auton Lab interests within SCS, I have stocked the 3rd floor fridge with Ice Cream from the last SCS Dec5 TG. Please help yourself :) -- *Chirag Nagpal* Graduate Student, Language Technologies Institute School of Computer Science Carnegie Mellon University cs.cmu.edu/~chiragn -------------- next part -------------- An HTML attachment was scrubbed... URL: From awd at cs.cmu.edu Tue Sep 3 06:20:29 2019 From: awd at cs.cmu.edu (Artur Dubrawski) Date: Tue, 3 Sep 2019 06:20:29 -0400 Subject: Ice cream in the fridge In-Reply-To: References: Message-ID: This is whom we call a Good Citizen! Thanks Chirag. Artur On Mon, Sep 2, 2019 at 2:49 PM Chirag Nagpal wrote: > As part of my relentless pursuit to advance Auton Lab interests within > SCS, I have stocked the 3rd floor fridge with Ice Cream from the last SCS > Dec5 TG. > > Please help yourself :) > > > -- > > *Chirag Nagpal* Graduate Student, Language Technologies Institute > School of Computer Science > Carnegie Mellon University > cs.cmu.edu/~chiragn > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeff4 at andrew.cmu.edu Tue Sep 3 16:07:26 2019 From: jeff4 at andrew.cmu.edu (Jeff Schneider) Date: Tue, 3 Sep 2019 16:07:26 -0400 Subject: welcome new Auton Lab postdoc Ramina Ghods! Message-ID: <577d874c-3b4f-5cbe-5bae-b44d39d485b2@andrew.cmu.edu> Hi Everyone, Ramina comes to us from Cornell and before that Iran and her office is now in NSH 3122.? Please drop by and say hello to her when you're in the area. Jeff. From predragp at andrew.cmu.edu Wed Sep 4 12:42:56 2019 From: predragp at andrew.cmu.edu (Predrag Punosevac) Date: Wed, 4 Sep 2019 12:42:56 -0400 Subject: using /tmp for cashing Message-ID: gpu10 probably has few days to live before it will needed be to reboot. Cachins is a really good idea. However using /tmp for chasing which is limited by the size of the root partition typically to about 10-20 GB depends on the application installed will sooner or later result in problems. Cheers, Predrag From ngisolfi at cs.cmu.edu Thu Sep 5 09:41:08 2019 From: ngisolfi at cs.cmu.edu (Nick Gisolfi) Date: Thu, 5 Sep 2019 09:41:08 -0400 Subject: [Lunch] Today 12-1pm @ NSH 4513 Message-ID: <278595C9-C393-4678-B506-7FB40C2FC60F@cs.cmu.edu> Hi Everyone, Lab lunch will be 12-1pm in NSH 4513 again. (We have this room reserved at this time up through December 12th ? Thank you Karen W!) Bring your meal and we?ll see you there! - Nick From predragp at andrew.cmu.edu Thu Sep 5 11:22:53 2019 From: predragp at andrew.cmu.edu (Predrag Punosevac) Date: Thu, 5 Sep 2019 11:22:53 -0400 Subject: Connection to lo1 In-Reply-To: References: Message-ID: It works now. Thank for bringing this to my attention. I applied some patches late last night and apparently home directories were not mounted properly after the reboot. Best, Predrag On Thu, Sep 5, 2019 at 8:44 AM Vincent Jeanselme wrote: > > Good morning, > > I am trying to connect to lop1 and it no longer recognizes my ssh keys ? > Did the config of the machine change ? > > Thank you, > > Vincent > > -- > Vincent Jeanselme > ----------------- > Analyst Researcher > Auton Lab - Robotics Institute > Carnegie Mellon University > From predragp at andrew.cmu.edu Mon Sep 9 20:38:40 2019 From: predragp at andrew.cmu.edu (Predrag Punosevac) Date: Mon, 09 Sep 2019 20:38:40 -0400 Subject: Is bash.autonlab.org down? In-Reply-To: References: Message-ID: <20190910003840.PsbE8JJNn%predragp@andrew.cmu.edu> Hitesh Arora wrote: > Hi Predrag, > > Unable to ssh. Can you please check? > > We needed some results for a meeting now. Sorry I went to Pitt at 3:30 PM and before that everything was working OK. I can confirm your problem report. People will have to use lop2.autonlab.org or lop1.autonlab.org if they have one of legacy accounts (home directory zfsauton or neill) until I reboot machine tomorrow. I apologize for the inconvenience. Predrag > > Thanks, > Hitesh From predragp at andrew.cmu.edu Tue Sep 10 22:08:13 2019 From: predragp at andrew.cmu.edu (Predrag Punosevac) Date: Tue, 10 Sep 2019 22:08:13 -0400 Subject: bash down again Message-ID: bash.autonlab.org just crashed so people will have to use lop2.autonlab.org or lop1.autonlab.org (only legacy accounts). I will reboot it tomorrow but I have a bad feeling that it is being used for what is not designed. Bash is only a gateway machine and should not be used for any kind of computing. Also we have special procedures in place for moving large datasets (single files larger than 4GB). Log files and mouse traps will be checked this time around. Cheers, Predrag From ngisolfi at cs.cmu.edu Thu Sep 12 11:57:23 2019 From: ngisolfi at cs.cmu.edu (Nick Gisolfi) Date: Thu, 12 Sep 2019 11:57:23 -0400 Subject: [Lunch] now in NSH 4513 Message-ID: <0F3070B4-0B1D-4AD8-AE98-3EFBC60827B5@cs.cmu.edu> Hi Everyone, Sorry for last minute notice. Lab lunch is in NSH 4513 until 1pm. - Nick From predragp at andrew.cmu.edu Thu Sep 12 13:36:13 2019 From: predragp at andrew.cmu.edu (Predrag Punosevac) Date: Thu, 12 Sep 2019 13:36:13 -0400 Subject: MATLAB R2019b released Message-ID: <20190912173613.Slgr_oM2V%predragp@andrew.cmu.edu> Hi Autonians, Just a heads-up. I am aware that MATLAB R2019b has been released yesterday. I have about a month to upgrade all our installation before university annual license expires. The plan is as usual to upgrade servers first and desktops of heavy MATLAB users. Everyone else is on the back burner. If you need my credentials to activate new installation on your personal laptop stop by my office. Predrag From shulij at andrew.cmu.edu Fri Sep 13 10:54:08 2019 From: shulij at andrew.cmu.edu (Shuli Jiang) Date: Fri, 13 Sep 2019 10:54:08 -0400 Subject: Is bash down again? Message-ID: Hi! Predrag, It seems bash might have some problems again. Could you please take a look? Thanks a lot. Best, Shuli -- *Shuli Jiang* Carnegie Mellon University B.S. Computer Science, 2019 M.S. Computer Science, 2020 -------------- next part -------------- An HTML attachment was scrubbed... URL: From predragp at andrew.cmu.edu Fri Sep 13 14:01:08 2019 From: predragp at andrew.cmu.edu (Predrag Punosevac) Date: Fri, 13 Sep 2019 14:01:08 -0400 Subject: Is bash down again? In-Reply-To: References: Message-ID: Hi Shuli, Yes. Bash had a problem again and was down twice in the past 48h due to the kernel crashes of the new 7.7 release. I worked with developers from Princeton university trying to track down regression. We think it is fixed now and I just rebooted with the patched kernel which hopefully will be more stable. Note that this didn't affect any other machines as they are still running 7.6 release. Note that 7.7 is the last feature "improvements" release of the branch 7. I am looking to upgrade machines to branch 8 once 8.1 is released and more packages are available. Best, Predrag P.S. We have another shell gateway lop2.autonlab.org which was not affected. Also legacy users can use lop1.autonlab.org as well. On Fri, Sep 13, 2019 at 10:54 AM Shuli Jiang wrote: > > Hi! Predrag, > It seems bash might have some problems again. > Could you please take a look? Thanks a lot. > > Best, > Shuli > > -- > Shuli Jiang > Carnegie Mellon University > B.S. Computer Science, 2019 > M.S. Computer Science, 2020 From shulij at andrew.cmu.edu Fri Sep 13 14:04:55 2019 From: shulij at andrew.cmu.edu (Shuli Jiang) Date: Fri, 13 Sep 2019 14:04:55 -0400 Subject: Is bash down again? In-Reply-To: References: Message-ID: Thanks for the update! On Fri, Sep 13, 2019 at 2:01 PM Predrag Punosevac wrote: > Hi Shuli, > > Yes. Bash had a problem again and was down twice in the past 48h due > to the kernel crashes of the new 7.7 release. I worked with developers > from Princeton university trying to track down regression. We think it > is fixed now and I just rebooted with the patched kernel which > hopefully will be more stable. Note that this didn't affect any other > machines as they are still running 7.6 release. Note that 7.7 is the > last feature "improvements" release of the branch 7. I am looking to > upgrade machines to branch 8 once 8.1 is released and more packages > are available. > > Best, > Predrag > > P.S. We have another shell gateway lop2.autonlab.org which was not > affected. Also legacy users can use lop1.autonlab.org as well. > > On Fri, Sep 13, 2019 at 10:54 AM Shuli Jiang > wrote: > > > > Hi! Predrag, > > It seems bash might have some problems again. > > Could you please take a look? Thanks a lot. > > > > Best, > > Shuli > > > > -- > > Shuli Jiang > > Carnegie Mellon University > > B.S. Computer Science, 2019 > > M.S. Computer Science, 2020 > -- *Shuli Jiang* Carnegie Mellon University B.S. Computer Science, 2019 M.S. Computer Science, 2020 -------------- next part -------------- An HTML attachment was scrubbed... URL: From sarveshj at andrew.cmu.edu Wed Sep 18 18:04:41 2019 From: sarveshj at andrew.cmu.edu (Sarveshwaran Jayaraman) Date: Wed, 18 Sep 2019 22:04:41 +0000 Subject: Auton Lab Welcome Documentation Message-ID: <29cb203a7db24ff0bfc1379e8e5e8db5@andrew.cmu.edu> Hi All, Please find attached a document detailing initial steps involved in settings up your compute evironment as part of Auton Lab ecosystem. It also includes other additional information which may smoothen the learning curve and ease the initial transition phase. These steps are what I gathered from multiple people who helped me - in particular I would like to give a big shout out to Nick and Vincent. Again, this is a work in progress and would require the efforts of everyone to make it more comprehensive. If you are interested in contributing please use the link provided to add more information and comments. Please feel free to share your thoughts. Thanks! Link to Document: https://cmu.box.com/s/9tjbstrnz9s2scd3r5u03iu33esmoicx [1562005799537] Sarvesh Jayaraman Sr. Research Analyst, Auton Lab Carnegie Mellon University Mob: +1-240-893-4287 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: OutlookEmoji-1562005799537fe8c241e-43a6-4a66-ab23-18698a06d1ac.png Type: image/png Size: 5461 bytes Desc: OutlookEmoji-1562005799537fe8c241e-43a6-4a66-ab23-18698a06d1ac.png URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: Welcome_Kit_Auton_Lab.pdf Type: application/pdf Size: 490220 bytes Desc: Welcome_Kit_Auton_Lab.pdf URL: From pnspence at andrew.cmu.edu Wed Sep 18 19:03:52 2019 From: pnspence at andrew.cmu.edu (Patricia N Spencer) Date: Wed, 18 Sep 2019 23:03:52 +0000 Subject: Auton Lab Welcome Documentation In-Reply-To: <29cb203a7db24ff0bfc1379e8e5e8db5@andrew.cmu.edu> References: <29cb203a7db24ff0bfc1379e8e5e8db5@andrew.cmu.edu> Message-ID: <74f67ef809524c4bae33ff33db0898a0@andrew.cmu.edu> Thank you everyone for this effort!! ? This will be a huge help to new members of the lab AND keeping it updated will be key. Trish From: Autonlab-users On Behalf Of Sarveshwaran Jayaraman Sent: Wednesday, September 18, 2019 6:05 PM To: users at autonlab.org Subject: Auton Lab Welcome Documentation Hi All, Please find attached a document detailing initial steps involved in settings up your compute evironment as part of Auton Lab ecosystem. It also includes other additional information which may smoothen the learning curve and ease the initial transition phase. These steps are what I gathered from multiple people who helped me - in particular I would like to give a big shout out to Nick and Vincent. Again, this is a work in progress and would require the efforts of everyone to make it more comprehensive. If you are interested in contributing please use the link provided to add more information and comments. Please feel free to share your thoughts. Thanks! Link to Document: https://cmu.box.com/s/9tjbstrnz9s2scd3r5u03iu33esmoicx [1562005799537] Sarvesh Jayaraman Sr. Research Analyst, Auton Lab Carnegie Mellon University Mob: +1-240-893-4287 -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.png Type: image/png Size: 5461 bytes Desc: image001.png URL: From predragp at andrew.cmu.edu Wed Sep 18 22:18:42 2019 From: predragp at andrew.cmu.edu (Predrag Punosevac) Date: Wed, 18 Sep 2019 22:18:42 -0400 Subject: Auton Lab Welcome Documentation In-Reply-To: <29cb203a7db24ff0bfc1379e8e5e8db5@andrew.cmu.edu> References: <29cb203a7db24ff0bfc1379e8e5e8db5@andrew.cmu.edu> Message-ID: IIRC the attachments are stripped from the emails sent to any of our mailing lists due to security reasons with perhaps brainstorming being an exception. Than you so much for your effort. I will make sure this gets distributed and you get the credit for your work. Best, Predrag On Wed, Sep 18, 2019 at 6:06 PM Sarveshwaran Jayaraman < sarveshj at andrew.cmu.edu> wrote: > Hi All, > > > Please find attached a document detailing initial steps involved in > settings up your compute evironment as part of Auton Lab ecosystem. It also > includes other additional information which may smoothen the learning curve > and ease the initial transition phase. These steps are what I gathered from > multiple people who helped me - in particular I would like to give a big > shout out to Nick and Vincent. > > > Again, this is a work in progress and would require the efforts of > everyone to make it more comprehensive. If you are interested in > contributing please use the link provided to add more information and > comments. > > > Please feel free to share your thoughts. Thanks! > > > Link to Document: https://cmu.box.com/s/9tjbstrnz9s2scd3r5u03iu33esmoicx > > > [image: 1562005799537] > > Sarvesh Jayaraman > Sr. Research Analyst, Auton Lab > Carnegie Mellon University > Mob: +1-240-893-4287 > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: OutlookEmoji-1562005799537fe8c241e-43a6-4a66-ab23-18698a06d1ac.png Type: image/png Size: 5461 bytes Desc: not available URL: From awd at cs.cmu.edu Thu Sep 19 14:56:57 2019 From: awd at cs.cmu.edu (Artur Dubrawski) Date: Thu, 19 Sep 2019 14:56:57 -0400 Subject: Maria De-Arteaga in Motherboard article Message-ID: Dear Autonians, Maria has been portrayed as one of the thought leaders of the fairness frontier of applied machine learning in this article in Motherboard (a tech vertical of Vice). She talks about biases in word embeddings and their impact on e.g. automated recruiting. If you are curious to learn more about Maria's exciting work in this area, try this: https://www.researchgate.net/publication/330264859_Bias_in_Bios_A_Case_Study_of_Semantic_Representation_Bias_in_a_High-Stakes_Setting Congrats Maria! Artur -------------- next part -------------- An HTML attachment was scrubbed... URL: From rk2x at cmu.edu Thu Sep 19 22:10:30 2019 From: rk2x at cmu.edu (Ramayya Krishnan) Date: Fri, 20 Sep 2019 02:10:30 +0000 Subject: Maria De-Arteaga in Motherboard article In-Reply-To: References: Message-ID: Congrats, Maria! From: Artur Dubrawski Sent: Thursday, September 19, 2019 2:57 PM To: users at autonlab.org Cc: Martial Hebert ; Srinivasa G Narasimhan ; Ramayya Krishnan ; Byron G Spice Subject: Maria De-Arteaga in Motherboard article Dear Autonians, Maria has been portrayed as one of the thought leaders of the fairness frontier of applied machine learning in this article in Motherboard (a tech vertical of Vice). She talks about biases in word embeddings and their impact on e.g. automated recruiting. If you are curious to learn more about Maria's exciting work in this area, try this: https://www.researchgate.net/publication/330264859_Bias_in_Bios_A_Case_Study_of_Semantic_Representation_Bias_in_a_High-Stakes_Setting Congrats Maria! Artur -------------- next part -------------- An HTML attachment was scrubbed... URL: From bspice at cs.cmu.edu Fri Sep 20 12:12:39 2019 From: bspice at cs.cmu.edu (Byron G Spice) Date: Fri, 20 Sep 2019 16:12:39 +0000 Subject: Maria De-Arteaga in Motherboard article In-Reply-To: References: Message-ID: <01ab480c23a84a8b98087687040355a8@cs.cmu.edu> Thanks for the heads up https://twitter.com/SCSatCMU/status/1175062057772900353 Byron Byron Spice 412 268 9068 From: Artur Dubrawski Sent: Thursday, September 19, 2019 2:57 PM To: users at autonlab.org Cc: Martial Hebert ; Srinivasa G Narasimhan ; Ramayya Krishnan ; Byron G Spice Subject: Maria De-Arteaga in Motherboard article Dear Autonians, Maria has been portrayed as one of the thought leaders of the fairness frontier of applied machine learning in this article in Motherboard (a tech vertical of Vice). She talks about biases in word embeddings and their impact on e.g. automated recruiting. If you are curious to learn more about Maria's exciting work in this area, try this: https://www.researchgate.net/publication/330264859_Bias_in_Bios_A_Case_Study_of_Semantic_Representation_Bias_in_a_High-Stakes_Setting Congrats Maria! Artur -------------- next part -------------- An HTML attachment was scrubbed... URL: From awd at cs.cmu.edu Tue Sep 24 19:57:32 2019 From: awd at cs.cmu.edu (Artur Dubrawski) Date: Tue, 24 Sep 2019 19:57:32 -0400 Subject: Auton Lab in Space Message-ID: ...in a great company of colleagues from RI and CEE :) https://www.webwire.com/ViewPressRel.asp?aId=247473 -------------- next part -------------- An HTML attachment was scrubbed... URL: From predragp at andrew.cmu.edu Tue Sep 24 23:53:32 2019 From: predragp at andrew.cmu.edu (Predrag Punosevac) Date: Tue, 24 Sep 2019 23:53:32 -0400 Subject: MATLAB R2019b update Message-ID: Dear Autonians, A quick update on the status of MATLAB upgrade. I installed R2019b on gpu13 and gpu14 and it appears to be fully functional. I can probe NVidia with gpuDevice command from MATLAB shell. I am pushing installation image as I type this message to gpu12 and gpu11. Replacing MATLAB on all servers will take a few days due to the size image I am pushing over the network. Cheers, Predrag From anirudds at andrew.cmu.edu Wed Sep 25 00:06:55 2019 From: anirudds at andrew.cmu.edu (Aniruddh Shetty) Date: Wed, 25 Sep 2019 00:06:55 -0400 Subject: Some Basic Doubts Message-ID: What is the fastest way to get training data on the gpu server? I wanted to install kitti dataset. I tried doing wget directly on the remote server (which was very slow). I tried downloading on local machine and then scp to the remote machine (also very slow). Took me around 10 hours to do it Is nvcc not installed on the gpu14 server? The code I was using had some custom gpu kernel codes written which needed cuda toolkit. I have installed it in virtual env for now. -------------- next part -------------- An HTML attachment was scrubbed... URL: From boecking at andrew.cmu.edu Wed Sep 25 07:29:37 2019 From: boecking at andrew.cmu.edu (Benedikt Boecking) Date: Wed, 25 Sep 2019 07:29:37 -0400 Subject: Some Basic Doubts In-Reply-To: References: Message-ID: You want to make sure that you download large datasets onto a server's local scratch directory: /home/scratch/anirudds/ Downloading a large dataset onto the network filesystem (zfs) where your home directory resides is slow and slows the system down for all other users as well. On Wed, Sep 25, 2019, 12:11 AM Aniruddh Shetty wrote: > What is the fastest way to get training data on the gpu server? > I wanted to install kitti dataset. I tried doing wget directly on the > remote server (which was very slow). I tried downloading on local machine > and then scp to the remote machine (also very slow). Took me around 10 > hours to do it > > Is nvcc not installed on the gpu14 server? The code I was using had some > custom gpu kernel codes written which needed cuda toolkit. I have installed > it in virtual env for now. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From boecking at andrew.cmu.edu Wed Sep 25 08:46:51 2019 From: boecking at andrew.cmu.edu (Benedikt Boecking) Date: Wed, 25 Sep 2019 08:46:51 -0400 Subject: Some Basic Doubts In-Reply-To: References: Message-ID: <96127682-185B-4563-990A-6E06F9FDF791@andrew.cmu.edu> And as Vincent rightfully pointed out to me, for datasets that are too large for the scratch directory please contact Predrag. He will download the dataset for you and mount it somewhere. If you want to check home much space is available, you can run $ df -h . In the directory you want to download the dataset into. > On Sep 25, 2019, at 7:29 AM, Benedikt Boecking wrote: > > You want to make sure that you download large datasets onto a server's local scratch directory: /home/scratch/anirudds/ > > Downloading a large dataset onto the network filesystem (zfs) where your home directory resides is slow and slows the system down for all other users as well. > > On Wed, Sep 25, 2019, 12:11 AM Aniruddh Shetty > wrote: > What is the fastest way to get training data on the gpu server? > I wanted to install kitti dataset. I tried doing wget directly on the remote server (which was very slow). I tried downloading on local machine and then scp to the remote machine (also very slow). Took me around 10 hours to do it > > Is nvcc not installed on the gpu14 server? The code I was using had some custom gpu kernel codes written which needed cuda toolkit. I have installed it in virtual env for now. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ngisolfi at cs.cmu.edu Thu Sep 26 10:18:46 2019 From: ngisolfi at cs.cmu.edu (Nick Gisolfi) Date: Thu, 26 Sep 2019 10:18:46 -0400 Subject: [Lunch] Today -- Emiliano's -- Meet @ NSH Entrance Bridge Message-ID: <8CF908FB-4AA5-439F-BECE-0229114B9C9E@cs.cmu.edu> Hi Everyone, Let?s try something different for lab lunch this week! Emiliano?s is a Mexican restaurant ~15mins walk away from campus. Let's meet in front of NSH (on the bridge) and walk over together at noon. We should be back before 1:30pm. To be clear, lunch will not be payed for by the lab. Sorry for the last minute notice for those of you who pack lunch. We hope to see you at lunch! - Nick