From ngisolfi at cmu.edu Thu May 2 13:10:57 2019 From: ngisolfi at cmu.edu (Nick Gisolfi) Date: Thu, 2 May 2019 13:10:57 -0400 Subject: [Lunch] Friday @ 12:15-1:30PM in NSH 3001 Message-ID: Hi Everyone, I'm sending this week's reminder a day early as there are some new folks in the lab. Welcome!! Bring-your-own-lunch will take place tomorrow @12:15-1:30PM in NSH 3001. Lab lunch is an opportunity to connect with others in the lab in an informal setting. Join us to meet our newest lab members! If you plan to connect remotely: https://meet.google.com/xax-dpdi-nta - Nick -------------- next part -------------- An HTML attachment was scrubbed... URL: From ngisolfi at andrew.cmu.edu Fri May 10 10:09:14 2019 From: ngisolfi at andrew.cmu.edu (Nicholas Gisolfi) Date: Fri, 10 May 2019 10:09:14 -0400 Subject: [Lunch] Today @ 12:15-1:30PM in NSH 3001 Message-ID: Hi Everyone, Lab lunch will be in NSH 3001 from 12:15-1:30 today. Bring your own meal and we'll see you there! If anyone is looking for ideas, I think today is the last day of the Greek food festival near Forbes and Dithridge! https://meet.google.com/xax-dpdi-nta - Nick -------------- next part -------------- An HTML attachment was scrubbed... URL: From predragp at andrew.cmu.edu Mon May 13 13:59:42 2019 From: predragp at andrew.cmu.edu (Predrag Punosevac) Date: Mon, 13 May 2019 13:59:42 -0400 Subject: lov5 rebooted Message-ID: Dear Autonians, lov5 has been rebooted. I do get that NIPS deadline is approaching but nothing will be achieved by overloading servers and running them into the ground. Please try to temper your expectations as otherwise we will have just sequence of setbacks. Cheers, Predrag From predragp at andrew.cmu.edu Mon May 13 16:23:20 2019 From: predragp at andrew.cmu.edu (Predrag Punosevac) Date: Mon, 13 May 2019 16:23:20 -0400 Subject: Caffe, Theano, Torch, TensorFlow on GPU10 Message-ID: Dear Autonians, I have just upgraded NVidia driver on GPU10 to 418.74 as well as libcudnn7 to 7.5.1.10-1 which is compatible with CUDA 10.1 (the default on this server). I have also upgrade MATLAB to 2019a relase. nvidia-smi works as expected as well as MATLAB. gpuDevice() which was broken on the previous release of MATLAB now correctly shows four available GeForce GTX 1081 Ti cards. I would like to help the Auton Lab deep learning community and fix Caffe, Theano, Torch/PyTorch, TensorFlow or whatever are people using these days. I need you to log into the server and try to install your usual tool and test it. I would also appreciate feedback about other 8 gpu servers(GPU7 is reserved for the special project). I know at the very least that GPU3 scratch directory is filled. I am aware of a dead GPU card on GPU4 computing node (I have 1081 Ti on my desk which I can use instead of the dead Titan X on that server). GPU3 perhaps has another dead GPU card. Most Kind Regards, Predrag Punosevac From awd at cs.cmu.edu Mon May 13 16:58:42 2019 From: awd at cs.cmu.edu (Artur Dubrawski) Date: Mon, 13 May 2019 16:58:42 -0400 Subject: Fwd: Thesis Proposal - May 15th - Eric Lei - Multi-View Relationships for Analytics and Inference In-Reply-To: <55afd54f-bd63-150e-6db9-a7ea9a17104c@cmu.edu> References: <55afd54f-bd63-150e-6db9-a7ea9a17104c@cmu.edu> Message-ID: Team, Please come and cheer one of our own when they are making a decisive step towards becoming "Doctor Lei"! It will be practically next door (NSH 3002) at 10am this Wednesday. Artur ---------- Forwarded message --------- From: Diane Stidle Date: Mon, May 13, 2019 at 10:26 AM Subject: Thesis Proposal - May 15th - Eric Lei - Multi-View Relationships for Analytics and Inference To: ml-seminar at cs.cmu.edu , , Mario Berges *Thesis Proposal* Date: May 15, 2019 Time: 10:00am Place: NSH 3002 Speaker: Eric Lei *Title: Multi-View Relationships for Analytics and Inference* Abstract: An interesting area of machine learning is methods for multi-view data. Classical multi-view literature typically exploits multiple views to reduce noise in data via conditional independence. However, the multi-view relationships themselves are underutilized as factors for analyzing the data. In this work, we investigate the usefulness of these relationships in descriptive analytics and inference. In Part I, we cover a problem in radiation detection about inference of latent variables that are shared dependencies of multiple views. We show how the views can be aggregated by filtering their inferences collectively using domain knowledge about their relationships. In Part II, we address the problem of modeling multi-view structure directly and applying such structure to descriptive analytics and inference. We propose a method for radiation detection that learns linear multi-view correlations and detects anomalies when these correlations are disrupted. Next, we extend this idea to nonlinear multi-view correlations by introducing a clustering method whose correlations are cluster-wise linear. This method is applied to perform descriptive analytics on a medical problem. Furthermore, we extend this work to classification and demonstrate it on a load monitoring problem. Lastly, we propose an extension to regression. *Thesis Committee:* Artur Dubrawski (Chair) Barnabas Poczos Mario Berges Simon Labov (Lawrence Livermore National Laboratory) Link to draft thesis document: https://www.dropbox.com/s/uhl0swsodthsbw3/main1.pdf?dl=0 -- Diane Stidle Graduate Programs Manager Machine Learning Department Carnegie Mellon Universitystidle at cmu.edu 412-268-1299 -------------- next part -------------- An HTML attachment was scrubbed... URL: From predragp at andrew.cmu.edu Tue May 14 17:38:09 2019 From: predragp at andrew.cmu.edu (Predrag Punosevac) Date: Tue, 14 May 2019 17:38:09 -0400 Subject: Caffe, Theano, Torch, TensorFlow on GPU10 In-Reply-To: <5cdb238c.1c69fb81.87357.005c@mx.google.com> References: <5cdb238c.1c69fb81.87357.005c@mx.google.com> Message-ID: Thanks for the report. Does PyTorch work on all other GPUs? This is an opportunity for me to go and fix the things. Anybody else? Predrag On Tue, May 14, 2019 at 4:22 PM Emre Yolcu wrote: > > PyTorch is working fine on gpu10. Thank you very much Predrag. > > > > Emre > > > > From: Predrag Punosevac > Sent: Monday, May 13, 2019 4:24 PM > To: users at autonlab.org > Subject: Caffe, Theano, Torch, TensorFlow on GPU10 > > > > Dear Autonians, > > > > I have just upgraded NVidia driver on GPU10 to 418.74 as well as > > libcudnn7 to 7.5.1.10-1 which is compatible with CUDA 10.1 (the > > default on this server). I have also upgrade MATLAB to 2019a relase. > > > > nvidia-smi works as expected as well as MATLAB. gpuDevice() which was > > broken on the previous release of MATLAB now correctly shows four > > available GeForce GTX 1081 Ti cards. > > > > I would like to help the Auton Lab deep learning community and fix > > Caffe, Theano, Torch/PyTorch, TensorFlow or whatever are people using > > these days. I need you to log into the server and try to install your > > usual tool and test it. > > > > I would also appreciate feedback about other 8 gpu servers(GPU7 is > > reserved for the special project). I know at the very least that GPU3 > > scratch directory is filled. I am aware of a dead GPU card on GPU4 > > computing node (I have 1081 Ti on my desk which I can use instead of > > the dead Titan X on that server). GPU3 perhaps has another dead GPU > > card. > > > > Most Kind Regards, > > Predrag Punosevac > > From predragp at andrew.cmu.edu Tue May 14 17:49:55 2019 From: predragp at andrew.cmu.edu (Predrag Punosevac) Date: Tue, 14 May 2019 17:49:55 -0400 Subject: Caffe, Theano, Torch, TensorFlow on GPU10 In-Reply-To: References: <5cdb238c.1c69fb81.87357.005c@mx.google.com> Message-ID: I would prefer that we try with with CUDA 10.1. CUDA 9 branch is considered retired by NVidia. If I am to break things which work now and reboot machines I would like to get similar mileage as I did this the last time around (GPU 1-9 have uptime of nearly 300 days). Predrag On Tue, May 14, 2019 at 5:43 PM Abhay Gupta wrote: > > Hi Predrag, > > Pytorch does work fine on other GPUs. Is it possible to have both cuda 9.1 and cuda 9.2 installed on the remaining GPUs? Currently, I think the GPUs (2-9) have only 9.1 and gpu1 has 8.0 and 9.2 also. > > On Tue, May 14, 2019 at 5:39 PM Predrag Punosevac wrote: >> >> Thanks for the report. Does PyTorch work on all other GPUs? This is an >> opportunity for me to go and fix the things. Anybody else? >> >> Predrag >> >> On Tue, May 14, 2019 at 4:22 PM Emre Yolcu wrote: >> > >> > PyTorch is working fine on gpu10. Thank you very much Predrag. >> > >> > >> > >> > Emre >> > >> > >> > >> > From: Predrag Punosevac >> > Sent: Monday, May 13, 2019 4:24 PM >> > To: users at autonlab.org >> > Subject: Caffe, Theano, Torch, TensorFlow on GPU10 >> > >> > >> > >> > Dear Autonians, >> > >> > >> > >> > I have just upgraded NVidia driver on GPU10 to 418.74 as well as >> > >> > libcudnn7 to 7.5.1.10-1 which is compatible with CUDA 10.1 (the >> > >> > default on this server). I have also upgrade MATLAB to 2019a relase. >> > >> > >> > >> > nvidia-smi works as expected as well as MATLAB. gpuDevice() which was >> > >> > broken on the previous release of MATLAB now correctly shows four >> > >> > available GeForce GTX 1081 Ti cards. >> > >> > >> > >> > I would like to help the Auton Lab deep learning community and fix >> > >> > Caffe, Theano, Torch/PyTorch, TensorFlow or whatever are people using >> > >> > these days. I need you to log into the server and try to install your >> > >> > usual tool and test it. >> > >> > >> > >> > I would also appreciate feedback about other 8 gpu servers(GPU7 is >> > >> > reserved for the special project). I know at the very least that GPU3 >> > >> > scratch directory is filled. I am aware of a dead GPU card on GPU4 >> > >> > computing node (I have 1081 Ti on my desk which I can use instead of >> > >> > the dead Titan X on that server). GPU3 perhaps has another dead GPU >> > >> > card. >> > >> > >> > >> > Most Kind Regards, >> > >> > Predrag Punosevac >> > >> > > > > > -- > Regards, > Abhay Gupta From vjeansel at andrew.cmu.edu Thu May 16 10:10:19 2019 From: vjeansel at andrew.cmu.edu (Vincent Jeanselme) Date: Thu, 16 May 2019 10:10:19 -0400 Subject: Lov3 - Home full Message-ID: <8914b449-b8cd-86b9-519e-cdf4dd5d7529@andrew.cmu.edu> Hi all, I think that lov3 /scratch/home/ is full, could you clean your directory ? Thank you, Vincent From predragp at andrew.cmu.edu Thu May 16 12:57:57 2019 From: predragp at andrew.cmu.edu (Predrag Punosevac) Date: Thu, 16 May 2019 12:57:57 -0400 Subject: Lov3 - Home full In-Reply-To: <8914b449-b8cd-86b9-519e-cdf4dd5d7529@andrew.cmu.edu> References: <8914b449-b8cd-86b9-519e-cdf4dd5d7529@andrew.cmu.edu> Message-ID: Not without a reboot. Any objections that I reboot that machine let say tomorrow at 3:00 PM? That scratch directory was messed up for a while. Predrag On Thu, May 16, 2019 at 10:11 AM Vincent Jeanselme wrote: > > Hi all, > > I think that lov3 /scratch/home/ is full, could you clean your directory ? > > Thank you, > > Vincent > From predragp at andrew.cmu.edu Thu May 16 13:23:58 2019 From: predragp at andrew.cmu.edu (Predrag Punosevac) Date: Thu, 16 May 2019 13:23:58 -0400 Subject: Lov3 - Home full In-Reply-To: <71FB6796-5431-4E93-86E3-F958C75E1F2F@andrew.cmu.edu> References: <8914b449-b8cd-86b9-519e-cdf4dd5d7529@andrew.cmu.edu> <71FB6796-5431-4E93-86E3-F958C75E1F2F@andrew.cmu.edu> Message-ID: Please explain to me how? The scratch is 100% full. Once the file system is 100% full you can't run any commands on it. On Thu, May 16, 2019 at 1:20 PM Benedikt Boecking wrote: > > Are you planning on wiping the entire directory? If so there are a bunch of files I will have to move first. > > Best, > Ben > > > > On May 16, 2019, at 11:57 AM, Predrag Punosevac wrote: > > Not without a reboot. Any objections that I reboot that machine let > say tomorrow at 3:00 PM? That scratch directory was messed up for a > while. > > Predrag > > On Thu, May 16, 2019 at 10:11 AM Vincent Jeanselme > wrote: > > > Hi all, > > I think that lov3 /scratch/home/ is full, could you clean your directory ? > > Thank you, > > Vincent > > From boecking at andrew.cmu.edu Thu May 16 13:25:38 2019 From: boecking at andrew.cmu.edu (Benedikt Boecking) Date: Thu, 16 May 2019 12:25:38 -0500 Subject: Lov3 - Home full In-Reply-To: References: <8914b449-b8cd-86b9-519e-cdf4dd5d7529@andrew.cmu.edu> <71FB6796-5431-4E93-86E3-F958C75E1F2F@andrew.cmu.edu> Message-ID: <957476B3-E8BF-483D-9BB1-844AE182CA2E@andrew.cmu.edu> It is not 100% full. > On May 16, 2019, at 12:23 PM, Predrag Punosevac wrote: > > Please explain to me how? The scratch is 100% full. Once the file > system is 100% full you can't run any commands on it. > > On Thu, May 16, 2019 at 1:20 PM Benedikt Boecking > wrote: >> >> Are you planning on wiping the entire directory? If so there are a bunch of files I will have to move first. >> >> Best, >> Ben >> >> >> >> On May 16, 2019, at 11:57 AM, Predrag Punosevac wrote: >> >> Not without a reboot. Any objections that I reboot that machine let >> say tomorrow at 3:00 PM? That scratch directory was messed up for a >> while. >> >> Predrag >> >> On Thu, May 16, 2019 at 10:11 AM Vincent Jeanselme >> wrote: >> >> >> Hi all, >> >> I think that lov3 /scratch/home/ is full, could you clean your directory ? >> >> Thank you, >> >> Vincent >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From predragp at andrew.cmu.edu Thu May 16 13:31:24 2019 From: predragp at andrew.cmu.edu (Predrag Punosevac) Date: Thu, 16 May 2019 13:31:24 -0400 Subject: Lov3 - Home full In-Reply-To: <957476B3-E8BF-483D-9BB1-844AE182CA2E@andrew.cmu.edu> References: <8914b449-b8cd-86b9-519e-cdf4dd5d7529@andrew.cmu.edu> <71FB6796-5431-4E93-86E3-F958C75E1F2F@andrew.cmu.edu> <957476B3-E8BF-483D-9BB1-844AE182CA2E@andrew.cmu.edu> Message-ID: Sorry. Are we talking about gpu3 or lov3? gpu3 scratch is 100% full and the only remedy is reboot and wipe out all of the scratch. It has been for few months now. lov3 scratch is "only" 94% so people still have a change to log and delete junk before the partition becomes unresponsive. I would be happy to reboot either machine but gpu3 is the one which truly needs a reboot. Predrag Predrag On Thu, May 16, 2019 at 1:25 PM Benedikt Boecking wrote: > > It is not 100% full. > > > On May 16, 2019, at 12:23 PM, Predrag Punosevac wrote: > > Please explain to me how? The scratch is 100% full. Once the file > system is 100% full you can't run any commands on it. > > On Thu, May 16, 2019 at 1:20 PM Benedikt Boecking > wrote: > > > Are you planning on wiping the entire directory? If so there are a bunch of files I will have to move first. > > Best, > Ben > > > > On May 16, 2019, at 11:57 AM, Predrag Punosevac wrote: > > Not without a reboot. Any objections that I reboot that machine let > say tomorrow at 3:00 PM? That scratch directory was messed up for a > while. > > Predrag > > On Thu, May 16, 2019 at 10:11 AM Vincent Jeanselme > wrote: > > > Hi all, > > I think that lov3 /scratch/home/ is full, could you clean your directory ? > > Thank you, > > Vincent > > > From ngisolfi at cs.cmu.edu Fri May 17 09:10:55 2019 From: ngisolfi at cs.cmu.edu (Nick Gisolfi) Date: Fri, 17 May 2019 09:10:55 -0400 Subject: [Lunch] Today @12:15-1:30pm NSH 3001 Message-ID: Hi Everyone, Bring your own lunch and join us in 3001! See you there! - Nick From ngisolfi at cs.cmu.edu Fri May 17 16:28:09 2019 From: ngisolfi at cs.cmu.edu (Nick Gisolfi) Date: Fri, 17 May 2019 16:28:09 -0400 Subject: [Lunch] Summer Scheduling - Mark your Preferences! Message-ID: Starting next week, we need to book a time slot for lab lunch over the summer. It seems like a good opportunity to crowd-source an hour or so that works for most schedules. Mark your preferences here: http://whenisgood.net/xgkig8t Note that the times are EST. - Nick -------------- next part -------------- An HTML attachment was scrubbed... URL: From predragp at andrew.cmu.edu Wed May 22 15:36:54 2019 From: predragp at andrew.cmu.edu (Predrag Punosevac) Date: Wed, 22 May 2019 15:36:54 -0400 Subject: bash.autonlab.org back on line Message-ID: Somebody tripped over the network cable connecting bash machine to the network. It took few minutes to realize what was going on. Everything is back online now. Predrag From awd at cs.cmu.edu Wed May 22 16:01:42 2019 From: awd at cs.cmu.edu (Artur Dubrawski) Date: Wed, 22 May 2019 16:01:42 -0400 Subject: Our counter-human-trafficking work in the media: "It is not about the data, it is really about the people" Message-ID: WQED just had our own Cara Jones on their Just-a-Minute, see it here: https://www.wqed.org/just-a-minute Congrats to the Traffic Jam team at CMU and Marinus Analytics! Artur -------------- next part -------------- An HTML attachment was scrubbed... URL: From awd at cs.cmu.edu Thu May 23 09:33:34 2019 From: awd at cs.cmu.edu (Artur Dubrawski) Date: Thu, 23 May 2019 09:33:34 -0400 Subject: fyi: TRACIR project is finally official at both CMU and Pitt Message-ID: https://www.upmc.com/media/news/052219-tracir-dod -------------- next part -------------- An HTML attachment was scrubbed... URL: From ngisolfi at cs.cmu.edu Fri May 24 08:10:41 2019 From: ngisolfi at cs.cmu.edu (Nick Gisolfi) Date: Fri, 24 May 2019 08:10:41 -0400 Subject: [Lunch] TODAY @12:15, Outdoor Seating between Smith and Hamburg Halls Message-ID: <42AB7C43-E30C-4640-8E79-937B0433BDB6@cs.cmu.edu> Hi Everyone, Today will be our last Friday lab lunch, 12:15-1:30pm. Since it looks nice outside, let?s plan to meet at the outdoor seating area between Smith and Hamburg halls. Going forward, we will be having lab lunches on Thursdays, 12-1pm in NSH 4201. I?ll remind everyone of the new time and location next week. We hope to see you at lunch today! - Nick From predragp at andrew.cmu.edu Tue May 28 21:32:03 2019 From: predragp at andrew.cmu.edu (Predrag Punosevac) Date: Tue, 28 May 2019 21:32:03 -0400 Subject: devtoolset-8 and rh-git218 upgrade Message-ID: <20190529013203.lxNU_YDho%predragp@andrew.cmu.edu> Dear Autonians, I just upgraded devtoolset to 8. In plain English /opt/rh now defaults to gcc version 8. I have also installed /opt/rh-git218 which is git 2.18 but I didn't remove git 2.9 until I am 100% sure that things will not break your workflow. As I am typing this email changes are also pushed onto the desktops managed by me. Best, Predrag From predragp at andrew.cmu.edu Tue May 28 23:12:27 2019 From: predragp at andrew.cmu.edu (Predrag Punosevac) Date: Tue, 28 May 2019 23:12:27 -0400 Subject: New Auton Lab PKI (public key infrastructure) Message-ID: <20190529031227.20_1OtTQ6%predragp@andrew.cmu.edu> Dear Autonians, I would like to give you heads up. Due to the sheer number of new hires and incoming interns I was forced to create new PKI (public key infrastructure) in order to accommodate new desktops and their VPN connections to the core of the lab. I would also like to use this opportunity to switch our main firewall/VPN gateways to 10 Gigabit servers we acquired last year. The infrastructure we currently have in place was set up by yours truly almost six years ago and it is due to be rebuild. Pushing credentials to the desktops and replacing old routing tables on all computing nodes will inevitably result in some down time for everyone. I have tested things extensively but I would not be surprised that and hour or two turn in the whole afternoon of fighting rough edges. Unless I hear very good reason not to pull the trigger changes will take place tomorrow afternoon. Most Kind Regards, Predrag Punosevac From ngisolfi at cs.cmu.edu Wed May 29 12:50:04 2019 From: ngisolfi at cs.cmu.edu (Nick Gisolfi) Date: Wed, 29 May 2019 12:50:04 -0400 Subject: [Lunch] Thursday, May 30, @12-1pm in NSH 3001 Message-ID: <07308C92-C4D9-47C5-82AD-A95E7C5720A6@cs.cmu.edu> Hi Everyone, This week marks the start of our Thursday afternoon Auton lunches for the summer (12pm-1pm time fixed, location will vary). We?ll be in NSH 3001 this week. See yinz tomorrow! - Nick From predragp at andrew.cmu.edu Wed May 29 14:25:57 2019 From: predragp at andrew.cmu.edu (Predrag Punosevac) Date: Wed, 29 May 2019 14:25:57 -0400 Subject: PKI changes pushed, majority of services restored Message-ID: Dear Autonians, I pushed new Public Key Infrastructure and hour ago. Most importantly new routing tables appear to work as designed as well as lab's main shell gateway bash.autonlab.org. I will continue to push new PKI on the users desktop but that affect a much smaller subset of our Lab users. Best, Predrag