From predragp at cs.cmu.edu Wed Jan 4 11:21:16 2017 From: predragp at cs.cmu.edu (Predrag Punosevac) Date: Wed, 04 Jan 2017 11:21:16 -0500 Subject: GPU machines In-Reply-To: <6B140A4F-679E-41CD-BE0D-2A210762361B@cs.cmu.edu> References: <6B140A4F-679E-41CD-BE0D-2A210762361B@cs.cmu.edu> Message-ID: <20170104162116.v9i_Kd3GN%predragp@cs.cmu.edu> Jayanth Koushik wrote: > Hey Predrag > > There seems to be something wrong with some GPU nodes. On GPU2, everything seems super slow - even nvidia-smi. And on GPU3, the third GPU shows full memory usage. even though there???s nothing running on it. I was running a process (Python+Theano) on this GPU when the error happened - the process threw an OutOfMemoryException, even though I was using only a fraction of the memory. > > Please take a look when you have the time. > GPUs don't stop working out of blue sky. There was an e-mail send by Samy before the holidays where he mentioned that he will hog all GPUs. Did you ask him if he finished his experiments? I see him logged on all GPUs. Predrag > Thanks! > ~Jayanth From predragp at cs.cmu.edu Thu Jan 5 13:49:29 2017 From: predragp at cs.cmu.edu (Predrag Punosevac) Date: Thu, 05 Jan 2017 13:49:29 -0500 Subject: New Printer is fully functional Message-ID: <20170105184929.nXDMTW5qv%predragp@cs.cmu.edu> Dear Autonians, I apologize for this mass e-mail as this message only concerns people who are using Auton Lab issued desktop computers. If you are not using such machine please stop reading the e-mail. Ever since somebody decided to replace McKinley printer with HP CM6040 MFP in order to empty an office space many of you had hard time printing. I could not do anything about it as I didn't control the hardware. Long story short Dr. Dubrawski bought nice monochromatic laser printer HL-L5100DN which is located in NSH 3125 popularly known as "Dungeon". The printer is fully functional now and you should be able to send your jobs to it (I configured the desktops). NSH 3125 is now permanently unlocked but please keep the door shut at all time. We are using obscurity as a measure of security. Most valuable things have being removed from the room. Best, Predrag P.S. Please ping me if you have any problems printing or setting double side printing preference. From predragp at cs.cmu.edu Sun Jan 8 19:33:14 2017 From: predragp at cs.cmu.edu (Predrag Punosevac) Date: Sun, 08 Jan 2017 19:33:14 -0500 Subject: trouble logging onto neill4 In-Reply-To: References: Message-ID: <20170109003314.ynUjlYveY%predragp@cs.cmu.edu> Mallory Nobles wrote: > Hi Predrag, > I am having trouble logging onto neill4. It is very slow to respond, or > never responds. Could you check to make sure that this machine is up? > Thanks! > Mallory M/Monit http://monit.autonlab.org:8080 username: auton password: Dr.Who lists it as up. However I could not log into neill4 even with local root account. Based upon memory report somebody caused kernel panic. I do get e-mail notification when the status of any server/services changes. However if the machine crashes it takes couple of ping cycles for machine to be reported as down. Thanks for the report. I will have a look tomorrow. Cheers, Predrag From awd at cs.cmu.edu Fri Jan 13 14:55:21 2017 From: awd at cs.cmu.edu (Artur Dubrawski) Date: Fri, 13 Jan 2017 14:55:21 -0500 Subject: Fwd: FW: DHTI - internships for the summer In-Reply-To: <44fd0c92bd82492484079d195f88b61a@PGH-MSGMLT-01.andrew.ad.cmu.edu> References: <44fd0c92bd82492484079d195f88b61a@PGH-MSGMLT-01.andrew.ad.cmu.edu> Message-ID: FYI: Summer internship openings at Highmark for phd and research oriented master students. A. Hi Lynn, We have finally managed to obtain an official job listing for two interns for the upcoming summer: (https://highmarkhealth.wd1.myworkdayjobs.com/highmark/job/Pittsburgh-PA-5th-Avenue-Place/Advanced-Analytics-Intern_J102481). We are looking for graduate students in CS / ML (and related disciplines) who have had some exposure to health data analytics. Would you be kind enough to circulate this within your network at DHTI ? Hopefully this will be a yearly thing, where CMU/DHTI becomes a pipeline for us to recruit quality interns. A great story of collaboration. Happy Friday! -- *Suman Giri,*PhD Medical Economics Consultant, Advanced Analytics & Reporting Highmark, Inc. ? :412-544-8019 ? : suman.giri at highmark.com ------------------------------------------------------------------------ This e-mail and any attachments to it are confidential and are intended solely for use of the individual or entity to whom they are addressed. If you have received this e-mail in error, please notify the sender immediately and then delete it. If you are not the intended recipient, you must not keep, use, disclose, copy or distribute this e-mail without the author's prior permission. The views expressed in this e-mail message do not necessarily represent the views of Highmark, its diversified business, or affiliates. -------------- next part -------------- An HTML attachment was scrubbed... URL: From yichongx at andrew.cmu.edu Fri Jan 13 21:04:33 2017 From: yichongx at andrew.cmu.edu (Yichong Xu) Date: Sat, 14 Jan 2017 10:04:33 +0800 Subject: DHTI - internships for the summer In-Reply-To: References: <44fd0c92bd82492484079d195f88b61a@PGH-MSGMLT-01.andrew.ad.cmu.edu> Message-ID: <50AAE8A1-DD54-46A9-8D96-0EFC3009A04F@andrew.cmu.edu> Hi Artur, As you know I?m looking for internships next summer - do you think there?s a fit between me and this job? Thanks! Thanks, Yichong > ? 2017?1?14????3:55?Artur Dubrawski ??? > > FYI: Summer internship openings at Highmark for phd and research oriented master students. > > A. > > > > Hi Lynn, > > > > We have finally managed to obtain an official job listing for two interns for the upcoming summer: > > (https://highmarkhealth.wd1.myworkdayjobs.com/highmark/job/Pittsburgh-PA-5th-Avenue-Place/Advanced-Analytics-Intern_J102481 ). > > > > We are looking for graduate students in CS / ML (and related disciplines) who have had some exposure to health data analytics. Would you be kind enough to circulate this within your network at DHTI ? > > > > Hopefully this will be a yearly thing, where CMU/DHTI becomes a pipeline for us to recruit quality interns. A great story of collaboration. > > > > Happy Friday! > > > > -- > > Suman Giri, PhD > > Medical Economics Consultant, > > Advanced Analytics & Reporting > > Highmark, Inc. > > ? : 412-544-8019 > > ? : suman.giri at highmark.com > > > > > > This e-mail and any attachments to it are confidential and are intended solely for use of the individual or entity to whom they are addressed. If you have received this e-mail in error, please notify the sender immediately and then delete it. If you are not the intended recipient, you must not keep, use, disclose, copy or distribute this e-mail without the author's prior permission. The views expressed in this e-mail message do not necessarily represent the views of Highmark, its diversified business, or affiliates. > -------------- next part -------------- An HTML attachment was scrubbed... URL: From predragp at imap.srv.cs.cmu.edu Tue Jan 17 11:08:03 2017 From: predragp at imap.srv.cs.cmu.edu (Predrag Punosevac) Date: Tue, 17 Jan 2017 11:08:03 -0500 Subject: Can't login via lop1 In-Reply-To: References: Message-ID: <1d37444423350faec0efa8d465b3bc51@imap.srv.cs.cmu.edu> On 2017-01-17 07:13, Chun-Liang Li wrote: > Hi Predrag, > > I can only login into the autonlab's machines via bash.autonlab.org > [1] but I can't use lop1 and shell. Do you have any idea about what > happened? Thanks a lot. > Apparently over the night the firewall which controls access to lop1 became inaccessible which triggered automatic failover to ;; ANSWER SECTION: lock.autonlab.org. 300 IN CNAME webfwd1.zoneedit.com. webfwd1.zoneedit.com. 300 IN A 198.199.92.59 webfwd1.zoneedit.com. 300 IN A 162.243.226.87 note Zoneedit (now the part of EasyDNS) is our DNS provider. There is nothing wrong with lock and lop1. I can log into them with my regular credentials from interior machines. You can't because you are actually hitting EasyDNS servers. I have to restore proper DNS records. Give me few minutes. Best, Predrag P.S. I noticed that sometimes our public static IP addresses become unavailable but I have not being able quite to point to exact source of the problem. There are two suspects. CMU or CMU's Internet service provider (I forgot the name of the name of the company but it is a government/business provider only). CMU also has few misconfigured authoritative DNS serves located in New York which I reported but they never did anything about it. > Chun-Liang > > Links: > ------ > [1] http://bash.autonlab.org From predragp at imap.srv.cs.cmu.edu Tue Jan 17 11:14:06 2017 From: predragp at imap.srv.cs.cmu.edu (Predrag Punosevac) Date: Tue, 17 Jan 2017 11:14:06 -0500 Subject: Lop1 Host Key Change? In-Reply-To: References: Message-ID: <480234ae2c4b39f4e6ae46ad12026788@imap.srv.cs.cmu.edu> On 2017-01-17 11:03, Junier Oliva wrote: > Hey Predrag, > > Just wanted to confirm that the following is expected and nothing > fishy is going on. > > Thanks, > Junier > Not expected! It is fixed now. Please see my previous message about accessibility of lop1. It was due to the failover which was triggered by the problems with CMU network. Predrag >> @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ >> >> @ WARNING: POSSIBLE DNS SPOOFING DETECTED! @ >> >> @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ >> >> The ECDSA host key for lop1.autonlab.org [1] has changed, >> >> and the key for the corresponding IP address 162.243.226.87 >> >> is unknown. This could either mean that >> >> DNS SPOOFING is happening or the IP address for the host >> >> and its host key have changed at the same time. >> >> @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ >> >> @ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @ >> >> @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ >> >> IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY! >> >> Someone could be eavesdropping on you right now (man-in-the-middle >> attack)! >> >> It is also possible that a host key has just been changed. >> >> The fingerprint for the ECDSA key sent by the remote host is >> >> SHA256:y+DmYd7YUcfWtF/xavIERlCVUTF8Y0uHkR7eZEVoS00. >> >> Please contact your system administrator. > > > Links: > ------ > [1] http://lop1.autonlab.org From awd at cs.cmu.edu Tue Jan 17 16:28:25 2017 From: awd at cs.cmu.edu (Artur Dubrawski) Date: Tue, 17 Jan 2017 16:28:25 -0500 Subject: A visiting scholar could use guidance on finding a place to live In-Reply-To: <49234255-7732-4f96-3cc3-02782de7103f@grenoble-inp.org> References: <513e57d9-24e3-5fda-7683-32c140e0a138@grenoble-inp.org> <4aacded4-4932-8eea-6427-7d73b31c7ed8@cs.cmu.edu>

<7c860249-9e54-71a3-b042-31232c40c877@grenoble-inp.org> <97bfc222-90bb-9c92-1769-2afe4760bb92@cs.cmu.edu> <76d1618b-3ba2-e60c-cc96-21e604517e81@grenoble-inp.org> <49234255-7732-4f96-3cc3-02782de7103f@grenoble-inp.org> Message-ID: <3982d496-ac49-87d5-3f6a-f7089b5898e3@cs.cmu.edu> Dear Autonians, Vincent (cc-d) is coming to work with us on his Master's thesis. He will arrive in a few weeks and stay for a few months. I am clearly out of circulation if it comes to temporary housing ideas around campus, but I bet I am a minority here. If you happen to know something about that could you please share your suggestions with Vincent (cc-d) directly. Thanks! Artur From sheath at andrew.cmu.edu Thu Jan 19 12:32:07 2017 From: sheath at andrew.cmu.edu (Simon Heath) Date: Thu, 19 Jan 2017 12:32:07 -0500 Subject: GPU maintenance, Monday Jan 23rd 3-5 pm Message-ID: <5b3ce9627655ea229064c956a73a61ae.squirrel@webmail.andrew.cmu.edu> Dear Autonians, The GPU compute nodes gpu1, gpu2 and gpu3 need maintenance done that will require a reboot. If nobody has any objections we will be shutting them down next Monday sometime between 3 and 5 pm, so please make sure any jobs you run will finish up before then so you don't lose any work. Thanks, Simon -- Simon Heath, Research Programmer and Analyst Robotics Institute - Auton Lab Carnegie Mellon University sheath at andrew.cmu.edu From predragp at imap.srv.cs.cmu.edu Mon Jan 23 15:24:32 2017 From: predragp at imap.srv.cs.cmu.edu (Predrag Punosevac) Date: Mon, 23 Jan 2017 15:24:32 -0500 Subject: GPU2 rebooted, fully functional Message-ID: Dear Autonians, I just rebooted GPU2. It is fully functional now and I see 4 GPU devices. We are waiting for Samy in order to reboot GPU1 and GPU3. I will let you know the schedule for GPU1 and GPU3 reboot as soon as Samy gives me ok. Best, Predrag From predragp at imap.srv.cs.cmu.edu Mon Jan 23 17:20:42 2017 From: predragp at imap.srv.cs.cmu.edu (Predrag Punosevac) Date: Mon, 23 Jan 2017 17:20:42 -0500 Subject: GPU2 rebooted, fully functional In-Reply-To: References: Message-ID: On 2017-01-23 16:39, Chun-Liang Li wrote: > Hi Predrag, > > I tried to use nvidia-smi to check the GPU status on gpu2, but there > is no response after I type the command. Also, I can't run tensorflow > with GPU. Are all GPU occupied? or is there any issue? Thanks. > > Chun-Liang > You know what. Something strange is happening to that server. After I sent you the replay I rebooted server one more time. I see lots of errors on the second GPU card. I will have to have a closer look at this server tomorrow. Please people if you going to use GPU2 plan that I will probably have to shutdown tomorrow again. Predrag > On Mon, Jan 23, 2017 at 3:24 PM, Predrag Punosevac > wrote: > >> Dear Autonians, >> >> I just rebooted GPU2. It is fully functional now and I see 4 GPU >> devices. We are waiting for Samy >> in order to reboot GPU1 and GPU3. I will let you know the schedule >> for GPU1 and GPU3 reboot as >> soon as Samy gives me ok. >> >> Best, >> Predrag From predragp at imap.srv.cs.cmu.edu Wed Jan 25 13:37:12 2017 From: predragp at imap.srv.cs.cmu.edu (Predrag Punosevac) Date: Wed, 25 Jan 2017 13:37:12 -0500 Subject: GPU1 and GPU2 to be rebooted Message-ID: <766d330cbd641bfaf5371360491e3b24@imap.srv.cs.cmu.edu> Dear Autonians, I have to shutdown GPU2 in order to pull the defective GPU card. I am planning to this today at 3:00 PM. I also have thumbs up from Sammy to reboot GPU1 which will clear the memory, kill zombie processes, and hopefully bring machine back to the useful state. I am planning to reboot GPU1 also at 3:00 PM. Predrag From predragp at imap.srv.cs.cmu.edu Wed Jan 25 15:39:01 2017 From: predragp at imap.srv.cs.cmu.edu (Predrag Punosevac) Date: Wed, 25 Jan 2017 15:39:01 -0500 Subject: GPU1 and GPU2 update Message-ID: GPU1 is now rebooted. Memory should be cleared and zombie process killed. If something is not looking OK please email me. I am having troubles to figure out how to identify the slot of the GPU card which needs to be taken out of the server. I have the BUS locations and I know the BUS number of the failed GPU card. I just don't know if I can read off that BUS number from the card itself. I do not want to do trial and error method as pulling the card is very physically demanding. I will have to remove the server from the rack. I anticipate that I will have to remove GPU2 from the rack tomorrow morning. PRedrag From predragp at imap.srv.cs.cmu.edu Thu Jan 26 14:07:12 2017 From: predragp at imap.srv.cs.cmu.edu (Predrag Punosevac) Date: Thu, 26 Jan 2017 14:07:12 -0500 Subject: GPU2 is "fixed" Message-ID: <09d892e88eb9ff3f62ea79da01422960@imap.srv.cs.cmu.edu> Dear Autonians, I removed the defective GPU card from GPU2 computing node. The server is now back in production with three working GPU cards. I will give you an advance notice when I receive spare part. For now my plan is to keep it running at the very least until Monday afternoon. Please plan accordingly as this server is not likely to stay up for more than two weeks. Cheers, Predrag From awd at cs.cmu.edu Fri Jan 27 18:16:05 2017 From: awd at cs.cmu.edu (Artur Dubrawski) Date: Fri, 27 Jan 2017 18:16:05 -0500 Subject: Auton Lab research made it to Sports Illustrated (and is going to the Superbowl) Message-ID: As Emily has just pointed out, our work has become a part of the Superbowl frenzy (thanks Emily!): http://www.si.com/nfl/2017/01/27/super-bowl-prostitution-sex-trafficking-ryan-murphy-broncos I am not sure if the author had quoted our numbers right, but we are monitoring the upcoming event in support of the City of Houston Mayor's office and local nonprofits, who in turn work with local law enforcement. So our knowledge will be updated with new evidence thanks to Kyle, Shari and Ben who put together a set of very pretty google worksheets showing data and trends in a very informative and quite actionable way, precooked for our Superbowl partners. At the same time, Marinus will be in the field helping generate and confirm leads using Traffic Jam. The only piece missing is the Steelers. Better luck next time. Cheers, Artur -------------- next part -------------- An HTML attachment was scrubbed... URL: From awm at cs.cmu.edu Sat Jan 28 11:10:37 2017 From: awm at cs.cmu.edu (Andrew Moore) Date: Sat, 28 Jan 2017 11:10:37 -0500 Subject: Auton Lab research made it to Sports Illustrated (and is going to the Superbowl) In-Reply-To: References: Message-ID: This is super-impressive. I will try to include it as an example in my conversations with the press, and Colleen Byron and Jim I would be grateful if you could steer potentially interested journalists towards Artur. On Fri, Jan 27, 2017 at 6:16 PM, Artur Dubrawski wrote: > As Emily has just pointed out, our work has become a part of the Superbowl > frenzy (thanks Emily!): > > http://www.si.com/nfl/2017/01/27/super-bowl-prostitution- > sex-trafficking-ryan-murphy-broncos > > I am not sure if the author had quoted our numbers right, but we are > monitoring the upcoming event > in support of the City of Houston Mayor's office and local nonprofits, who > in turn work with local law > enforcement. So our knowledge will be updated with new evidence thanks to > Kyle, Shari and Ben > who put together a set of very pretty google worksheets showing data and > trends in a very informative > and quite actionable way, precooked for our Superbowl partners. At the > same time, Marinus will be in > the field helping generate and confirm leads using Traffic Jam. The only > piece missing is the Steelers. > Better luck next time. > > Cheers, > Artur > > > > -- Andrew Moore , Dean, School of Computer Science , Carnegie Mellon. Twitter feed -------------- next part -------------- An HTML attachment was scrubbed... URL: From colleen at 0to5.com Mon Jan 30 10:04:21 2017 From: colleen at 0to5.com (Colleen Martin) Date: Mon, 30 Jan 2017 10:04:21 -0500 Subject: Auton Lab research made it to Sports Illustrated (and is going to the Superbowl) In-Reply-To: References: Message-ID: Good Morning Andrew and Team, Congrats on the nice media mention! Artur - as you may recall - we had reached out to reporters about this story back in the fall. For media to bite again - it would help if there was some sort of update. Have there been any milestones or updates to the technology or any new elements to the story that we can share with the media? We are happy to jump on a call to discuss. Please let us know. Thanks! Colleen > On Jan 28, 2017, at 11:10 AM, Andrew Moore wrote: > > This is super-impressive. I will try to include it as an example in my conversations with the press, and Colleen Byron and Jim I would be grateful if you could steer potentially interested journalists towards Artur. > > > On Fri, Jan 27, 2017 at 6:16 PM, Artur Dubrawski > wrote: > As Emily has just pointed out, our work has become a part of the Superbowl frenzy (thanks Emily!): > > http://www.si.com/nfl/2017/01/27/super-bowl-prostitution-sex-trafficking-ryan-murphy-broncos > > I am not sure if the author had quoted our numbers right, but we are monitoring the upcoming event > in support of the City of Houston Mayor's office and local nonprofits, who in turn work with local law > enforcement. So our knowledge will be updated with new evidence thanks to Kyle, Shari and Ben > who put together a set of very pretty google worksheets showing data and trends in a very informative > and quite actionable way, precooked for our Superbowl partners. At the same time, Marinus will be in > the field helping generate and confirm leads using Traffic Jam. The only piece missing is the Steelers. > Better luck next time. > > Cheers, > Artur > > > > > > > -- > Andrew Moore , Dean, School of Computer Science , Carnegie Mellon. Twitter feed > _________________ Colleen Martin Principal Zer0 to 5ive www.0to5.com M: 570-259-0915 colleen at 0to5.com Twitter: @devinecolleen -------------- next part -------------- An HTML attachment was scrubbed... URL: From predragp at cs.cmu.edu Mon Jan 30 23:44:56 2017 From: predragp at cs.cmu.edu (Predrag Punosevac) Date: Mon, 30 Jan 2017 23:44:56 -0500 Subject: Linux containers with Singularity Message-ID: <20170131044456.VMsJ88QlE%predragp@cs.cmu.edu> Dear Autonians, I just want to give you heads up. I am about to wrap up my testing of Singularity http://singularity.lbl.gov/ https://github.com/singularityware/singularity I anticipate that various Ubuntu container images will be made available on our computing nodes in the next week or so. Yes on GPU computing nodes as well. Singularity is essentially virtual-kernel technology for Linux. Virtual kernels have being, to my knowledge, first time introduced in DragonDfy BSD circa 2007 to enable kernel hackers to run and debug safely multiple kernels in userland. Singularity for Linux is mostly created out of frustration with the lack of portability of newly written code and frustration with LXC/Docker security problems. Singularity will essentially allow you to run your favorite Linux flavour and application in the userland without root privileges and have complete access to the file system. This is a nice intro video. https://www.youtube.com/watch?v=xuIQoth0r4E Best, Predrag From predragp at cs.cmu.edu Tue Jan 31 16:34:45 2017 From: predragp at cs.cmu.edu (Predrag Punosevac) Date: Tue, 31 Jan 2017 16:34:45 -0500 Subject: Fwd: NSH & Smith Power Shutdown Thursday Message-ID: <20170131213445.IkvnNCK4B%predragp@cs.cmu.edu> -------- Original Message -------- To: Jim Skees From: Jim Skees Subject: NSH & Smith Power Shutdown Thursday Date: Tue, 31 Jan 2017 11:00:43 -0500 *Division of Operations* *Facilities Management Services* ** *TO: Shutdown Group, Hamburg Hall, Newell Simon Hall, Smith Hall, FMS Building* *FROM: Service Response Center* *DATE: January 26, 2017* *SUBJECT: Hamburg, Smith, Newell Simon Halls and FMS Building Power Shutdown 2/2* There will be a 30-minute power shutdown in Hamburg, Smith, and Newell Simon Halls and the FMS Building on Thursday, 2/2, from 6:00AM to 6:30AM. In preparation for the power shutdown, FMS will be turning off large motor loads and HVAC systems and securing the building elevators. Those systems will be turned off starting at 5:30AM and will be restored immediately after the power is turned back on. All buildings should be back to normal by 7:00AM. This shutdown is necessary for construction. If you have any questions or concerns, please contact Shannon Wetzel at 8-6332 or swetzel at andrew.cmu.edu .