From awd at cs.cmu.edu Fri Sep 2 09:28:13 2016 From: awd at cs.cmu.edu (Artur Dubrawski) Date: Fri, 2 Sep 2016 09:28:13 -0400 Subject: Did you know that the best way to celebrate Labor Day is by riding a bicycle? Message-ID: Dear Autonians, The weather forecast shows sunny and dry weather for Monday. Please let me know if you'd like to join for a relaxing morning ride at 9am. Cheers, Artur From awd at cs.cmu.edu Tue Sep 6 12:27:41 2016 From: awd at cs.cmu.edu (Artur Dubrawski) Date: Tue, 6 Sep 2016 12:27:41 -0400 Subject: Fwd: CFP: NIPS 2016 Workshop on Machine Learning for Health In-Reply-To: <0E1F0927-6B37-4F16-9905-49280E63FB0D@stanford.edu> References: <0E1F0927-6B37-4F16-9905-49280E63FB0D@stanford.edu> Message-ID: <164bb53a-1036-3c9c-7d22-765985a8e8c0@cs.cmu.edu> Team, This is a good venue to consider submitting your semi-mature work into ML for healthcare. Cheers Artur =============================================================================== CALL FOR PAPERS NIPS 2016 Workshop on Machine Learning for Health (NIPS ML4HC) http://www.nipsml4hc.ws/ A workshop at the Twenty-Ninth Annual Conference on Neural Information Processing Systems (NIPS 2016) Barcelona, Spain Fri Dec 9th 08:00 AM -- 06:30 PM DATES: Sept 22, 2016: NIPS ML4HC Travel Award Submission Deadline Oct 3, 2016: Round 1 Acceptance Notification (coinciding with NIPS Early Registration Deadline) Oct 3, 2016: NIPS Early Registration Deadline Oct 28, 2016: Round 2 Submission Deadline Nov 11 2016: Round 2 Acceptance Notification Dec 1, 2016: Final papers due Dec 9, 2016: Workshop =============================================================================== ABSTRACT: The last decade has seen unprecedented growth in the availability and size of digital health data, including electronic health records, genetics, and wearable sensors. These rich data sources present opportunities to develop and apply machine learning methods to enable precision medicine. The aim of this workshop is to engender discussion between machine learning and clinical researchers about how statistical learning can enhance both the science and the practice of medicine. Of particular interest to this year?s workshop is a phrase recently coined by the British Medical Journal, "Big Health Data", where the focus is on modeling and improving health outcomes across large numbers of patients with diverse genetic, phenotypic, and environmental characteristics. The majority of clinical informatics research has focused on narrow populations representing, for example, patients from a single institution or sharing a common disease, and on modeling clinical factors, such as lab test results and treatments. Big health considers large and diverse cohorts, often reaching over 100 million patients in size, as well as environmental factors that are known to impact health outcomes, including socioeconomic status, health care delivery and utilization, and pollution. Big Health Data problems pose a variety of challenges for standard statistical learning, many of them nontraditional. Including a patient?s race and income in statistical analysis, for example, evokes concerns about patient privacy. Novel approaches to differential privacy may help alleviate such concerns. Other examples include modeling biased measurements and non-random missingness and causal inference in the presence of latent confounders. In this workshop we will bring together clinicians, health data experts, and machine learning researchers working on healthcare solutions. The goal is to have a discussion to understand clinical needs and the technical challenges resulting from those needs including the development of interpretable techniques which can adapt to noisy, dynamic environments and the handling of biases inherent in the data due to being generated during routine care. SUBMISSION INSTRUCTIONS: Researchers interested in contributing should upload an extended abstract of 4 pages in PDF format to the MLCB submission web site: https://easychair.org/conferences/?conf=nips16mlhc by October 28, 2016, 11:59pm (time zone of your choice). No special style is required. Authors should use the NIPS style file, and submissions should be suitably anonymized and meet the requirements for double-blind reviewing. Send any questions to nips16mlhc at gmail.com . All submissions will be anonymously peer reviewed and will be evaluated on the basis of their technical content. The workshop allows submissions of papers that are under review or have been recently published in a conference or a journal. Authors should state any overlapping published work at time of submission. Part of our workshop includes a clinician pitch, a five-minute presentation of open clinical problems that need data-driven solutions. These presentations will be followed by a discussion between invited clinicians and attending ML ?researchers to understand how machine learning can play a role in solving the problem presented. Finally, the pitch plays a secondary role of enabling new collaborations between machine learning researchers and clinicians: an important step for machine learning to have a meaningful role in healthcare. A general call for clinician pitches will be disseminated to clinical researchers and major physician organizations, including clinician social networks such as Doximity. We will invite submission of two? page abstracts (not including references) for poster contributions and short oral presentations describing innovative machine learning research on relevant clinical problems and data. Topics of interest include but are not limited to models for diseases and clinical data, temporal models, Markov decision processes for clinical decision support, multi?scale data-?integration, modeling with missing or biased data, learning with non-stationary data, uncertainty and uncertainty propagation, non ?i.i.d. structure in the data, critique of models, causality, model biases, transfer learning, and incorporation of non-clinical (e.g., socioeconomic) factors. CONFIRMED SPEAKERS: Leo Anthony Celi (MIMIC Project Lead) Eric Xing (CMU) Jenna Wiens (University of Michigan) Sendhil Mullainathan (Harvard) Julien Cornebise (DeepMind) Neil Lawrence (University of Sheffield) Joel Dudley (Mount Sinai) Niels Peek (University of Manchester) ORGANIZERS: Madalina Fiterau (Stanford) Jason Fries (Stanford) Marzyeh Ghassemi (MIT) Theofanis Karaletsos (Geometric Intelligence) Rajesh Ranganath (Princeton) Peter Schulam (Johns Hopkins) Uri Shalit (NYU) David Kale (University of Southern California) SENIOR PC: Artur Dubrawski (CMU) Christopher Re (Stanford) Cynthia Rudin (MIT) David Sontag (NYU) Deborah Estrin (Cornell-tech) Fei Wang (Cornell) Gunnar R?tsch (ETH) Jimeng Sun (Georgia Institute of Technology) Joyce Ho (Emory) Neil Lawrence (U Sheffield) Nigam Shah (Stanford) Rosalind Picard (MIT) Scott Delp (Stanford) Suchi Saria (JHU) Susan Murphy (U. Michigan) Trevor Hastie (Stanford) Zak Kohane (Harvard) Samantha Kleinberg (Stevens Institute) George M Hripcsak (Columbia University) -------------- next part -------------- An HTML attachment was scrubbed... URL: From predragp at cs.cmu.edu Tue Sep 6 13:54:47 2016 From: predragp at cs.cmu.edu (Predrag Punosevac) Date: Tue, 06 Sep 2016 13:54:47 -0400 Subject: joblib-memory and foxconn Message-ID: <20160906175447.vI1X76QE6%predragp@cs.cmu.edu> Dear Autonians, I know that many of you are pushing deadlines so I will try not to touch the system next 2 weeks. However I have to give you heads up. joblib-memory on foxconn has ballooned to 7GB which combined with 11GB of MATLAB code has pushed the root directory located on 37GB SSD to about 95% full. If the joblib-memory fills the root directory I will have no other choice but to reboot the server and clean that. The downtime is not going to be 5 minutes as I will also have to update MATLAB (due to older license). As I said I hope I will have to do none of that for the next 2 weeks. Best, Predrag From predragp at cs.cmu.edu Wed Sep 7 13:04:34 2016 From: predragp at cs.cmu.edu (Predrag Punosevac) Date: Wed, 07 Sep 2016 13:04:34 -0400 Subject: TensorFlow In-Reply-To: References: <5989f0a58b6e1ce0f826fdb3a92adb02@imap.srv.cs.cmu.edu> <20160814030626.aI_ezzL5H%predragp@cs.cmu.edu> <20160903011221.4zTd8tFen%predragp@cs.cmu.edu> <20160903022743.8wzM2c6op%predragp@cs.cmu.edu> <20160906020336.i8dxCj6jI%predragp@cs.cmu.edu> Message-ID: <20160907170434.GScFSKZt8%predragp@cs.cmu.edu> Fish Tung wrote: > Dear Predreg, > > I faced some issues when I was trying to run tensorflow on gpu1 and gpu2, > I sincerely appreciate if you can address theses questions for me: > > 1. It seems like gpu1 doesn't have tensorflow installed. (I am using python > under /usr/local/bin) > 2. gpu2 has tensorflow installed but it is attempted to call libcudart7.0 > while under /usr/local, we only have cudart7.5. Thus I got error > whenever I try to import it > > Can you give me guideline on how to correctly set up tensorflow? > Thanks! > Best, > Fish Tung Let me deflect your question to users at autonlab.org which is our mailing list. There are people who are using tensorflow on the daily base who will be much more helpful than I. @Artur Yes I know that we should have Wiki pages for situation like this. Let us complete the migration of old web to DokuWiki before we start putting those documents on. The first one will be tutorial on Git and Gogs. Best, Predrag From predragp at cs.cmu.edu Thu Sep 8 20:02:45 2016 From: predragp at cs.cmu.edu (Predrag Punosevac) Date: Thu, 08 Sep 2016 20:02:45 -0400 Subject: TensorFlow Message-ID: <20160909000245.RYArOIC8q%predragp@cs.cmu.edu> The TensorFlow is upgraded to the latest and greatest on GPU1 and GPU2. Due to the fact that there was a non-trivial step involving manually compiling libproto libraries and replacing existing /usr/lib64/libproto* please report ASAP if you see that anything is broken. Best, Predrag From predragp at cs.cmu.edu Mon Sep 12 11:04:08 2016 From: predragp at cs.cmu.edu (Predrag Punosevac) Date: Mon, 12 Sep 2016 11:04:08 -0400 Subject: Printing fixed In-Reply-To: References: <20160831165721.BjsKXbf-4%predragp@cs.cmu.edu> Message-ID: <20160912150408.GTO453y78%predragp@cs.cmu.edu> jieshic at andrew.cmu.edu wrote: > > > > Hi Predrag, > Do you have any idea why printer is so slow in > processing jobs? Sometimes my printing jobs were even lost. > Welcome to the wonderful world of CS CMU printer server printing. They plugged port 9100 so we can't print jet direct anymore. There is nothing I can do about their services. If this persists I propose that we buy $300 laser printer from our own fonds and just put it in my office. Predrag > > Thanks, > Jessie > > I can print now which means that you guys will be able to print as > soon > > as I propagate my configuration file (give or take 5-10 > minutes). > > > > Predrag > > > > From predragp at cs.cmu.edu Mon Sep 12 13:24:17 2016 From: predragp at cs.cmu.edu (Predrag Punosevac) Date: Mon, 12 Sep 2016 13:24:17 -0400 Subject: Fwd: [Power Outage] SCS Wean Hall machine room - October 4th 2016 Message-ID: <20160912172417.YWsg4FJd8%predragp@cs.cmu.edu> Dear Autonians, Please see below. This is going to affect us in the major way! Please start planning for the down time of at least two days starting October 4th. I will try to use necessary down time to do the work which normally requires rebooting. That is why I am saying that we should allocate no less than 2 days for this event. Sincerely, Predrag Punosevac -------- Original Message -------- From: Edward Walter Subject: [Power Outage] SCS Wean Hall machine room - October 4th 2016 To: Edward J Walter Date: Mon, 12 Sep 2016 08:01:44 -0400 SCS Computing Facilities and FMS are planning a partial power outage in the SCS Wean Hall machine room. We expect this work to begin on Oct 4th, 2016 and to take less than 24 hours. The outage may run into 48 hours in the event that the electrical contractor encounters something unexpected. The following servers or computational clusters will be affected by this power outage: Affected clusters: ACTR.HPC1.CS.CMU.EDU AUTON COMA.HPC1.CS.CMU.EDU CORTEX.ML.CMU.EDU LATEDAYS.ANDREW.CMU.EDU PSYCH-O.HPC1.CS.CMU.EDU ROCKS.IS.CS.CMU.EDU WORKHORSE.LTI.CS.CMU.EDU YODA.GRAPHICS.CS.CMU.EDU Affected servers: OMEPSLID.COMPBIO SLIF.COMPBIO PACIFIC.DB GPUSERVER.PERCEPTION GPUSERVER2.PERCEPTION GPUSERVER3.PERCEPTION GPUSERVER5.PERCEPTION GPUSERVER6.PERCEPTION GPUSERVER7.PERCEPTION DENVER.LTI LOR.LTI MIAMI.LTI SASKIA.ML MARTEN.ML JAN.ML ARNOUT.ML LYSBET.ML FLORIS.ML Please contact the SCS Help Desk at x8-4231 or send mail to help at cs.cmu.edu with any questions or concerns regarding this maintenance period. Thank you for your attention, SCS Help Desk From awd at cs.cmu.edu Tue Sep 13 11:19:39 2016 From: awd at cs.cmu.edu (Artur Dubrawski) Date: Tue, 13 Sep 2016 11:19:39 -0400 Subject: Annual Auton Lab Picnic: Saturday October 1st at Schenley Park Message-ID: <07b2034c-153b-d4e3-5e46-8e1b65f8a2ac@cs.cmu.edu> Dear Autonians, We will be celebrating the 23rd anniversary of the Lab this year at a nearby location. We have reserved Vietnam Veterans Pavilion at the Schenley Park: https://www.google.com/maps/place/Vietnam+Veterans+Pavilion/@40.434036,-79.9453924,17z/data=!4m12!1m6!3m5!1s0x8834f18b186c4403:0xd24a0faef8f7e126!2sSchenley+Park!8m2!3d40.4318311!4d-79.9462078!3m4!1s0x0:0x4bcca3ccfd92c919!8m2!3d40.4338281!4d-79.9439894 We have it booked from 11am till 9pm, we will have lunch food, and the Auton Lab Entertainment Committee led by our CEO (Chief Entertainment Officer) Jessie is working on the - you've guessed it - program of entertainment and activities. Would you please go to this google doc form to rsvp and provide information useful for planning: https://docs.google.com/forms/d/e/1FAIpQLSdf75XSHzcACDi9eiiUWjVSFuUzTT4mwTPNBWr1kIH7z0H69Q/viewform?usp=send_form Quick hint for the new members of our team: each year we run a contest for the most fitting slogan to put on the Auton Lab birthday cake. The bids are judged by a few wise men and the author of the winning slogan feels the pride of seeing it on the cake and basks in glory forever. Cheers! Artur From predragp at cs.cmu.edu Wed Sep 14 08:02:34 2016 From: predragp at cs.cmu.edu (Predrag Punosevac) Date: Wed, 14 Sep 2016 08:02:34 -0400 Subject: Cannot ssh into cluster In-Reply-To: References: Message-ID: <20160914120234.jN-J2qglC%predragp@cs.cmu.edu> Kirthevasan Kandasamy wrote: > Hi Predrag, > > Is the cluster down? > I cannot ssh into it or even ping any of the machines. > > samy No it is not! There are three machines you can ssh from your home network lop1.autonlab.org (cpu.autonlab.org) checked works fine shell.autonlab.org checked works fine and bash.autonlab.org (which indeed appear to be down at the moment). I will check as soon as I get to university as that is my desktop machine. Oh by the way I checked the internal machines as well and they are 100% up and functional. Please log into http://monit.autonlab.org:8080 with userneme: Auton password:Dr.Who to see for yourself. I didn't check just Monit I actaully logged with my normal user credentials with bunch of computing nodes. Now you might be onto something. Please check this post of mine https://marc.info/?l=openbsd-misc&m=147338117217028&w=2 Nobody was able to help me and with the reason. I did little tracerouting as that happen again and the network was dying in Virginia out of all places. I talked to my ISP and they saw it too. Why would I go through Virginia to reach 128.0.0.0/16 CMU addresses? Strangely enough I think I have the answer. There was a massive security compromise of NREC network which is still being investigated by three letter agencies. I learned about that incident about a week ago. At this point I have reason to believe that all CMU traffic is going through Virginia and that you should talk to somebody at NSA about your ssh problems. Predrag From predragp at imap.srv.cs.cmu.edu Thu Sep 15 16:27:35 2016 From: predragp at imap.srv.cs.cmu.edu (Predrag Punosevac) Date: Thu, 15 Sep 2016 16:27:35 -0400 Subject: shell.autonlab.org Message-ID: Dear Autonians, shell.autonlab.org is down due to the work currently done in DMZ. You have two other shell gateways lop1.autonlab.org (aka. cpu.autonlab.org) and bash.autonlab.org The work in DMZ is necessary so that we can prepare RACK 3 for new GPU computing nodes which we are receiving. I am hoping to complete the work by October 4th and use 2 day downtime to do final migration of few machines necessary for power management. When everything is finished and done the lab will have 5 GPU computing nodes out of which three will have 3 GPU cards and the last two might even have up to 10 GPU cards. Thanks, Predrag From predragp at cs.cmu.edu Thu Sep 15 21:22:53 2016 From: predragp at cs.cmu.edu (Predrag Punosevac) Date: Thu, 15 Sep 2016 21:22:53 -0400 Subject: shell.autonlab.org fixed! Message-ID: <20160916012253.qInRRngXP%predragp@cs.cmu.edu> Dear Autonians, I just fixed shell.autonlab.org so you can use it. Dugal mentioned that lop1 didn't work for him. It works for me from my home which means that all tree shell gateways should be available. I am really trying to shild you from any disruptions caused by the work that is currently being performed. predrag at oko$ uname -a OpenBSD oko.bagdala2.net 6.0 GENERIC.MP#0 amd64 predrag at oko$ ssh lop1 Host key fingerprint is SHA256:cggjV+1rpnNLpCL/I23f8vwcgtfSbrkzKHSvsxp6R+I +---[ECDSA 256]---+ | .. | | . . | | . + . | | o o .. | | o S. | | =*.oo | | . ...*+++o+. | | o..*oE=+=*. | | .+o*+B**=+ | +----[SHA256]-----+ Last login: Thu Sep 15 21:17:51 2016 from dynamic-acs-24-154-54-187.zoominternet.net OpenBSD 5.6 (GENERIC.MP) #333: Fri Aug 8 00:20:21 MDT 2014 Welcome to Auton Lab shell gateway! This gateway is only meant to provide Auton Lab LDAP account holders with the complete access to the Lab's computing infrastructure. It also can be used as a X2Go proxy to provide GUI access to computing nodes. To connect for example to low1 with X2Go client open the shell on your computer and type ssh -L 8080:low1.int.autonlab.org:22 lop1.autonlab.org Then open the X2Go client and for the name of ssh host use localhost and change default port number to 8080. From predragp at cs.cmu.edu Thu Sep 15 22:43:29 2016 From: predragp at cs.cmu.edu (Predrag Punosevac) Date: Thu, 15 Sep 2016 22:43:29 -0400 Subject: python3 and virtualenv In-Reply-To: References: Message-ID: <20160916024329.5jbIuZSK3%predragp@cs.cmu.edu> Kirthevasan Kandasamy wrote: > Hi Predrag, > > Can you please install python3 in the machines? Please check /opt/rh/rh-python35/root/bin use virtualenv-3.5 and pip3.5 to set up your environment. You must use absolute paths or adjust your shell profile. python35 is only available on the computing nodes running RHEL 7.2. I am planning to phase out 6.8 firstly from desktops and then remaining computing nodes gradually. Predrag P.S. Before people jump on 3.5 I must remind you that 3.5 is a new language comparing to 2.7 branch. If you are writing new code great. However there is a tons of code which is written for 2.7 and will not work on 3.5. Porting is highly non-trivial and you will have to work with upstream. > > I wanted to set up a virtualenv with python3 but I get this error: > kkandasa at gpu1$ virtualenv --always-copy -p python3 test1 > The executable python3 (from --python=python3) does not exist > > But even if I try without python3 I get the followign error: > > kkandasa at gpu1$ virtualenv --always-copy test1 > Cannot find file lib (bad symlink) > New python executable in test1/bin/python > Traceback (most recent call last): > File "/usr/bin/virtualenv", line 3, in > virtualenv.main() > File "/usr/lib/python2.7/site-packages/virtualenv.py", line 821, in main > symlink=options.symlink) > File "/usr/lib/python2.7/site-packages/virtualenv.py", line 956, in > create_environment > site_packages=site_packages, clear=clear, symlink=symlink)) > File "/usr/lib/python2.7/site-packages/virtualenv.py", line 1377, in > install_python > shutil.copyfile(py_executable_base, full_pth) > File "/usr/lib64/python2.7/shutil.py", line 82, in copyfile > with open(src, 'rb') as fsrc: > IOError: [Errno 2] No such file or directory: 'python' > > I will mostly be using gpu1 and gpu2 for this project. > > samy From awd at cs.cmu.edu Fri Sep 16 13:30:55 2016 From: awd at cs.cmu.edu (Artur Dubrawski) Date: Fri, 16 Sep 2016 13:30:55 -0400 Subject: Maria De Arteaga wins an award Message-ID: Dear Autonians, I am distinctively pleased to report that Maria has just won the 1st place Innovation Award on Data Science funded by London Innovation Society for her paper titled "Discovery of complex anomalous patterns of sexual violence in El Salvador" presented at the Data for Policy 2016 conference in London, UK. Congratulations Maria! Artur From predragp at imap.srv.cs.cmu.edu Fri Sep 16 17:28:16 2016 From: predragp at imap.srv.cs.cmu.edu (Predrag Punosevac) Date: Fri, 16 Sep 2016 17:28:16 -0400 Subject: lop1 to recycling bin Message-ID: Dear Autonians, Lop1 is going to recycling bin due to its age and also to create rack space with appropriate power supply for those 5 GPU computing nodes. For now we will have two shell gateways shell.autonlab.org and bash.autonlab.org If necessary I will add more shell gateways (traditionally before I came to Auton Lab there were 2 shell gateways). Best, Predrag From rk2x at cmu.edu Fri Sep 16 17:45:26 2016 From: rk2x at cmu.edu (Ramayya Krishnan) Date: Fri, 16 Sep 2016 21:45:26 +0000 Subject: Maria De Arteaga wins an award In-Reply-To: References: Message-ID: <5d724c0593824fcf8972b48cdc07a555@PGH-MSGMLT-03.andrew.ad.cmu.edu> Congratulations Maria! Congratulations Artur. Sent from my iPhone > On Sep 16, 2016, at 1:30 PM, Artur Dubrawski wrote: > > Dear Autonians, > > I am distinctively pleased to report that Maria has just won the 1st place Innovation Award on Data Science funded by London Innovation Society for her paper titled "Discovery of complex anomalous patterns of sexual violence in El Salvador" presented at the Data for Policy 2016 conference in London, UK. > > Congratulations Maria! > > Artur From yichongx at andrew.cmu.edu Fri Sep 16 17:49:39 2016 From: yichongx at andrew.cmu.edu (Yichong Xu) Date: Fri, 16 Sep 2016 17:49:39 -0400 Subject: Maria De Arteaga wins an award In-Reply-To: References: Message-ID: <9345E0E9-E61F-40A1-BFF8-E409A6368BC1@andrew.cmu.edu> Congratulations! Thanks, Yichong > ? 2016?9?16????1:30?Artur Dubrawski ??? > > Dear Autonians, > > I am distinctively pleased to report that Maria has just won the 1st place Innovation Award on Data Science funded by London Innovation Society for her paper titled "Discovery of complex anomalous patterns of sexual violence in El Salvador" presented at the Data for Policy 2016 conference in London, UK. > > Congratulations Maria! > > Artur -------------- next part -------------- An HTML attachment was scrubbed... URL: From mille856 at gmail.com Fri Sep 16 17:54:08 2016 From: mille856 at gmail.com (Kyle Miller) Date: Fri, 16 Sep 2016 17:54:08 -0400 Subject: Maria De Arteaga wins an award In-Reply-To: References: Message-ID: Way to go Maria! On Sep 16, 2016 1:31 PM, "Artur Dubrawski" wrote: > Dear Autonians, > > I am distinctively pleased to report that Maria has just won the 1st place > Innovation Award on Data Science funded by London Innovation Society for > her paper titled "Discovery of complex anomalous patterns of sexual > violence in El Salvador" presented at the Data for Policy 2016 conference > in London, UK. > > Congratulations Maria! > > Artur > -------------- next part -------------- An HTML attachment was scrubbed... URL: From prateek1.tandon at gmail.com Fri Sep 16 19:10:05 2016 From: prateek1.tandon at gmail.com (Prateek Tandon) Date: Fri, 16 Sep 2016 16:10:05 -0700 Subject: Maria De Arteaga wins an award In-Reply-To: References: Message-ID: Congrats Maria! Awesome Job! -Prateek On Fri, Sep 16, 2016 at 2:54 PM, Kyle Miller wrote: > Way to go Maria! > > On Sep 16, 2016 1:31 PM, "Artur Dubrawski" wrote: > >> Dear Autonians, >> >> I am distinctively pleased to report that Maria has just won the 1st >> place Innovation Award on Data Science funded by London Innovation Society >> for her paper titled "Discovery of complex anomalous patterns of sexual >> violence in El Salvador" presented at the Data for Policy 2016 conference >> in London, UK. >> >> Congratulations Maria! >> >> Artur >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From junieroliva at gmail.com Fri Sep 16 20:32:19 2016 From: junieroliva at gmail.com (Junier Oliva) Date: Fri, 16 Sep 2016 20:32:19 -0400 Subject: Maria De Arteaga wins an award In-Reply-To: References: Message-ID: Awesome, congrats! On Fri, Sep 16, 2016 at 7:10 PM, Prateek Tandon wrote: > Congrats Maria! Awesome Job! > > -Prateek > > On Fri, Sep 16, 2016 at 2:54 PM, Kyle Miller wrote: > >> Way to go Maria! >> >> On Sep 16, 2016 1:31 PM, "Artur Dubrawski" wrote: >> >>> Dear Autonians, >>> >>> I am distinctively pleased to report that Maria has just won the 1st >>> place Innovation Award on Data Science funded by London Innovation Society >>> for her paper titled "Discovery of complex anomalous patterns of sexual >>> violence in El Salvador" presented at the Data for Policy 2016 conference >>> in London, UK. >>> >>> Congratulations Maria! >>> >>> Artur >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From otiliastr at gmail.com Fri Sep 16 21:10:47 2016 From: otiliastr at gmail.com (Otilia Stretcu) Date: Fri, 16 Sep 2016 21:10:47 -0400 Subject: Maria De Arteaga wins an award In-Reply-To: References: Message-ID: Congrats, Maria!!! On Fri, Sep 16, 2016 at 8:32 PM, Junier Oliva wrote: > Awesome, congrats! > > On Fri, Sep 16, 2016 at 7:10 PM, Prateek Tandon > wrote: > >> Congrats Maria! Awesome Job! >> >> -Prateek >> >> On Fri, Sep 16, 2016 at 2:54 PM, Kyle Miller wrote: >> >>> Way to go Maria! >>> >>> On Sep 16, 2016 1:31 PM, "Artur Dubrawski" wrote: >>> >>>> Dear Autonians, >>>> >>>> I am distinctively pleased to report that Maria has just won the 1st >>>> place Innovation Award on Data Science funded by London Innovation Society >>>> for her paper titled "Discovery of complex anomalous patterns of sexual >>>> violence in El Salvador" presented at the Data for Policy 2016 conference >>>> in London, UK. >>>> >>>> Congratulations Maria! >>>> >>>> Artur >>>> >>> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From predragp at cs.cmu.edu Sun Sep 18 21:54:29 2016 From: predragp at cs.cmu.edu (Predrag Punosevac) Date: Sun, 18 Sep 2016 21:54:29 -0400 Subject: bhyve.int.autonlab.org down (git, monit, sftp) Message-ID: <20160919015429.4RSm_k9r4%predragp@cs.cmu.edu> Dear Autonians, It appears that the Jail host bhyve.int.autonlab.org is down which means that the following services are currently not available (git/gogs, monit, sftp). I will physically inspect the host first thing in the morning. The hardware is brand new and really good so my first hunch is to blame some kind power failure. Predrag From predragp at cs.cmu.edu Mon Sep 19 12:04:00 2016 From: predragp at cs.cmu.edu (Predrag Punosevac) Date: Mon, 19 Sep 2016 12:04:00 -0400 Subject: bhyve fried power supply Message-ID: <20160919160400.V9mZuj3wX%predragp@cs.cmu.edu> Dear Autonians, Power supply appears to be fried on bhyve. Expected down time for the services it provides is at least 24h. Predrag From predragp at imap.srv.cs.cmu.edu Mon Sep 19 16:25:51 2016 From: predragp at imap.srv.cs.cmu.edu (Predrag Punosevac) Date: Mon, 19 Sep 2016 16:25:51 -0400 Subject: bhyve (Git/Gogs) is fixed! Message-ID: bhyve is fixed! Git/Gogs is available again. Predrag From sheath at andrew.cmu.edu Tue Sep 20 16:48:19 2016 From: sheath at andrew.cmu.edu (Simon Heath) Date: Tue, 20 Sep 2016 16:48:19 -0400 Subject: Server downtime Message-ID: Hi all, We're doing some infrastructure work which will make ZFS and cvs/svn/git temporarily unavailable. It shouldn't take long, we'll send out a notification when it's done. Thanks, Simon -- Simon Heath, Research Programmer and Analyst Robotics Institute - Auton Lab Carnegie Mellon University sheath at andrew.cmu.edu From predragp at cs.cmu.edu Tue Sep 20 17:42:33 2016 From: predragp at cs.cmu.edu (Predrag Punosevac) Date: Tue, 20 Sep 2016 17:42:33 -0400 Subject: Upgrade completed Message-ID: <20160920214233.nGyrSzdx5%predragp@cs.cmu.edu> All network related servers as of 5:15 PM today run latest and greatest OpenBSD 6.0. All services are restored. If you are using Linux based desktop issued by the Auton Lab please reboot the machine if it feels slow (Jarod, Saswati). I will reboot all machines I have access to tonight. Best, Predrag P.S. OpenVPN is stateless protocol so the daemon needs to be restarted. Linux thinks that NFS is stateful protocol so you will have same stale file handles. From predragp at cs.cmu.edu Wed Sep 21 08:09:27 2016 From: predragp at cs.cmu.edu (Predrag Punosevac) Date: Wed, 21 Sep 2016 08:09:27 -0400 Subject: Reminder: svn/git port forwards not working In-Reply-To: <199accab-bc24-2c67-7b4d-88896d59d499@ri.cmu.edu> References: <20160920214233.nGyrSzdx5%predragp@cs.cmu.edu> <199accab-bc24-2c67-7b4d-88896d59d499@ri.cmu.edu> Message-ID: <20160921120927.o4ENab31U%predragp@cs.cmu.edu> Rob Maclachlan wrote: > ram at TREMOR ~/Documents/work/rad/rtba > $ git pull > ssh_exchange_identification: read: Connection reset by peer > fatal: Could not read from remote repository. > > Please make sure you have the correct access rights > and the repository exists. > > ram at TREMOR ~/Documents/work/rad/rtba > $ > > In other window: > > ram at TREMOR ~ > $ cat `which ssh_ports` > #!/bin/bash > ssh -L6667:ari.int.autonlab.org:22 -L2222:svnhub.int.autonlab.org:3690 -L7777:git.autonlab.org:2222 $1 > > ram at TREMOR ~ > $ ssh_ports bash.autonlab.org > Last login: Wed Sep 21 06:04:03 2016 from pool-74-98-200-86.pitbpa.fios.verizon.net > -bash-4.1$ channel 7: open failed: administratively prohibited: open failed > git.autonlab.org doesn't exist anymore and the port 2222 on the firewall leading to git.int.autonlab.org is closed. DNS record git.autonlab.org is deleted. Per Anthony Wertz we decided to force people to ssh to bash before they can check out from the git.int.autonlab.org with ssh or with http. The behavior is consistent across the network internal network and you can ssh or http check code from git.int.autonlab.org from any of the computing nodes. We are releasing new Wiki tomorrow and this will be well documented. This is not a big deal if you want to check out the code from outside the lab because you can use bash address in the proxy option of ssh. Please read the man pages for now. svnhub will be decommission soon and port 3690 will be closed. If you are trying to move some of your code from svn to git please talk to Simon and Anthony. SVN and Git have radically different behavior and we will do code migration to ensure consistency of the transfer. You also using bunch of other ports above which are closed on all our network equipment although they might work on Linux computing nodes which have permissive iptables settings. Predrag > Rob From predragp at cs.cmu.edu Wed Sep 21 08:15:21 2016 From: predragp at cs.cmu.edu (Predrag Punosevac) Date: Wed, 21 Sep 2016 08:15:21 -0400 Subject: Meaning of Lop1 Recycled? In-Reply-To: References: Message-ID: <20160921121521.AL5kkYbRE%predragp@cs.cmu.edu> Mallory Nobles wrote: > Hi Predrag, > You can reclaim LOU1. I do have a quick question I hope you can answer. I > used to log onto WinSCP through lop1 to transfer files between my > /zfsneill/home/mnobles directory and my desktop. When I log onto WinSCP > through shell.autonlab.org, I don't see the files. I've tried looking > around the directory structure, but can't seem to find the directory with > my files. Is there a step I need to take to be able to access those files? I will rebuild lop1 with new hardware today. Actually it is already rebuilt I just need to configure few things. You will be able to continue to copy files as you have done in the past probably by 6:00 PM. shell.autonlab.org had a fake home directories. NFS exports from file servers have never being mounted there. It was primarily used as the access to svnhub in the case file servers become unavailable. svnhub is going down soon when we complete migration go Git and shell.autonlab.org is going down tomorrow when I complete the cleaning of the third rack for new GPU units which we are receiving this week. Best, Predrag > Thanks, > Mallory From jieshic at andrew.cmu.edu Wed Sep 21 11:31:02 2016 From: jieshic at andrew.cmu.edu (jieshic at andrew.cmu.edu) Date: Wed, 21 Sep 2016 11:31:02 -0400 Subject: Annual Auton Lab Picnic: Saturday October 1st at Schenley Park In-Reply-To: <07b2034c-153b-d4e3-5e46-8e1b65f8a2ac@cs.cmu.edu> References: <07b2034c-153b-d4e3-5e46-8e1b65f8a2ac@cs.cmu.edu> Message-ID: <4788ac801943b5fcf02f771e948e1a70.squirrel@webmail.andrew.cmu.edu> Hi Autonians, Pls register on the following form ASAP so that our Auton Lab Entertainment Committee could prepare enough food, drinks, games and GIFTS for you and your beloved family members and guests. https://docs.google.com/forms/d/e/1FAIpQLSdf75XSHzcACDi9eiiUWjVSFuUzTT4mwTPNBWr1kIH7z0H69Q/viewform?usp=send_form If you have any idea for the picnic, pls don't hesitate to let me know. Let's make it a fun picnic together! Thank you! Best, Chief Entertainment Officer Jessie > Dear Autonians, > > We will be celebrating the 23rd anniversary of the Lab this year at a > nearby location. > We have reserved Vietnam Veterans Pavilion at the Schenley Park: > > https://www.google.com/maps/place/Vietnam+Veterans+Pavilion/@40.434036,-79.9453924,17z/data=!4m12!1m6!3m5!1s0x8834f18b186c4403:0xd24a0faef8f7e126!2sSchenley+Park!8m2!3d40.4318311!4d-79.9462078!3m4!1s0x0:0x4bcca3ccfd92c919!8m2!3d40.4338281!4d-79.9439894 > > We have it booked from 11am till 9pm, we will have lunch food, and the > Auton Lab Entertainment > Committee led by our CEO (Chief Entertainment Officer) Jessie is working > on the - you've guessed it - program > of entertainment and activities. > > Would you please go to this google doc form to rsvp and provide > information useful for planning: > > https://docs.google.com/forms/d/e/1FAIpQLSdf75XSHzcACDi9eiiUWjVSFuUzTT4mwTPNBWr1kIH7z0H69Q/viewform?usp=send_form > > Quick hint for the new members of our team: each year we run a contest > for the most fitting slogan to put > on the Auton Lab birthday cake. The bids are judged by a few wise men > and the author of the winning slogan > feels the pride of seeing it on the cake and basks in glory forever. > > Cheers! > Artur > -------------- next part -------------- An HTML attachment was scrubbed... URL: From predragp at cs.cmu.edu Wed Sep 21 21:42:17 2016 From: predragp at cs.cmu.edu (Predrag Punosevac) Date: Wed, 21 Sep 2016 21:42:17 -0400 Subject: LOP1 back in business Message-ID: <20160922014217.Hz1QRxe-n%predragp@cs.cmu.edu> Dear Autonians, As I promised earlier lop1 is configured. Before you attempt to login make sure your delete the record for old lop1 from ~/.ssh/known_hosts otherwise you will get a very scary message from your computer and you will not be able to log into the lop1.autonlab.org. Upon first logging you will be asked to accept new cryptographic key The new RSA key fingerprint is: 9d:72:b7:ab:e3:dd:aa:25:55:2e:9a:3e:78:b5:c0:4b ECDSA key fingerprint is SHA256:pvrXGlYOcrBOtI5b7xt4sItRkIbqRMhJ+qLRlTrgIts Your home directories are NFS mounted which means that this shell gateway can be used to transfer files from your home computers. lop1 is capable of seeing host in both LAN and DMZ zone. Please use full name to access the hosts in DMZ zone. Best, Predrag From predragp at cs.cmu.edu Thu Sep 22 10:46:59 2016 From: predragp at cs.cmu.edu (Predrag Punosevac) Date: Thu, 22 Sep 2016 10:46:59 -0400 Subject: www.autonlab.org and shell Message-ID: <20160922144659.GoL25byIT%predragp@cs.cmu.edu> Dear Autonians, xen is dead and with it our old website hosted by Daisy. The new Auton Lab website is based on DokuWiki. At the moment the old content is a bit disorganized but Simon and I are working on it. Old shell.autonlab.org which had those fake home directories is gone as well. We are reclaiming rack space for new GPU servers and can't effort to keep 15 year old hardware which should have being decomissioned long time ago. Best, Predrag From predragp at imap.srv.cs.cmu.edu Thu Sep 22 17:12:30 2016 From: predragp at imap.srv.cs.cmu.edu (Predrag Punosevac) Date: Thu, 22 Sep 2016 17:12:30 -0400 Subject: Fwd: [Power Outage] Wean Hall machine room postponed In-Reply-To: <0aee22ff-fd00-56d4-4563-f912cfe06184@cs.cmu.edu> References: <0aee22ff-fd00-56d4-4563-f912cfe06184@cs.cmu.edu> Message-ID: <897666d3e52f3bda4e43963ec87dda6a@imap.srv.cs.cmu.edu> Dear Autonians, I am happy to report that all the work needed on rack 3 has being completed. lop1 and bash should be available 24/7. In the case we need another shell gateway that would be no problem to add. I have already received 3 GPU cards and one of new servers is on the way. 17.3 kW PDU I installed today in rack 3 is not connected to the power grid but we have the circuit and it will be done as soon as we get the new server. I also learned today that electric contractors who are hired to preform the work on October 4th are trying to postpone things for at least 2 weeks so tentatively we are looking at October 15th now. That is not going to affect us as I already ensured the circuit for new GPU servers. Please make your plans accordingly. Finally, we will address printing problem in Auton Lab old fashion way. We will purchase a group work printer for $200 just for Auton Lab and I will maintain it. You are still welcome to send the your jobs to CS CMU facilities printers but you will have to take your complaints directly to them. Best, Predrag -------- Original Message -------- Subject: [Power Outage] SCS Wean Hall machine room - October 4th 2016 Date: 2016-09-12 08:01 From: Edward Walter To: Edward J Walter SCS Computing Facilities and FMS are planning a partial power outage in the SCS Wean Hall machine room. We expect this work to begin on Oct 4th, 2016 and to take less than 24 hours. The outage may run into 48 hours in the event that the electrical contractor encounters something unexpected. The following servers or computational clusters will be affected by this power outage: Affected clusters: ACTR.HPC1.CS.CMU.EDU AUTON COMA.HPC1.CS.CMU.EDU CORTEX.ML.CMU.EDU LATEDAYS.ANDREW.CMU.EDU PSYCH-O.HPC1.CS.CMU.EDU ROCKS.IS.CS.CMU.EDU WORKHORSE.LTI.CS.CMU.EDU YODA.GRAPHICS.CS.CMU.EDU Affected servers: OMEPSLID.COMPBIO SLIF.COMPBIO PACIFIC.DB GPUSERVER.PERCEPTION GPUSERVER2.PERCEPTION GPUSERVER3.PERCEPTION GPUSERVER5.PERCEPTION GPUSERVER6.PERCEPTION GPUSERVER7.PERCEPTION DENVER.LTI LOR.LTI MIAMI.LTI SASKIA.ML MARTEN.ML JAN.ML ARNOUT.ML LYSBET.ML FLORIS.ML Please contact the SCS Help Desk at x8-4231 or send mail to help at cs.cmu.edu with any questions or concerns regarding this maintenance period. Thank you for your attention, SCS Help Desk From predragp at cs.cmu.edu Fri Sep 23 22:45:18 2016 From: predragp at cs.cmu.edu (Predrag Punosevac) Date: Fri, 23 Sep 2016 22:45:18 -0400 Subject: MATLAB 2016b released Message-ID: <20160924024518.N2zmCXAk7%predragp@cs.cmu.edu> Dear Autonians, MATLAB 2016b has been released 8 days ago. We will have to start upgrading MATLAB version starting Monday as the current license will start expiring within a week. In plain English I have no choice but to upgrade as the current installation of MATLAB will simply stop working. The highest priority will be given to people who actually use MATLAB for their daily work. If you are one of them please notify me by e-mail or even better stop by my office on Monday with your laptop if that is where you want to have MATLAB installed. If you have something running and you are waiting for scripts to finish please notify me by Monday so that I can plan upgrade around your work. Predrag From predragp at imap.srv.cs.cmu.edu Mon Sep 26 11:45:48 2016 From: predragp at imap.srv.cs.cmu.edu (Predrag Punosevac) Date: Mon, 26 Sep 2016 11:45:48 -0400 Subject: X2Go server on Springdale 7.2 Message-ID: Dear Autonians, It appears that when you try to use icewm desktop on Springdale 7.2 machines (gpu1, gpu2, ari, foxconn, ...) the session crashes. That is probably caused by some nightly update. There is a very simple work around for this problem. Please select to launch a single application xterm instead of icewm. Once you have an xterm you can launch multiple xterms and detach them as $ xterm & In each one of those you can have a separate graphics application. Best, Predrag From predragp at cs.cmu.edu Tue Sep 27 22:50:54 2016 From: predragp at cs.cmu.edu (predragp at cs.cmu.edu) Date: Wed, 28 Sep 2016 02:50:54 +0000 (UTC) Subject: Lou1 now available Message-ID: <7B1F10ED8F1045F6.00aaf355-2885-4c3a-a410-e6d756faeed3@mail.outlook.com> Dear Autonians, Some of you remember lou1 computing node which had 48 CPUs and 128 GB of RAM. The server was withdrawn from general use few years ago as we needed additional virtual hosts. We no longer need lou1 as a virtual host so I rebuilt the server. Please feel free to use. Predrag P.S. I was not able to get MATLAB working due to the chane in the installer and license server. That is top priority now. Get Outlook for Android -------------- next part -------------- An HTML attachment was scrubbed... URL: From jieshic at andrew.cmu.edu Fri Sep 30 13:21:11 2016 From: jieshic at andrew.cmu.edu (jieshic at andrew.cmu.edu) Date: Fri, 30 Sep 2016 13:21:11 -0400 Subject: Annual Auton Lab Picnic: Saturday October 1st at Schenley Park In-Reply-To: <07b2034c-153b-d4e3-5e46-8e1b65f8a2ac@cs.cmu.edu> References: <07b2034c-153b-d4e3-5e46-8e1b65f8a2ac@cs.cmu.edu> Message-ID: Hi Everyone, Tomorrow's picnic will start at 11:30am. The shelter is Vietnam Veterans Pavilion at the Schenley Park. We'll have lunch with BBQ food, a large Taro cake, and beers & wines (pls bring your ID if you'll drink :P ). We have prepared some long games and you are also welcome to bring your bikes, games, cameras, etc.. BTW, here's the weather forecast for your reference. "Showers in the morning, then partly cloudy in the afternoon. Thunder possible. High 73F. Winds SSE at 5 to 10 mph. Chance of rain 50%."? (source: www.weather.com) Pls feel free to let me know if you have any question.? Looking forward to seeing you tomorrow! Cheers, Jessie > Dear Autonians, > > We will be celebrating the 23rd anniversary of the Lab this year at a > nearby location. > We have reserved Vietnam Veterans Pavilion at the Schenley Park: > > https://www.google.com/maps/place/Vietnam+Veterans+Pavilion/@40.434036,-79.9453924,17z/data=!4m12!1m6!3m5!1s0x8834f18b186c4403:0xd24a0faef8f7e126!2sSchenley+Park!8m2!3d40.4318311!4d-79.9462078!3m4!1s0x0:0x4bcca3ccfd92c919!8m2!3d40.4338281!4d-79.9439894 > > We have it booked from 11am till 9pm, we will have lunch food, and the > Auton Lab Entertainment > Committee led by our CEO (Chief Entertainment Officer) Jessie is working > on the - you've guessed it - program > of entertainment and activities. > > Would you please go to this google doc form to rsvp and provide > information useful for planning: > > https://docs.google.com/forms/d/e/1FAIpQLSdf75XSHzcACDi9eiiUWjVSFuUzTT4mwTPNBWr1kIH7z0H69Q/viewform?usp=send_form > > Quick hint for the new members of our team: each year we run a contest > for the most fitting slogan to put > on the Auton Lab birthday cake. The bids are judged by a few wise men > and the author of the winning slogan > feels the pride of seeing it on the cake and basks in glory forever. > > Cheers! > Artur > -------------- next part -------------- An HTML attachment was scrubbed... URL: From predragp at cs.cmu.edu Fri Sep 30 22:26:37 2016 From: predragp at cs.cmu.edu (Predrag Punosevac) Date: Fri, 30 Sep 2016 22:26:37 -0400 Subject: MATLAB upgrade status Message-ID: <20161001022637.d0AGr-77S%predragp@cs.cmu.edu> Dear Autonians, The MATLAB has being updated to the latest and the greatest 2016b on the following servers: neill3, neill4, low1, lou1, lov3 as well as nips.dmz.autonlab.org. Also on the desktops lean, loom, lake, lyra. As I type this message MATLAB is being updated on the foxconn, lot1, charity, and lov4. I need about 30 minutes to finish the last four servers. That leaves us with the following servers which I can't update because people are currently using MATLAB: gpu1, gpu2, ari, compute-0-0, compute-0-1, neill1, and neill2. MATLAB is going to stop working on these servers due to the licenses expiration within a day or two. I propose that people wind down their jobs by Sunday night on these seven servers so that I can complete upgrade by Monday morning. If you are waiting for job to finish and need that extra half a day please send me an e-mail. Predrag From ahmaurya at gmail.com Fri Sep 30 22:41:31 2016 From: ahmaurya at gmail.com (Abhinav Maurya) Date: Fri, 30 Sep 2016 22:41:31 -0400 Subject: MATLAB upgrade status In-Reply-To: <20161001022637.d0AGr-77S%predragp@cs.cmu.edu> References: <20161001022637.d0AGr-77S%predragp@cs.cmu.edu> Message-ID: Tidbit: Matlab 2016b finally allows defining functions within a script file like other scripting languages. No need to create separate files for every function or use other hacks. Regards, Abhinav On Fri, Sep 30, 2016 at 10:26 PM, Predrag Punosevac wrote: > Dear Autonians, > > The MATLAB has being updated to the latest and the greatest 2016b on the > following servers: neill3, neill4, low1, lou1, lov3 as well as > nips.dmz.autonlab.org. Also on the desktops lean, loom, lake, lyra. > As I type this message MATLAB is being updated on the foxconn, lot1, > charity, and lov4. I need about 30 minutes to finish the last four > servers. > > That leaves us with the following servers which I can't update because > people are currently using MATLAB: gpu1, gpu2, ari, compute-0-0, > compute-0-1, neill1, and neill2. MATLAB is going to stop working on > these servers due to the licenses expiration within a day or two. I > propose that people wind down their jobs by Sunday night on these seven > servers so that I can complete upgrade by Monday morning. If you are > waiting for job to finish and need that extra half a day please send me > an e-mail. > > Predrag > -------------- next part -------------- An HTML attachment was scrubbed... URL: