From chiragn at cs.cmu.edu  Sun Sep  3 15:24:32 2017
From: chiragn at cs.cmu.edu (Chirag Nagpal)
Date: Sun, 3 Sep 2017 15:24:32 -0400
Subject: Poster Tube
Message-ID: <CAKH1gVoyqH1b7JkB1Pz-XcSEPsX=11Uqow+bU8GnLEaOrm33eQ@mail.gmail.com>

Hi! does anyone have a poster tube I could borrow,

I leave on Tuesday, so would need it by, tomorrow.


-- 

*Chirag Nagpal* Graduate Student, Language Technologies Institute
School of Computer Science
Carnegie Mellon University
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/autonlab-users/attachments/20170903/d43621ff/attachment.html>

From predragp at cs.cmu.edu  Sun Sep  3 16:08:32 2017
From: predragp at cs.cmu.edu (Predrag Punosevac)
Date: Sun, 03 Sep 2017 16:08:32 -0400
Subject: Poster Tube
In-Reply-To: <CAKH1gVoyqH1b7JkB1Pz-XcSEPsX=11Uqow+bU8GnLEaOrm33eQ@mail.gmail.com>
References: <CAKH1gVoyqH1b7JkB1Pz-XcSEPsX=11Uqow+bU8GnLEaOrm33eQ@mail.gmail.com>
Message-ID: <20170903200832.3amDt_Wzc%predragp@cs.cmu.edu>

Chirag Nagpal <chiragn at cs.cmu.edu> wrote:

> Hi! does anyone have a poster tube I could borrow,

Please check the dungeon. 

Predrag

> 
> I leave on Tuesday, so would need it by, tomorrow.
> 
> 
> -- 
> 
> *Chirag Nagpal* Graduate Student, Language Technologies Institute
> School of Computer Science
> Carnegie Mellon University

From sheath at andrew.cmu.edu  Wed Sep  6 14:54:38 2017
From: sheath at andrew.cmu.edu (Simon Heath)
Date: Wed, 6 Sep 2017 14:54:38 -0400
Subject: Brainstorming and Code&Coffee schedules for Fall 2017
Message-ID: <CAGeWgZTsT=W3y_J25gtWK0qN+OpO+95gnzaiQSXuD_Ts4e1NRQ@mail.gmail.com>

Hi all,

Through the heroic efforts of Karen W, we have NSH 3001 from 12:30 to 1:30
pm most Friday's this semester.  Unless anyone objects, the plan is to have
some sort of get-together every Friday, alternating brainstorming (research
focused) with code-and-coffee (engineering focused).

Next week Fabian has some work on neural nets to present, so let's do
brainstorming this Friday (Sep 8th) if anyone has anything to brainstorm,
both for the Remaining-Usable-Life machine reliability efforts and anything
else we want to do.  Code-and-coffee will then be on Sep 15th,
brainstorming on Sep 22nd, and so on.

There are a few exceptions to the schedule:

Sept 22: 1-2 pm instead of 12:30-1:30 pm
Oct 6th: 1-2 pm instead of 12:30-1:30 pm
Nothing on Oct 20th, instead the room is booked Oct 18th 12:30-1:30 pm
Nothing on Oct 27th, instead the room is booked Oct 25th 12:30-1:30 pm

Thanks,
Simon

-- 
Simon Heath, Research Programmer and Analyst
Robotics Institute - Auton Lab
Carnegie Mellon University
sheath at andrew.cmu.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/autonlab-users/attachments/20170906/cafa53ae/attachment.html>

From predragp at andrew.cmu.edu  Fri Sep  8 21:33:20 2017
From: predragp at andrew.cmu.edu (Predrag Punosevac)
Date: Fri, 08 Sep 2017 21:33:20 -0400
Subject: GPU1 functional again
Message-ID: <20170909013320._tUinBp61%predragp@andrew.cmu.edu>

Dear Autonians,

Nvidia driver issue on GPU1 seems to be fixed with a simple reboot

root at gpu1$ nvidia-smi
Fri Sep  8 21:31:16 2017       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 384.66                 Driver Version: 384.66                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Tesla K80           Off  | 00000000:04:00.0 Off |                    0 |
| N/A   43C    P0    56W / 149W |      0MiB / 11439MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   1  Tesla K80           Off  | 00000000:05:00.0 Off |                    0 |
| N/A   34C    P0    71W / 149W |      0MiB / 11439MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   2  Tesla K80           Off  | 00000000:08:00.0 Off |                    0 |
| N/A   42C    P0    60W / 149W |      0MiB / 11439MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   3  Tesla K80           Off  | 00000000:09:00.0 Off |                    0 |
| N/A   34C    P0    73W / 149W |      0MiB / 11439MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   4  Tesla K80           Off  | 00000000:84:00.0 Off |                    0 |
| N/A   43C    P0    60W / 149W |      0MiB / 11439MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   5  Tesla K80           Off  | 00000000:85:00.0 Off |                    0 |
| N/A   35C    P0    73W / 149W |      0MiB / 11439MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   6  Tesla K80           Off  | 00000000:88:00.0 Off |                    0 |
| N/A   42C    P0    58W / 149W |      0MiB / 11439MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   7  Tesla K80           Off  | 00000000:89:00.0 Off |                    0 |
| N/A   35C    P0    71W / 149W |      0MiB / 11439MiB |     38%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+


Please let me know if I need to upgrade the driver. MATLAB seems to be
working as well. Please don't panic I know that we have 23 days left on
the license. I am waiting R2017b release to upgrade.

Cheers,
Predrag

From predragp at andrew.cmu.edu  Fri Sep  8 22:13:14 2017
From: predragp at andrew.cmu.edu (Predrag Punosevac)
Date: Fri, 08 Sep 2017 22:13:14 -0400
Subject: GPU1 functional again
Message-ID: <20170909021314.O3IxT2J9X%predragp@andrew.cmu.edu>

Dear Autonians,

Nvidia driver issue on GPU1 seems to be fixed with a simple reboot

root at gpu1$ nvidia-smi
Fri Sep  8 21:31:16 2017       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 384.66                 Driver Version: 384.66                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Tesla K80           Off  | 00000000:04:00.0 Off |                    0 |
| N/A   43C    P0    56W / 149W |      0MiB / 11439MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   1  Tesla K80           Off  | 00000000:05:00.0 Off |                    0 |
| N/A   34C    P0    71W / 149W |      0MiB / 11439MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   2  Tesla K80           Off  | 00000000:08:00.0 Off |                    0 |
| N/A   42C    P0    60W / 149W |      0MiB / 11439MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   3  Tesla K80           Off  | 00000000:09:00.0 Off |                    0 |
| N/A   34C    P0    73W / 149W |      0MiB / 11439MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   4  Tesla K80           Off  | 00000000:84:00.0 Off |                    0 |
| N/A   43C    P0    60W / 149W |      0MiB / 11439MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   5  Tesla K80           Off  | 00000000:85:00.0 Off |                    0 |
| N/A   35C    P0    73W / 149W |      0MiB / 11439MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   6  Tesla K80           Off  | 00000000:88:00.0 Off |                    0 |
| N/A   42C    P0    58W / 149W |      0MiB / 11439MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   7  Tesla K80           Off  | 00000000:89:00.0 Off |                    0 |
| N/A   35C    P0    71W / 149W |      0MiB / 11439MiB |     38%      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+


Please let me know if I need to upgrade the driver. MATLAB seems to be
working as well. Please don't panic I know that we have 23 days left on
the license. I am waiting R2017b release to upgrade.

Cheers,
Predrag

From predragp at andrew.cmu.edu  Fri Sep  8 23:55:34 2017
From: predragp at andrew.cmu.edu (Predrag Punosevac)
Date: Fri, 08 Sep 2017 23:55:34 -0400
Subject: GPU[5-7] status update
In-Reply-To: <CAPfEFgdWiNve_YLbDo_rVdwYmJEnupkrMRpMp-HcVG7F+5Quwg@mail.gmail.com>
References: <CAPfEFgdWiNve_YLbDo_rVdwYmJEnupkrMRpMp-HcVG7F+5Quwg@mail.gmail.com>
Message-ID: <20170909035534.ZO3ACp3CF%predragp@andrew.cmu.edu>

Michael Andrews <mbandrews at cmu.edu> wrote:

> Hi Predrag,
> 
> Many of the gpus on the auton machines seem to have their memory maxed out,
> and for those that do have memory (gpu3 for instance), it seems to take
> hours for a session to initialize... is this something expected?
> 
> Thanks,
> Michael

Dear Autonians,

This is a status update on the long expected hardware edition to our
lab.

GPU5 built, 4 Titan Xp cards installed, up and running. You can log into
the machine and use it. However installed driver 

http://www.nvidia.com/download/driverResults.aspx/123103/en-us

seems to be not loaded into the kernel. I did install cuda-8.0 toolkit
non the less. I am out of fuel tonight to see what is going on. If
somebody see something please let me know.

GPU6 built, 2 Titan Xp cards installed, up and running. NVidia/CUDA has
the same issue as on GPU6. I ordered 2 Titan X (note that p is missing)
to complete the server. If it is not too late I will try to switch the
order on Monday.

GPU7 built, missing GPU cards. However 4 Titan X cards ordered. You can
log and use CPUs. I am kind reluctant to switch the order to Titan Xp
due to the driver issues. Titan X has being rock solid for us. I am not
sure when I will receive the GPU cards. 

MATLAB is not installed on GPU[5-7]. I will wait a week or so for R2017b
release. We will see if this release is going to work with Titan X cards
on GPU[2-4] which use older Nvidia driver. I am not too optimistic that
MATLAB is going to work with the latest driver. 

Please don't bother me with the questions about TensorFlow, Caffe, and
similar until I sort out things with the hardware. 

Finally I think I will have enough HDDs to create additional 7 disk RAID
6 on GPU5 with the storage capacity of 10TB. The OS HDDs have scratch
space of about 2TB.

GPU6 and GPU7 just like GPU3 and GPU4 will only have 2TB scratch space. 

Cheers,
Predrag

P.S. I sent earlier two e-mails about GPU1 but I didn't see that e-mails
got posted. GPU1 driver problems appears to be fixed and the unit is
fully functional.

From chiragn at cs.cmu.edu  Wed Sep 13 11:50:43 2017
From: chiragn at cs.cmu.edu (Chirag Nagpal)
Date: Wed, 13 Sep 2017 11:50:43 -0400
Subject: SOCML
Message-ID: <CAKH1gVoPFwDZhxmScXmZT9Pj9FsJ0yyYaxWE36WRu_OADh_Y-Q@mail.gmail.com>

Thought this might be of interest to Auton Members

https://sites.google.com/view/socml/home?authuser=0



-- 

*Chirag Nagpal* Graduate Student, Language Technologies Institute
School of Computer Science
Carnegie Mellon University
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/autonlab-users/attachments/20170913/4d6bf57a/attachment.html>

From sheath at andrew.cmu.edu  Thu Sep 14 11:24:54 2017
From: sheath at andrew.cmu.edu (Simon Heath)
Date: Thu, 14 Sep 2017 10:24:54 -0500
Subject: Code and coffee, Friday sep 15th
Message-ID: <CAGeWgZTv5z03etg7UCruM8YUJKaWE_BnGMQNNivZNWhgQe93-A@mail.gmail.com>

Hey Autonians,

Just a reminder that Code and Coffee is tomorrow at 12:30 in NSH 3001.
Fabian wanted to go over some of his neural net stuff, if I recall
correctly. I am currently out of town, so people will have to self-organize
a bit.

Next Friday will be brainstorming, perhaps to dig more into
remaining-usable-life predictions. It will be at 1 pm in nsh3001 rather
than the usual 12:30 due to room conflicts.

Simon

From predragp at andrew.cmu.edu  Fri Sep 15 12:45:27 2017
From: predragp at andrew.cmu.edu (Predrag Punosevac)
Date: Fri, 15 Sep 2017 12:45:27 -0400
Subject: GPU[6-7] going down
Message-ID: <20170915164527.sMrSaZtX1%predragp@andrew.cmu.edu>

I apologize for super short notice but these machines were not fully
functional anyway. I got the reaming GPU cards which needs to be put
into the servers. I hope to have things back (still not configured)
within few hours.

Predrag

From predragp at andrew.cmu.edu  Fri Sep 15 14:27:44 2017
From: predragp at andrew.cmu.edu (Predrag Punosevac)
Date: Fri, 15 Sep 2017 14:27:44 -0400
Subject: GPU[6-7] going down  
Message-ID: <20170915182744.xQZILNGKv%predragp@andrew.cmu.edu>

Both servers are back online with complete set of four GPU cards per
machine. I need now to resolve drivers issue.

Predrag

From mdeartea at andrew.cmu.edu  Sat Sep 16 10:35:27 2017
From: mdeartea at andrew.cmu.edu (Maria De Arteaga Gonzalez)
Date: Sat, 16 Sep 2017 14:35:27 +0000
Subject: NIPS 2017 Workshop on Machine Learning for the Developing World
Message-ID: <1505572527357.75135@andrew.cmu.edu>

Dear Autonians,


This workshop might be of interest to some of you, including those working on applications of machine learning for development, as well as folks researching algorithms that work under constraints that are common to development settings, e.g. limited computational power. We would also appreciate your help spreading the word! Regards,

Call for Papers - NIPS 2017 Workshop on Machine Learning for the Developing World


*********************************************************************************
Workshop on Machine Learning for the Developing World, NIPS 2017
Date: December 8th, 2017
Location: Long Beach, California, USA
https://sites.google.com/site/ml4development/
*********************************************************************************


Call for papers:

This one-day workshop is focussed on machine learning for the developing world (ML4D). We will discuss impactful applications of machine learning to address core global development concerns, as well as limitations to ML in developing countries and novel algorithms inspired by development challenges, such as limited computational capacity.

We invite researchers to submit their recent work on this topic, including:
* Applications of ML to development issues including health, education, institutional integrity, violence mitigation, economics, societal analysis, and environment.
* Novel ML techniques inspired by limitations in developing countries.
* Limitations and risks of data science and ML for development.
* Practical systems using ML in developing regions.

Please submit 2-4 page extended abstracts to ml4d.nips at gmail.com, following the NIPS style<https://nips.cc/Conferences/2017/PaperInformation/StyleFiles> guidelines. Accepted papers will be presented as posters or contributed talks, and may optionally be published in an arXiv proceedings.


Key dates:
Submission deadline: October 20, 2017
Acceptance notification: November 1, 2017
Workshop: December 8, 2017


Speakers:
-- Emma Brunskill (Stanford)
-- Stefano Ermon (Stanford)
-- Daniel Neill (CMU)
-- Patrick Ball (Human Rights Data Analysis Group)
-- Jen Ziemke (International Network of Crisis Mappers)
-- John Quinn (UN Global Pulse)

Workshop overview:
Six billion people live in developing world countries. The unique development challenges faced by these regions have long been studied by researchers ranging from sociology to statistics and ecology to economics. With the emergence of mature machine learning methods in the past decades, researchers from many fields - including core machine learning - are increasingly turning to machine learning to study and address challenges in the developing world. This workshop is about delving into the intersection of machine learning and development research.

Machine learning present tremendous potential to development research and practice. Supervised methods can provide expert telemedicine decision support in regions with few resources; deep learning techniques can analyze satellite imagery to create novel economic indicators; NLP algorithms can preserve and translate obscure languages, some of which are only spoken. Yet, there are notable challenges with machine learning in the developing world. Data cleanliness, computational capacity, power availability, and internet accessibility are more limited than in developed countries. Additionally, the specific applications differ from what many machine learning researchers normally encounter. The confluence of machine learning's immense potential with the practical challenges posed by developing world settings has inspired a growing body of research at the intersection of machine learning and the developing world.

This one-day workshop is focussed on machine learning for the developing world, with an emphasis on developing novel methods and technical applications that address core concerns of developing regions. We will consider a wide range of development areas including health, education, institutional integrity, violence mitigation, economics, societal analysis, and environment. >From the machine learning perspective we are open to all methodologies with an emphasis on novel techniques inspired by particular use cases in the developing world.

Invited speakers will address particular areas of interest, while poster sessions and a guided panel discussion will encourage interaction between attendees. We wish to review the current approaches to machine learning in the developing world, and inspire new approaches and paradigms that can lay the groundwork for substantial innovation.



Mar?a De Arteaga
PhD Student in Machine Learning and Public Policy
Carnegie Mellon University
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/autonlab-users/attachments/20170916/f832cbc0/attachment-0001.html>

From mbarnes1 at andrew.cmu.edu  Wed Sep 20 16:38:05 2017
From: mbarnes1 at andrew.cmu.edu (Matthew Barnes)
Date: Wed, 20 Sep 2017 20:38:05 +0000
Subject: Theano on GPU machines?
Message-ID: <CAB7OVwD+swWBKVrZ+Gz10OOXzx7Pqqtcr2tfgyL1cro-D64mtg@mail.gmail.com>

Has anyone here successfully run Theano on the GPU machines? I'm able to
run on the cpu on machines 1, 2 and 4, but its missing some packages to run
on the GPU.

- Matt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/autonlab-users/attachments/20170920/4b4bdbc1/attachment.html>

From jieshic at andrew.cmu.edu  Wed Sep 20 16:39:29 2017
From: jieshic at andrew.cmu.edu (Chen Jieshi)
Date: Wed, 20 Sep 2017 16:39:29 -0400
Subject: Annual Auton Lab Annual Picnic: Saturday October 7th at Schenley Park
Message-ID: <819AB1CB-C358-41CD-8D30-3D09FE5D6E55@andrew.cmu.edu>

Dear Autonians,

To celebrate the 24th birthday of the Auton Lab, we will be organizing a picnic at Schenley Park on Saturday, October 7th.
Please RSVP through the web form below so that we could plan resources properly. Looking forward to seeing all of you!
https://goo.gl/forms/JD2LKVQtiV0xgXoT2 <https://goo.gl/forms/JD2LKVQtiV0xgXoT2>


Also, we are delighted to introduce our lab's first LinkedIn page https://www.linkedin.com/company/cmu-auton-lab <https://www.linkedin.com/company/cmu-auton-lab>. The goal with this LinkedIn page is to provide an easier way to keep connected with our members, alumni and friends all over the world. You are very welcome to add/update your profile and  follow "CMU Auton Lab" on LinkedIn! 



Best,
Jessie


Jieshi (Jessie) Chen
Research Analyst 
Auton Lab, Robotics Institute
Carnegie Mellon University
Newell-Simon Hall, Room 3123
5000 Forbes Ave, Pittsburgh, PA 15213






-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/autonlab-users/attachments/20170920/6eb16d95/attachment.html>

From predragp at andrew.cmu.edu  Thu Sep 21 09:25:16 2017
From: predragp at andrew.cmu.edu (Predrag Punosevac)
Date: Thu, 21 Sep 2017 09:25:16 -0400
Subject: ari down
Message-ID: <20170921132516.lMdWO-fNO%predragp@andrew.cmu.edu>

Dear Autonians,

Ari went down over night due to the users abuse. I will try to power up
the machine as soon as I get to the campus.

Predrag

From predragp at andrew.cmu.edu  Fri Sep 22 11:56:44 2017
From: predragp at andrew.cmu.edu (Predrag Punosevac)
Date: Fri, 22 Sep 2017 11:56:44 -0400
Subject: GPU2 was crashed
Message-ID: <20170922155644.cdry_0WMt%predragp@andrew.cmu.edu>

I am heading to server room to restart GPU2 which apparently was crashed
due to the high load.

Best,
Predrag

From predragp at andrew.cmu.edu  Mon Sep 25 15:36:17 2017
From: predragp at andrew.cmu.edu (Predrag Punosevac)
Date: Mon, 25 Sep 2017 15:36:17 -0400
Subject: Bhyve down
Message-ID: <20170925193617.kEA7DYbTu%predragp@andrew.cmu.edu>

Dear Autonians,

Our BSD Jails virtual host have gone down twice in the past 45 minutes.
Something is wrong with the hardware and I am trying to figure out what.
The following services are affected. 

1. Git/Gogs version control system and repository.
2. Monit, functional monitoring system. I am using right now Observium
for situation awareness.
3. Auton Lab sftp server
4. Jankins continuous integration (we have another instance on Athena
which I believe is in actual use).

I am working on restoring these services.

Best,
Predrag

From predragp at andrew.cmu.edu  Mon Sep 25 18:29:25 2017
From: predragp at andrew.cmu.edu (Predrag Punosevac)
Date: Mon, 25 Sep 2017 18:29:25 -0400
Subject: GPUs status update
Message-ID: <20170925222925.Cx5fId-vL%predragp@andrew.cmu.edu>

I just spent over an hour with NVidia customer support. We went through
installation check list. I focused on GPU7 machine which was completed
last. On one hand it appears to be fully functional. 

/usr/local/cuda-8.0/extras/demo_suite/deviceQuery

shows all the devices. However I still don't see the command 

nvidia-smi in /usr/bin. If any of you have little time I would appreciate
if you log and check if the machine works for you. I have not installed
MATLAB which just had a new release R2017b (our servers should be
running that before the end of the week). However I did install conda
Python installer in

/opt/miniconda2 

which should enable you to add bunch of python specific deep learning
software easier. 

Please let me know if GPU7 appears to be functional.

Predrag

From predragp at andrew.cmu.edu  Mon Sep 25 21:02:47 2017
From: predragp at andrew.cmu.edu (Predrag Punosevac)
Date: Mon, 25 Sep 2017 21:02:47 -0400
Subject: GPU7 now fully functional
Message-ID: <20170926010247.wQTr2Ql3x%predragp@andrew.cmu.edu>

Good news Autonians,

I have figured out how to attach the latest driver to Titan Xp cards.
GPU7 appears to be fully functional. Let me install the latest drivers
and fix the CUDA issues with GPU1-GPU6 before you start asking be about
TensorFlow and Caffe. On the plus side you have Conda installer in 

/opt/miniconda2 

which should make things easier.

Cheers,
Predrag

From predragp at andrew.cmu.edu  Mon Sep 25 21:42:29 2017
From: predragp at andrew.cmu.edu (Predrag Punosevac)
Date: Mon, 25 Sep 2017 21:42:29 -0400
Subject: GPU[5-7] fully functional
Message-ID: <20170926014229.2xivE48jy%predragp@andrew.cmu.edu>

GPU[5-7] are now fully functional. I am not planning to upgrade the
driver on GPU[3-4] since it is unnecessary until the reboot. That leaves
us with broken driver on GPU2 and slow GPU1 (uses different GPU card all
together). I am working right now to fix GPU2. 

Predrag

From predragp at andrew.cmu.edu  Mon Sep 25 22:40:19 2017
From: predragp at andrew.cmu.edu (Predrag Punosevac)
Date: Mon, 25 Sep 2017 22:40:19 -0400
Subject: GPU2 fixed
Message-ID: <20170926024019.2T39_RImU%predragp@andrew.cmu.edu>

root at gpu2$ nvidia-smi
Failed to initialize NVML: Driver/library version mismatch

is now fixed

I upgraded the driver to 

root at gpu2$ cat /proc/driver/nvidia/version
NVRM version: NVIDIA UNIX x86_64 Kernel Module  384.66  Tue Aug  1
16:02:12 PDT 2017
GCC version:  gcc version 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC) 

nvidia-smi appears to work correctly 

root at gpu2$ nvidia-smi
Mon Sep 25 22:39:03 2017       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 384.66                 Driver Version: 384.66
    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile
Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util
Compute M. |
|===============================+======================+======================|
|   0  TITAN X (Pascal)    Off  | 00000000:02:00.0 Off |
N/A |
| 23%   37C    P0    57W / 250W |      0MiB / 12189MiB |      0%
Default |
+-------------------------------+----------------------+----------------------+
|   1  TITAN X (Pascal)    Off  | 00000000:03:00.0 Off |
N/A |
| 23%   36C    P0    56W / 250W |      0MiB / 12189MiB |      0%
Default |
+-------------------------------+----------------------+----------------------+
|   2  TITAN X (Pascal)    Off  | 00000000:82:00.0 Off |
N/A |
| 23%   35C    P0    55W / 250W |      0MiB / 12189MiB |      0%
Default |
+-------------------------------+----------------------+----------------------+
|   3  TITAN X (Pascal)    Off  | 00000000:83:00.0 Off |
N/A |
| 23%   37C    P0    54W / 250W |      0MiB / 12189MiB |      0%
Default |
+-------------------------------+----------------------+----------------------+

     
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU
Memory |
|  GPU       PID  Type  Process name                               Usage
    |
|=============================================================================|
|  No running processes found
    |
+-----------------------------------------------------------------------------+

From predragp at andrew.cmu.edu  Mon Sep 25 22:57:55 2017
From: predragp at andrew.cmu.edu (Predrag Punosevac)
Date: Mon, 25 Sep 2017 22:57:55 -0400
Subject: GPU1 fully functional
Message-ID: <20170926025755.AlZxATw1_%predragp@andrew.cmu.edu>

It appears that GPU1 is fully functional. I see lots of people on the
machine and nvidia-smi is really responsive. Unless otherwise reported
I am considering machines GPU[1-7] fully functional now. Drivers on
GPU[3-4] will be upgraded once the machines are rebooted in the future.

Conda for Python 2.7.13 is also installed on GPU[1-7]. I will be adding
it on CPU computing nodes during MATLAB upgrade. I do expect at this
time that people will start bothering me about TensorFlow and Caffe. I
will discuss tomorrow with Simon what would be the fastest way to have
TensorFlow and Caffe fully functional on all GPU nodes.

Best,
Predrag

From mdeartea at andrew.cmu.edu  Tue Sep 26 11:06:28 2017
From: mdeartea at andrew.cmu.edu (Maria De Arteaga Gonzalez)
Date: Tue, 26 Sep 2017 15:06:28 +0000
Subject: Fw: Second Paper Presentation - Maria DeArteaga - Friday, September
 29 at 10:30 - Room 1204
In-Reply-To: <1a3deb521e0249f6958a4f48ffb9ba97@PGH-MSGMLT-02.andrew.ad.cmu.edu>
References: <1a3deb521e0249f6958a4f48ffb9ba97@PGH-MSGMLT-02.andrew.ad.cmu.edu>
Message-ID: <1506438386944.57069@andrew.cmu.edu>

Hi Autonians,

?

On Friday I will be presenting my qualifier work at Heinz College, you are all invited to attend. Best,


Maria


Mar?a De Arteaga
PhD Student in Machine Learning and Public Policy
Carnegie Mellon University
________________________________
From: Heinz-phd <heinz-phd-bounces+mdeartea=andrew.cmu.edu at lists.andrew.cmu.edu> on behalf of Michelle Wirtz <mwirtz at andrew.cmu.edu>
Sent: Friday, September 22, 2017 3:49 PM
To: heinz-faculty at lists.andrew.cmu.edu; Heinz-phd at lists.andrew.cmu.edu; clermontg at upmc.edu
Subject: Second Paper Presentation - Maria DeArteaga - Friday, September 29 at 10:30 - Room 1204

All,
Please join us Friday, September 29, 2017 in Hamburg Hall Room 1204 at 10:30 when Maria DeArteaga will be presenting her second paper.
Date and time: Friday, September 29, 10:30 in Hamburg Hall 1204.

Committee:  Artur Dubrawski (Chair), Gilles Clermont (UPMC), Alexandra Chouldechova

Title: Predicting Neurological Recovery with Canonical Autocorrelation Embeddings

Abstract:

In this work we present Canonical Autocorrelation Embeddings, a method for embedding sets of data points onto a space in which they are characterized in terms of their latent complex correlation structures, and where a distance metric enables the comparison of such structures. This methodology is particularly fitting to tasks where each individual or object of study has a batch of data points associated to it, as in for instance patients for whom several vital signs or other health related parameters are recorded over time.

We apply this new methodology to characterize patterns of brain activity of comatose survivors of cardiac arrest, aiming to predict whether they would have a positive neurological recovery. Clinicians routinely face the ethically and emotionally charged decision of whether to continue life support for such patients or not. Both scenarios have potentially grave implications on patients and their close ones, so regardless of whether they believe they have enough information, clinicians are often forced to make a prediction. Our results show that we can identify with high confidence a substantial number of patients who are likely to have a good neurological outcome. Providing this information to support clinical decisions could motivate the continuation of life-sustaining therapies for patients whose data suggest it to be the right choice.

Paper:  Attached
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/autonlab-users/attachments/20170926/6998a000/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: DeArteaga_SecondPaper.pdf
Type: application/pdf
Size: 1128730 bytes
Desc: DeArteaga_SecondPaper.pdf
URL: <http://mailman.srv.cs.cmu.edu/pipermail/autonlab-users/attachments/20170926/6998a000/attachment-0001.pdf>

From chiragn at cs.cmu.edu  Tue Sep 26 19:48:33 2017
From: chiragn at cs.cmu.edu (Chirag Nagpal)
Date: Tue, 26 Sep 2017 19:48:33 -0400
Subject: ipython sqlite errors
Message-ID: <CAKH1gVp1xQ9Hhj2grhPiu6w0cR-5icaPin0OHbOyogxxJ9o+rg@mail.gmail.com>

So today ipython notebook randomly stopped working for me on the nodes.

After a bit of troubleshooting, I discovered that ipython/jupyter uses
sqlite for storing notebook histories and sqlite over the NFS is not very
efficient, especially if your trying concurrent access to a single notebook
from multiple nodes.

Incase you are facing such issues a quick workaround is

$rm -rf ~/.local/share/jupyter/

or removing the file nbsignatures.db from wherever it is on your home.

I thought ill post this in public interest.



-- 

*Chirag Nagpal* Graduate Student, Language Technologies Institute
School of Computer Science
Carnegie Mellon University
cs.cmu.edu/~chiragn <http://cs.cmu.edu/%7Echiragn>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/autonlab-users/attachments/20170926/00241e13/attachment.html>

From predragp at andrew.cmu.edu  Tue Sep 26 20:50:54 2017
From: predragp at andrew.cmu.edu (Predrag Punosevac)
Date: Tue, 26 Sep 2017 20:50:54 -0400
Subject: cuDNN
Message-ID: <20170927005054.GR7Irqxyt%predragp@andrew.cmu.edu>

Dear Autonians,

I got an e-mail off the list regarding cuDNN. For the record I am not a
NVidia developer (not even a user of any of their products) so I can't
download the library

https://developer.nvidia.com/cudnn

However I can share with you a link to a blog which explains
installation of many "deep learning" tools including cuDNN to our
computing nodes

http://kehang.github.io/tools/2017/03/31/install-CUDA-cuDNN-on-Red-Hat/

As long as you use your scratch directory as a target you should be
golden. 

Cheers,
Predrag

From chiragn at cs.cmu.edu  Tue Sep 26 22:03:01 2017
From: chiragn at cs.cmu.edu (Chirag Nagpal)
Date: Tue, 26 Sep 2017 22:03:01 -0400
Subject: Fwd: ipython sqlite errors
In-Reply-To: <CAKH1gVp1xQ9Hhj2grhPiu6w0cR-5icaPin0OHbOyogxxJ9o+rg@mail.gmail.com>
References: <CAKH1gVp1xQ9Hhj2grhPiu6w0cR-5icaPin0OHbOyogxxJ9o+rg@mail.gmail.com>
Message-ID: <CAKH1gVr8JxusUHzYYESoCH+AALjTfY07s3A4Ds8jdRWQ25g-4g@mail.gmail.com>

So I digged in a little more and it seemed to me initially the problem is
in ipython itself rather than ipython[notebook]/jupyter.

and indeed, running just ipython stuck on the nodes as well. I fixed that
by adding the line

export IPYTHONDIR=/home/scratch/<user>/.ipython

to ~/.bash_profile

This forces ipython to use the scratch directory for  storing its sqlite
histories rather than the NFS and fixes ipython.

<Predrag: if you see this, request if you could add this environment
variable to the global profile>

I was reasonably sure that this would fix the notebook too, but i haven't
still been able to fix the notebooks. Something more sinister is going on
with the notebooks.

I am on it.

Chirag



---------- Forwarded message ----------
From: Chirag Nagpal <chiragn at cs.cmu.edu>
Date: Tue, Sep 26, 2017 at 7:48 PM
Subject: ipython sqlite errors
To: users at autonlab.org


So today ipython notebook randomly stopped working for me on the nodes.

After a bit of troubleshooting, I discovered that ipython/jupyter uses
sqlite for storing notebook histories and sqlite over the NFS is not very
efficient, especially if your trying concurrent access to a single notebook
from multiple nodes.

Incase you are facing such issues a quick workaround is

$rm -rf ~/.local/share/jupyter/

or removing the file nbsignatures.db from wherever it is on your home.

I thought ill post this in public interest.



-- 

*Chirag Nagpal* Graduate Student, Language Technologies Institute
School of Computer Science
Carnegie Mellon University
cs.cmu.edu/~chiragn <http://cs.cmu.edu/%7Echiragn>



-- 

*Chirag Nagpal* Graduate Student, Language Technologies Institute
School of Computer Science
Carnegie Mellon University
cs.cmu.edu/~chiragn
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/autonlab-users/attachments/20170926/69b51a94/attachment.html>

From predragp at andrew.cmu.edu  Tue Sep 26 23:32:41 2017
From: predragp at andrew.cmu.edu (Predrag Punosevac)
Date: Tue, 26 Sep 2017 23:32:41 -0400
Subject: MATLAB R2017b on lou1
Message-ID: <20170927033241.niyqEszae%predragp@andrew.cmu.edu>

Dear Autonians,

I just installed MATLAB R2017b on lou1 as a proof of concept. Unlike
last couple of years, I had lot of troubles activating license. If you
are heavy MATLAB user please log into lou1 to make sure it works for
you. I am already ready to install MATLAB to neill1 and ari but I want
to talk to MathWorks customer service first. The upgrade should be
completed in next couple of days.

Best,
Predrag

From ffalck at andrew.cmu.edu  Wed Sep 27 14:56:11 2017
From: ffalck at andrew.cmu.edu (Fabian Falck)
Date: Wed, 27 Sep 2017 18:56:11 +0000
Subject: Tensorflow/Theano related BLAS/ATLAS issue
Message-ID: <1816795F-8116-40E0-B888-B3D4CCDCA1C2@andrew.cmu.edu>

Dear Autonians,

This is a shout out to all people using Tensorflow/Theano (either as a backend or directly). Since very recently, I receive the following warnings:

"WARNING (theano.tensor.blas): We did not found a dynamic library into the library_dir of the library we use for blas. If you use ATLAS, make sure to compile it with dynamics library."

The consequence is that Tensorflow/Theano computations are extremely slow or probably not computing at all. Is anyone facing or was in the past faced with the same issue?

Many thanks,
Fabian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/autonlab-users/attachments/20170927/4b2f3b78/attachment.html>

From predragp at andrew.cmu.edu  Thu Sep 28 00:54:55 2017
From: predragp at andrew.cmu.edu (Predrag Punosevac)
Date: Thu, 28 Sep 2017 00:54:55 -0400
Subject: MATLAB R2017b installed
Message-ID: <20170928045455.MYTCiBOzM%predragp@andrew.cmu.edu>

Dear Autonians,

The licensing issues with MATLAB R2017b have being resolved now. The
following servers run the latest and the greatest version:

neill1, lou1, ari, gpu[5-7]

I am happy to report that R2017b works like a charm with the latest
NVidia driver on the latest Titan Xp cards. 

At the moment I am pushing R2017b tarball to neill[2-4]. I anticipate
that it will take few days due to the size of tarball to update all
computing nodes. Once the servers are done I will update CDC virtual
server. Desktops and your personal laptops will be updated the last
unless you have urgent need in which case you should send me an e-mail.

This is the last e-mail regarding MATLAB.

Best,
Predrag

From jieshic at andrew.cmu.edu  Thu Sep 28 11:49:32 2017
From: jieshic at andrew.cmu.edu (Jieshi Chen)
Date: Thu, 28 Sep 2017 15:49:32 +0000
Subject: Annual Auton Lab Annual Picnic: Saturday October 7th at Schenley
 Park
In-Reply-To: <819AB1CB-C358-41CD-8D30-3D09FE5D6E55@andrew.cmu.edu>
References: <819AB1CB-C358-41CD-8D30-3D09FE5D6E55@andrew.cmu.edu>
Message-ID: <1506613772940.57521@andrew.cmu.edu>

Hi Autonians,


Just a reminder if you have not RSVP for the picnic yet. :)

https://goo.gl/forms/JD2LKVQtiV0xgXoT2



Thanks,

Jessie

________________________________
From: Autonlab-users <autonlab-users-bounces at autonlab.org> on behalf of Chen Jieshi <jieshic at andrew.cmu.edu>
Sent: Wednesday, September 20, 2017 4:39 PM
To: users at autonlab.org
Subject: Annual Auton Lab Annual Picnic: Saturday October 7th at Schenley Park

Dear Autonians,

To celebrate the 24th birthday of the Auton Lab, we will be organizing a picnic at Schenley Park on Saturday, October 7th.
Please RSVP through the web form below so that we could plan resources properly. Looking forward to seeing all of you!
https://goo.gl/forms/JD2LKVQtiV0xgXoT2


Also, we are delighted to introduce our lab's first LinkedIn page https://www.linkedin.com/company/cmu-auton-lab. The goal with this LinkedIn page is to provide an easier way to keep connected with our members, alumni and friends all over the world. You are very welcome to add/update your profile and  follow "CMU Auton Lab" on LinkedIn!



Best,
Jessie


Jieshi (Jessie) Chen
Research Analyst
Auton Lab, Robotics Institute
Carnegie Mellon University
Newell-Simon Hall, Room 3123
5000 Forbes Ave, Pittsburgh, PA 15213






-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/autonlab-users/attachments/20170928/99e7cbc7/attachment.html>

From sheath at andrew.cmu.edu  Thu Sep 28 16:07:36 2017
From: sheath at andrew.cmu.edu (Simon Heath)
Date: Thu, 28 Sep 2017 16:07:36 -0400
Subject: GPU machine cuDNN upgrade -- GPU1[3-7] node reboot MONDAY
Message-ID: <CAGeWgZSYfBLt6LFrQyZdNfGtVrpFufpqxN4p_gO5nbEUh2d9qw@mail.gmail.com>

Hi all,

We've upgraded the cuDNN library to version 6.0 on all the GPU nodes.
cuDNN 5.0 is also still installed.  However for the install to complete we
really need to reboot the GPU nodes.  Unless anyone objects we will do the
reboot Monday at 3 pm; hopefully this gives everyone to finish up any
long-running jobs.  GPU2 had no users on it and has already been rebooted,
so if there's anything you super need done on Monday, do it on GPU2.

If you do any neural network stuff with Tensorflow or Torch, this should
bring significant performance improvements; just make sure your installs
are up to date.

Regards,
Simon

-- 
Simon Heath, Research Programmer and Analyst
Robotics Institute - Auton Lab
Carnegie Mellon University
sheath at andrew.cmu.edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/autonlab-users/attachments/20170928/88c54490/attachment.html>

From fabian.falck at web.de  Fri Sep 29 08:50:15 2017
From: fabian.falck at web.de (Fabian Falck)
Date: Fri, 29 Sep 2017 08:50:15 -0400
Subject: Thank you and goodbye note ... and free BAGEL note
Message-ID: <C8CA2329-8B1B-45F0-A46F-3D62D0429C56@web.de>

Dear Autonians, 

My time in this awesome lab has come to an end. I am very thankful for this truly remarkable experience and am happy to have met all of you! 

As a small farewell gift, you can find some free bagels in room 3128. I heard system admins and GPU-fixing engineers have priority access. And before people complain: Yes, I would also prefer some nice German pretzels with butter, but unfortunately, I could not find proper instances in a 5000 km radius.

All the best,
Fabian

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/autonlab-users/attachments/20170929/e4ef06d2/attachment.html>

From predragp at andrew.cmu.edu  Fri Sep 29 20:09:00 2017
From: predragp at andrew.cmu.edu (Predrag Punosevac)
Date: Fri, 29 Sep 2017 20:09:00 -0400
Subject: Git/Gogs update
Message-ID: <20170930000900.PrVin-3x2%predragp@andrew.cmu.edu>

Hi Autonians,

I just want to give you a quick heads up about our Git/Gogs repo. After
3 days of trying to troubleshoot that moody Supermicro server
(bhyve.int.autonlab) where the git was running I had to reach to
Sillicon Mechanics customer support. Long story short it appears that we
are victimized by a tiny electric problem on CPU or motherboard itself.
I will do RMA with them on Monday (server is less than 2 years old and
they are suppose to fix it for us free of charge). In the mean time I
have a plan how to reuse existing resources to resurrect affected
services.

I am not making any promises in the terms of time but fixing this is a
top priority for me and I will be working on this first thing on Monday.


Best,
Predrag