From predragp at andrew.cmu.edu  Fri Nov  1 21:48:57 2019
From: predragp at andrew.cmu.edu (Predrag Punosevac)
Date: Fri, 1 Nov 2019 21:48:57 -0400
Subject: Zombie processes running on GPU9 and GPU4
In-Reply-To: <CAM7MTAEHZ64=49R38qs4F6UvdoiVmHNb4DReKCqpj_Xe1NaUEA@mail.gmail.com>
References: <CAM7MTAEHZ64=49R38qs4F6UvdoiVmHNb4DReKCqpj_Xe1NaUEA@mail.gmail.com>
Message-ID: <CAEFFt-HYxuNDmLy2Z4pwo=ZemnCyuKCAjem_nN=cin=onGiRCg@mail.gmail.com>

Hi Tanmay,

The only sure thing would be to reboot the machines. It is against my
religion to do that on Friday evening. Rebooting will likely brake
NVidia driver as the machines will boot into Springdale 7.7 instead of
current 7.6. I will need to get confirmation that these machines are
not usable  at least from a few other people who were recently using
them before I reschedule reboot.

Best,
Predrag

On Fri, Nov 1, 2019 at 9:41 PM Tanmay Agarwal <tanmaya at andrew.cmu.edu> wrote:
>
> Hi Predrag,
>
> Hope you are doing well. I wanted to bring to your attention that I have noticed some kind of a zombie process that are running on GPU9 (0) and GPU4 (0) which are not even visible in nvidia-smi and are consuming the entire gpu memory. Can you help us reset those gpus?
>
> Also, i have some runs running on these GPU machines, so "in case" you need to reboot the machine, kindly let me know before you have to reset the machines.
>
> P.S Attaching a screenshot of one of those GPU machines.
>
> Thanking you,
>
> Best Regards,
>
> Tanmay Agarwal | MSR Graduate Student
> Robotics Institute @ CMU
> mailto: tanmaya at andrew.cmu.edu


From awd at cs.cmu.edu  Mon Nov  4 21:52:21 2019
From: awd at cs.cmu.edu (Artur Dubrawski)
Date: Mon, 4 Nov 2019 21:52:21 -0500
Subject: Maria De Arteaga shares opinions in New Scientist
Message-ID: <CAJvAoyv8qFDThyktq5LSXvbAbWR5WWQxqo81-jR5AXdfd2GO1g@mail.gmail.com>

See the attachment :)

Cheers,
Artur
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/autonlab-users/attachments/20191104/7d8bd9fe/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: asking the earth New Scientist PDF (1).pdf
Type: application/pdf
Size: 792471 bytes
Desc: not available
URL: <http://mailman.srv.cs.cmu.edu/pipermail/autonlab-users/attachments/20191104/7d8bd9fe/attachment-0001.pdf>

From mdeartea at andrew.cmu.edu  Tue Nov  5 07:22:15 2019
From: mdeartea at andrew.cmu.edu (Maria De Arteaga Gonzalez)
Date: Tue, 5 Nov 2019 07:22:15 -0500
Subject: Maria De Arteaga shares opinions in New Scientist
In-Reply-To: <CAJvAoyv8qFDThyktq5LSXvbAbWR5WWQxqo81-jR5AXdfd2GO1g@mail.gmail.com>
References: <CAJvAoyv8qFDThyktq5LSXvbAbWR5WWQxqo81-jR5AXdfd2GO1g@mail.gmail.com>
Message-ID: <CACsvUsaSzVcHMC23TQSNMo6B5dvhYAXJZS_3hHAzX+XyKK=+tg@mail.gmail.com>

Thanks for sharing, Artur!

Hi all, please make sure you keep the PDF Artur shared internal. The link
to the online article (which includes a paywall, sadly) is here:
https://www.newscientist.com/article/mg24432542-700-all-seeing-eyes-the-epic-plan-to-track-almost-everything-on-earth/

Have a great day!

Mar?a



On Mon, Nov 4, 2019, 21:57 Artur Dubrawski <awd at cs.cmu.edu> wrote:

> See the attachment :)
>
> Cheers,
> Artur
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/autonlab-users/attachments/20191105/3fe25092/attachment.html>

From ngisolfi at cs.cmu.edu  Thu Nov  7 11:09:10 2019
From: ngisolfi at cs.cmu.edu (Nick Gisolfi)
Date: Thu, 7 Nov 2019 11:09:10 -0500
Subject: [Lunch] Today, 12pm @ NSH 4513
Message-ID: <9722CB5F-4996-46A7-9680-712225973DEB@cs.cmu.edu>

Hi Everyone,

Lab lunch is in NSH 4513 at noon today.

Bring your meal and meet us there!

- Nick

From awd at cs.cmu.edu  Thu Nov  7 17:14:29 2019
From: awd at cs.cmu.edu (Artur Dubrawski)
Date: Thu, 7 Nov 2019 17:14:29 -0500
Subject: Ina Fiterau: Triumphant return!
Message-ID: <CAJvAoyuOf8R+TjG2P5y7yorkNPTZD05BU7bjZS2BS0S17DNweA@mail.gmail.com>

Guess what? Our own Ina is coming back to town this Monday, to give a talk
at the RI:

https://www.ri.cmu.edu/event/hybrid-methods-for-the-integration-of-heterogeneous-multimodal-biomedical-data/


Conveniently, the talk is scheduled for our default Auton brainstorming
slot, so everyone should be able to join it as it is now officially aslo
the Auton Lab seminar.

Cheers,
Artur
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/autonlab-users/attachments/20191107/293b2562/attachment.html>

From predragp at andrew.cmu.edu  Sat Nov  9 07:15:16 2019
From: predragp at andrew.cmu.edu (Predrag Punosevac)
Date: Sat, 09 Nov 2019 07:15:16 -0500
Subject: bash is down
Message-ID: <e0cf7dd1-bed2-41e9-ab84-867503c4fd7a@email.android.com>

An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/autonlab-users/attachments/20191109/e74b931c/attachment.html>

From predragp at andrew.cmu.edu  Mon Nov 11 14:17:06 2019
From: predragp at andrew.cmu.edu (Predrag Punosevac)
Date: Mon, 11 Nov 2019 14:17:06 -0500
Subject: bash updates
Message-ID: <CAEFFt-GG8Lfdm8S=CyKf49yUFv-_ELvWy9_1UoSDHbwdSAe88w@mail.gmail.com>

It appears that the bash network after upgrade to 7.7 is unstable.
Unfortunately there is no way back to 7.6 as there are already three
different kernels past 7.7 release.

I disabled network manager which was pulled as automatic dependency.
If that doesn't fix the problem we are dealing with a kernel/network
stack bug which was reintroduced with 7.7. If Springdale guys release
RPMs for 8.1 release bash will be the first one to be upgraded.

Cheers,
Predrag

From awd at cs.cmu.edu  Tue Nov 12 15:03:09 2019
From: awd at cs.cmu.edu (Artur Dubrawski)
Date: Tue, 12 Nov 2019 15:03:09 -0500
Subject: Fwd: Thesis Proposal - Nov. 14, 2019 - Yichong Xu - Learning and
 Decision Making from Diverse Forms of Information
In-Reply-To: <f613c2de-8bcb-3f7c-3fdb-7ffdd4c993a6@cmu.edu>
References: <f613c2de-8bcb-3f7c-3fdb-7ffdd4c993a6@cmu.edu>
Message-ID: <CAJvAoyt5H6K6DGGO_YRg3E2gn5756NZKGDk_VcStSMGA5W459A@mail.gmail.com>

Dear Autonians,

Please come and enjoy an exceptional opportunity to learn about learning
from diverse forms of information from Yichong, who will be presenting his
thesis proposal this Thursday at 1pm in GHC 4405.

Cheers
Artur


---------- Forwarded message ---------
From: Diane Stidle <stidle at andrew.cmu.edu>
Date: Fri, Nov 1, 2019 at 3:12 PM
Subject: Thesis Proposal - Nov. 14, 2019 - Yichong Xu - Learning and
Decision Making from Diverse Forms of Information
To: ml-seminar at cs.cmu.edu <ML-SEMINAR at cs.cmu.edu>, <jcl at microsoft.com>


*Thesis Proposal*

Date: November 14, 2019
Time: 1:00pm (EST)
Place: GHC 4405
Speaker: Yichong Xu

*Title: Learning and Decision Making from Diverse Forms of Information*

Abstract:
Classical machine learning posits that data are independently and
identically distributed, in a single format usually the same as test data.
In modern applications however, additional information in other formats
might be available freely or at a lower cost. For example, in data
crowdsourcing we can collect preferences over the data points instead of
directly asking the labels of a single data point at a lower cost. In
natural language understanding problems, we might have limited amount of
data in the target domain, but can use a large amount of general domain
data for free.

The main topic of this thesis is to study how to efficiently incorporate
these diverse forms of information into the learning and decision making
process. We study two representative paradigms in this thesis. Firstly, we
study learning and decision making problems with direct labels and
comparisons. Our algorithms can efficiently combine comparisons with direct
labels so that the total learning cost can be greatly reduced. Secondly, we
study multi-task learning problems from multiple domain data, and design
algorithms to transfer the data from a general, abundant domain to the
target domain. We show theoretical guarantees of our algorithms as well as
their statistical minimaxity through information-theoretic limits. On the
practical side, we demonstrate promising experimental results on price
estimation and natural language understanding tasks.

*Thesis Committee:*
Artur Dubrawski (Co-Chair)
Aarti Singh (Co-Chair)
Sivaraman Balakrishnan
John Langford (Microsoft Research)

Link to the draft document:
https://www.dropbox.com/s/6r6qk3d7hkfkl8p/proposal.pdf?dl=0

-- 
Diane Stidle
Graduate Programs Manager
Machine Learning Department
Carnegie Mellon Universitystidle at cmu.edu
412-268-1299
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/autonlab-users/attachments/20191112/7f5bfe2e/attachment.html>

From ngisolfi at cs.cmu.edu  Thu Nov 14 09:59:08 2019
From: ngisolfi at cs.cmu.edu (Nick Gisolfi)
Date: Thu, 14 Nov 2019 09:59:08 -0500
Subject: [Lunch] Today, 12-1pm in NSH 4513
Message-ID: <9FBD3B58-4AF4-4336-94B8-09C04586569A@cs.cmu.edu>

Hi Everyone,

Bring your meal and we?ll see you in NSH 4513 for lab lunch!

- Nick

From boecking at andrew.cmu.edu  Thu Nov 14 11:35:12 2019
From: boecking at andrew.cmu.edu (Benedikt Boecking)
Date: Thu, 14 Nov 2019 11:35:12 -0500
Subject: Server load
Message-ID: <CAB7E167-FB77-4D94-A55B-B6723921B115@andrew.cmu.edu>

Hi all,

We are currently running into problems on several servers due to some users spawning too many threads. On several compute nodes we have several thousand threads running in parallel. 

To be a good lab member please:
1. Use top/htop and nvidia-smi to monitor your resource usage and keep it reasonable (memory, cpu, and gpu usage). 

2.  Check that you aren?t using more threads than intended due to automatic multiprocessing. This happens for example with numpy and spicy linalg functions on the servers.You can control most of this behavior by setting the following variables before running scripts or notebooks, setting them to however many threads you want:

export MKL_NUM_THREADS=1
export NUMEXPR_NUM_THREADS=1
export OMP_NUM_THREADS=1

3. If you have (interactive) session open that you don?t need anymore please close them to free memory (in particular Matlab and jupyter notebooks)



Thanks,
Ben





From awd at cs.cmu.edu  Thu Nov 14 12:23:59 2019
From: awd at cs.cmu.edu (Artur Dubrawski)
Date: Thu, 14 Nov 2019 12:23:59 -0500
Subject: IMPORTANT [Re: Server load]
In-Reply-To: <CAB7E167-FB77-4D94-A55B-B6723921B115@andrew.cmu.edu>
References: <CAB7E167-FB77-4D94-A55B-B6723921B115@andrew.cmu.edu>
Message-ID: <CAJvAoyu3u-EOzi8_p9tkHAV6YXZJv36HA_n1z8MoBs9e2BtARQ@mail.gmail.com>

Team,

I can't stress enough how important is being good citizens in how we
utilize our shared computing resources.

The key realization is that their capacity is not infinite, and so we all
need to play along nicely.

Please for now follow the recommendations provided by Ben below.

In the meantime, Predrag is preparing formal guidelines, to be published
shortly.

Barnabas, Jeff and I will convene to consider implementing more systematic
measures of resource consumption control, if it turns out to be necessary
due
to persistence of the problems we are facing over the past few days.

Cheers,
Artur


On Thu, Nov 14, 2019 at 11:36 AM Benedikt Boecking <boecking at andrew.cmu.edu>
wrote:

> Hi all,
>
> We are currently running into problems on several servers due to some
> users spawning too many threads. On several compute nodes we have several
> thousand threads running in parallel.
>
> To be a good lab member please:
> 1. Use top/htop and nvidia-smi to monitor your resource usage and keep it
> reasonable (memory, cpu, and gpu usage).
>
> 2.  Check that you aren?t using more threads than intended due to
> automatic multiprocessing. This happens for example with numpy and spicy
> linalg functions on the servers.You can control most of this behavior by
> setting the following variables before running scripts or notebooks,
> setting them to however many threads you want:
>
> export MKL_NUM_THREADS=1
> export NUMEXPR_NUM_THREADS=1
> export OMP_NUM_THREADS=1
>
> 3. If you have (interactive) session open that you don?t need anymore
> please close them to free memory (in particular Matlab and jupyter
> notebooks)
>
>
>
> Thanks,
> Ben
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/autonlab-users/attachments/20191114/7fb0b5c4/attachment.html>

From predragp at andrew.cmu.edu  Mon Nov 18 14:50:53 2019
From: predragp at andrew.cmu.edu (Predrag Punosevac)
Date: Mon, 18 Nov 2019 14:50:53 -0500
Subject: Can't connect to lov5
In-Reply-To: <CALR8o8kAL+GtgsNYa7a9Ur9ufzvRW79Gu40Giui3JiDfKA0tWQ@mail.gmail.com>
References: <CALR8o8kAL+GtgsNYa7a9Ur9ufzvRW79Gu40Giui3JiDfKA0tWQ@mail.gmail.com>
Message-ID: <CAEFFt-GGqCL6TFGWSyX6Pb5B=N5vhmNZt_L7uarm8YDE_vTrUw@mail.gmail.com>

The server crashed probably due to a runaway script. I just rebooted
it 30 min ago.

Cheers,
Predrag

P.S. You should be sleeping at 2:46 AM not playing with computers :-)

On Mon, Nov 18, 2019 at 2:46 AM Naji Shajarisales <najis at cmu.edu> wrote:
>
> Hi Predrag,
>
> I am not able to connect to lov5. What's wrong?
>
> Best,
> Naji

From predragp at andrew.cmu.edu  Wed Nov 20 12:19:08 2019
From: predragp at andrew.cmu.edu (Predrag Punosevac)
Date: Wed, 20 Nov 2019 12:19:08 -0500
Subject: ari, athena, foxconn
Message-ID: <CAEFFt-Gxzn9-Q_kzn_EWX_CTGOaUEEKhmK0vpu77p_sDQf5EcQ@mail.gmail.com>

I see people wrestling for the resources on lov1-6, low1, and gpu2-14
machines. However ari, athena, foxconn are idling.

Predrag

From ngisolfi at cs.cmu.edu  Thu Nov 21 09:48:15 2019
From: ngisolfi at cs.cmu.edu (Nick Gisolfi)
Date: Thu, 21 Nov 2019 09:48:15 -0500
Subject: [Lunch] Today @noon in NSH 4513
Message-ID: <054AF905-3AF2-4EC9-833D-E9D0FDEA5EB5@cs.cmu.edu>

Hi Everyone,

We?ll meet for lab lunch at noon in NSH 4513.  Bring your meal and we?ll see you there!

- Nick

P.S. NO LUNCH NEXT WEEK (That?s Thanksgiving Day!)

From awd at cs.cmu.edu  Mon Nov 25 05:36:33 2019
From: awd at cs.cmu.edu (Artur Dubrawski)
Date: Mon, 25 Nov 2019 05:36:33 -0500
Subject: If you've curious about what Andrew has been up to...
Message-ID: <CAJvAoyvmXAwvrUe40PjVU1OvMQsvW==x7g79Z6i9POw5oDqprQ@mail.gmail.com>

...then read this:

https://www.bbc.com/news/technology-50506431

Cheers
Artur
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/autonlab-users/attachments/20191125/9149dea3/attachment.html>

From awd at cs.cmu.edu  Tue Nov 26 15:39:16 2019
From: awd at cs.cmu.edu (Artur Dubrawski)
Date: Tue, 26 Nov 2019 15:39:16 -0500
Subject: NeurIPS poster practice lunch: Thursday Dec 5th @noon in NSH 4513
In-Reply-To: <054AF905-3AF2-4EC9-833D-E9D0FDEA5EB5@cs.cmu.edu>
References: <054AF905-3AF2-4EC9-833D-E9D0FDEA5EB5@cs.cmu.edu>
Message-ID: <CAJvAoyuXOHd0KFyhr5h55JK_+P3anW2ZW74CWY7-9m=k=CNX2g@mail.gmail.com>

Team: we are going to consume the usual team lunch slot next week to give
all the Lab members who are to present their work at NeurIPS or NeurIPS
Workshops, a chance to run their posters by us and to practice presenting
them.

Rules as usual: bring your food if you'd like to eat, but most importantly
bring your critical eyes and ears, ask good questions, help improve how our
work will be presented at the conference.

Thanks
Artur

PS Nick, do you think we could extend our time slot at NSH 4513 to from
>=15 minutes before noon to enable poster setup, till 30 or so minutes past
1pm? Or alternatively, get another perhaps a larger room instead?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/autonlab-users/attachments/20191126/5787aefa/attachment.html>