GPU2 upgraded

[Predrag Punosevac]

On 2016-10-26 13:25, Dougal Sutherland wrote:
> Yes, that's happening for me too. I assumed it was just me because
> 
> a) it doesn't happen for the auton-local account
> b) it also happens for my anaconda-installed ipython
> 
> but this started happening for me today. Something very weird
> happening there.
> 


I just checked and it hangs for me as well with the regular account but 
it doesn't hang with auton-local account. The difference is that 
auton-local stores info on the local drive while my regular account is 
storing data on NFS shares. Since it is happening across the computing 
nodes it is not a NFS client issue but NFS server issue. I am clueless 
at the point why is this happening. I don't want to reboot file server 
until power outage.

I am going to think about this. I am guessing it would be possible to 
install ipython in your local scratch directory and that one should 
start.

Predrag

> On Wed, Oct 26, 2016 at 6:03 PM Junier Oliva <junieroliva at gmail.com>
> wrote:
> 
>> Not sure if this at all related, but is ipython broken for anyone
>> else? It seems to just hang upon launching it on several auton
>> machines (GPU1, GPU2, LOV4, LOW1).
>> 
>> Thanks,
>> Junier
>> 
>> On Tue, Oct 25, 2016 at 2:14 PM, Dougal Sutherland
>> <dougal at gmail.com> wrote:
>> 
>> The same version of TensorFlow as on gpu3 is now installed on gpu2,
>> along with cudnn; let me know if there are issues.
>> 
>> I didn't do a global install of Caffe on either machine, because
>> Caffe is kind of dumb and doesn't really do global installs. If
>> anyone wants this, talk to me and we can figure out what makes
>> sense.
>> 
>> - Dougal
>> 
>> On Tue, Oct 25, 2016 at 6:09 PM Predrag Punosevac
>> <predragp at imap.srv.cs.cmu.edu> wrote:
>> Dear Autonians,
>> 
>> As scheduled I upgraded GPU2 to 256 GB of RAM and 4 Titan (Pascal)
>> cards
>> per Barnabas. Dugal is currently upgrading TensorFlow and Caffe to
>> the
>> latest and the greatest. Please wait until his e-mail until you hit
>> the
>> machine. MATLAB is broken just like on GPU3 due to cuda-8.0. I am
>> working now on GPU1.
>> 
>> Predrag
>> 
>> P.S. I will escalate MATLAB issue with MathWorks but I don't expect
>> to
>> fixed before R2017a.