CUDA hangs

[Predrag Punosevac]

Matthew Barnes <mbarnes1 at andrew.cmu.edu> wrote:

> Is anyone else having issues with CUDA since this week? Even simple pytorch
> commands hang:
> 

Do you have issues on all 8 GPU servers (GPU 7 is used for special
project) you can access? 

I upgraded driver and CUDA to 9.2 on GPU1. I would not expect pytorch to
work after that without reinstalling. 

GPU3 and GPU4 are reporting 3 GPU cards. That is a bad sign and means
dead hardware. I am planning to reboot it and play with it little bit
before the final diagnosis. 

Predrag


> (torch) bash-4.2$ python
> Python 2.7.5 (default, Jul  3 2018, 19:30:05)
> [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import torch
> x>>> x = torch.zeros(4)
> >>> x.cuda()
> 
> 
> nvidia-smi works, and torch.cuda.is_available() returns True.