GPUs 11/13/14 nvidia-smi not working

[Predrag Punosevac]

Thanks for the report! I can reproduce the problem. This is typically
due to the upgraded kernel for which nvidia module has to be rebuilt
(If i reboot the machine that will happen automatically). Welcome to
the wonderful world of proprietary software. So this is the notice to
the rest of the group that I will be rebooting non functional GPU
nodes in next hour or two until things are fixed.

Cheers,
Predrag

On Mon, Aug 26, 2019 at 3:47 PM Abhay Gupta <abhayg at andrew.cmu.edu> wrote:
>
> Hi Predrag,
>
> I was trying to access GPUs on servers 11/13/14 got this error while running 'nvidia-smi':
> 'Failed to initialize NVML: Driver/library version mismatch'
>
> I think either the driver or the library needs to be updated for Nvidia drivers on these servers. Can you please have a look into this? Thanks.
>
> --
> Regards,
> Abhay Gupta