cuda problem

[Ifigeneia Apostolopoulou]

Hi Kyle,
Thanks a lot for your reply!

I also had this issue and I solved it as you did. However, this seems to be
another issue:
I currently can't see CUPTI in usr/local/cuda/extras/CUPTI (or anywhere in
gpu1 to set it to my path) which causes the issue.
I am also attaching the screenshot with the working (gpu3) and not-working
(gpu1) case. In gpu1, gpu2, gpu13, it seems that the directory cuda (and
all its content) has been moved (and I can't find it in any other
directory).





On Tue, Aug 18, 2020 at 9:32 AM Kyle Miller <mille856 at andrew.cmu.edu> wrote:

> Ifi,
>    I recently had difficulty on GPU13, having not used it in a long while.
> For me, the issue was that miniconda had moved. I added
> /opt/miniconda-py38/bin to my path and rebuilt my environment (not sure if
> that was necessary). Then it worked.
> -Kyle
>
> On Tue, Aug 18, 2020 at 2:14 AM Predrag Punosevac <predragp at andrew.cmu.edu>
> wrote:
>
>> Ifigeneia Apostolopoulou <iapostol at andrew.cmu.edu> wrote:
>>
>> > Hi Predrag,
>> >
>> > I hope that this (weird) summer is going well!
>> >
>> > I noticed a change in servers gpu1, gpu2, gpu13, gpu14.
>> > Specifically, I no longer can find
>>
>> I have not touch those servers in a very long time. I am CC-ing users
>> mailing list. My brain is shutting down at this late hour. Maybe
>> somebody could be of more help tomorrow morning.
>>
>> >
>> > /usr/local/cuda/extras/CUPTI
>> >
>>
>> I believe you.
>>
>>
>> > which results in the error when I'm building my tensorflow models.
>> >
>> >  Not found: ./bin/ptxas not found. Relying on driver to perform ptx
>> > compilation. This message will be only logged once.
>> >
>> > Any ideas, how could I solve this issue? Would it be possible to restore
>> > the cuda directory?
>> >
>> > Also, I currently do not have access to gpu21.
>>
>> It is fixed now. I just restarted sssd daemon. Please don't use gpu20
>> and gpu21 unless you are training 3D neuronal networks for which you
>> need lot of GPU memory.
>>
>> Predrag
>>
>>
>> >
>> > Thanks a lot in advance!
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.srv.cs.cmu.edu/pipermail/autonlab-users/attachments/20200818/cc420fe3/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: gpu1.png
Type: image/png
Size: 197088 bytes
Desc: not available
URL: <http://mailman.srv.cs.cmu.edu/pipermail/autonlab-users/attachments/20200818/cc420fe3/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: gpu3.png
Type: image/png
Size: 157090 bytes
Desc: not available
URL: <http://mailman.srv.cs.cmu.edu/pipermail/autonlab-users/attachments/20200818/cc420fe3/attachment-0003.png>