From ngisolfi at andrew.cmu.edu Mon Mar 4 08:48:00 2019 From: ngisolfi at andrew.cmu.edu (Nicholas Gisolfi) Date: Mon, 4 Mar 2019 08:48:00 -0500 Subject: [Cookies!] Today in NSH 3111 Message-ID: Hi Everyone, In an effort to build a stronger sense of community in our lab, we want to invite each of you to stop by NSH 3111 any time (before 5pm) today to grab a cookie and chat. We can talk shop, or not, it's up to you. Perhaps you have a new problem where it has been difficult to gain traction, or you want to share how awesome your weekend was. The point is, we want to say hi and talk to you! If we need an excuse to celebrate, Costco swapped out their oatmeal raisin cookies for triple chocolate cookies (hooray! much better!!) - Nick, Ben, and Sibi P.S. for those of you outside Pittsburgh, just pretend we sent cookies to you via your web browser :P -------------- next part -------------- An HTML attachment was scrubbed... URL: From predragp at andrew.cmu.edu Mon Mar 4 10:22:32 2019 From: predragp at andrew.cmu.edu (Predrag Punosevac) Date: Mon, 04 Mar 2019 10:22:32 -0500 Subject: pandas and sklearn for python3? In-Reply-To: References: Message-ID: <20190304152232._q_Bw2tXE%predragp@andrew.cmu.edu> Ifigeneia Apostolopoulou wrote: > Hi Predrag, > > Is it possible to install these modules for python3? > I can only find them for python 2.7 > > thanks! The core of Python 3.6 is installed in /opt/rh/rh-python36. You also have miniconda3 in /opt/miniconda3 which will enable you to install the latest packages for python 3.7.2. Python 3 comes with built in virtual environment which should be used to build your workspace in the scratch directory. Cheers, Predrag From ngisolfi at cs.cmu.edu Tue Mar 5 14:45:25 2019 From: ngisolfi at cs.cmu.edu (Nick Gisolfi) Date: Tue, 5 Mar 2019 14:45:25 -0500 Subject: [Community] Weekly Auton Lab Lunch, Fridays in NSH 3104 @ 11:45am-1:30pm EST Message-ID: Hi Everyone, Due to popular demand, we'll be extending our community building activities to include a weekly Auton Lab lunch! We have NSH 3104 reserved from 11:45am to 1:30pm EST for each Friday for the rest of the semester (March 8th will be our first event). The room is small, but we can reserve a larger space as participation grows from week to week. This first point may seem counterintuitive...*lunch will not be provided*. Instead, the purpose of this weekly lab lunch is to give us the opportunity to connect, whether you pack or buy on/off campus. We all eat, so let's take our lunch break together. For those of you off-campus, we'll set up a Google Meeting so you can connect and chat with us over lunch. Time zone permitting, feel free to eat your meal while connected :) While the timing of this event will be consistent week-to-week for the spring semester, I'll still send out a reminder email before each lunch with a url to connect our colleagues outside Pittsburgh. For the first few weeks, we will consider this experimental, and we are open to any and all thoughts/suggestions for how to make it better. Shoot me an email or let us know what you think over lunch this Friday! - Nick -------------- next part -------------- An HTML attachment was scrubbed... URL: From ngisolfi at andrew.cmu.edu Fri Mar 8 11:24:50 2019 From: ngisolfi at andrew.cmu.edu (Nicholas Gisolfi) Date: Fri, 8 Mar 2019 11:24:50 -0500 Subject: [Community] Weekly Auton Lab Lunch, Fridays in NSH 3104 @ 11:45am-1:30pm EST In-Reply-To: References: Message-ID: Hi Everyone, We'll get started at 11:45 today for our first of many Auton Lab lunches! We'll be meeting in NSH 3104, and have the space until 1:30pm. Stop by or connect any time in between. Remember, *no lunch will be provided*, so this is BYO...L (?). There's your neologism for the day! To join via web browser (I don't believe there is Safari support): https://meet.google.com/oho-jjok-ack Otherwise, to join by phone, dial +1 515-518-6321 and enter this PIN: 643 666 823# I'm looking forward to this opportunity to catch up with many of you! Even if you can only connect for a second just to say hello, at least you can let us know what you think about using Google Meetings for future events. See you soon! - Nick On Tue, Mar 5, 2019 at 2:46 PM Nick Gisolfi wrote: > Hi Everyone, > > Due to popular demand, we'll be extending our community building > activities to include a weekly Auton Lab lunch! We have NSH 3104 reserved > from 11:45am to 1:30pm EST for each Friday for the rest of the semester > (March 8th will be our first event). The room is small, but we can reserve > a larger space as participation grows from week to week. > > This first point may seem counterintuitive...*lunch will not be provided*. > Instead, the purpose of this weekly lab lunch is to give us the opportunity > to connect, whether you pack or buy on/off campus. We all eat, so let's > take our lunch break together. > > For those of you off-campus, we'll set up a Google Meeting so you can > connect and chat with us over lunch. Time zone permitting, feel free to > eat your meal while connected :) > > While the timing of this event will be consistent week-to-week for the > spring semester, I'll still send out a reminder email before each lunch > with a url to connect our colleagues outside Pittsburgh. > > For the first few weeks, we will consider this experimental, and we are > open to any and all thoughts/suggestions for how to make it better. Shoot > me an email or let us know what you think over lunch this Friday! > > - Nick > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ngisolfi at andrew.cmu.edu Fri Mar 8 13:25:06 2019 From: ngisolfi at andrew.cmu.edu (Nicholas Gisolfi) Date: Fri, 8 Mar 2019 13:25:06 -0500 Subject: [Community] Weekly Auton Lab Lunch, Fridays in NSH 3104 @ 11:45am-1:30pm EST In-Reply-To: References: Message-ID: Hi Everyone, Thanks to all who attended and connected for making this first lunch a huge success! We already need a bigger room! I'll send details same time, next week. Have a great weekend! - Nick On Fri, Mar 8, 2019 at 11:24 AM Nicholas Gisolfi wrote: > Hi Everyone, > > We'll get started at 11:45 today for our first of many Auton Lab lunches! > We'll be meeting in NSH 3104, and have the space until 1:30pm. Stop by or > connect any time in between. Remember, *no lunch will be provided*, so > this is BYO...L (?). There's your neologism for the day! > > To join via web browser (I don't believe there is Safari support): > https://meet.google.com/oho-jjok-ack > Otherwise, to join by phone, dial +1 515-518-6321 and enter this PIN: 643 > 666 823# > > I'm looking forward to this opportunity to catch up with many of you! > Even if you can only connect for a second just to say hello, at least you > can let us know what you think about using Google Meetings for future > events. > > See you soon! > > - Nick > > On Tue, Mar 5, 2019 at 2:46 PM Nick Gisolfi wrote: > >> Hi Everyone, >> >> Due to popular demand, we'll be extending our community building >> activities to include a weekly Auton Lab lunch! We have NSH 3104 reserved >> from 11:45am to 1:30pm EST for each Friday for the rest of the semester >> (March 8th will be our first event). The room is small, but we can reserve >> a larger space as participation grows from week to week. >> >> This first point may seem counterintuitive...*lunch will not be provided*. >> Instead, the purpose of this weekly lab lunch is to give us the opportunity >> to connect, whether you pack or buy on/off campus. We all eat, so let's >> take our lunch break together. >> >> For those of you off-campus, we'll set up a Google Meeting so you can >> connect and chat with us over lunch. Time zone permitting, feel free to >> eat your meal while connected :) >> >> While the timing of this event will be consistent week-to-week for the >> spring semester, I'll still send out a reminder email before each lunch >> with a url to connect our colleagues outside Pittsburgh. >> >> For the first few weeks, we will consider this experimental, and we are >> open to any and all thoughts/suggestions for how to make it better. Shoot >> me an email or let us know what you think over lunch this Friday! >> >> - Nick >> >> >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From donghanw at cs.cmu.edu Fri Mar 8 13:50:03 2019 From: donghanw at cs.cmu.edu (Donghan Wang) Date: Fri, 8 Mar 2019 13:50:03 -0500 Subject: [Community] Weekly Auton Lab Lunch, Fridays in NSH 3104 @ 11:45am-1:30pm EST In-Reply-To: References: Message-ID: Nick, Thank you for organizing the lunch. Charlotte, Thank you for introducing us to the new website. Looking forward to launching the site. Thanks, Jarod On Fri, Mar 8, 2019 at 1:26 PM Nicholas Gisolfi wrote: > Hi Everyone, > > Thanks to all who attended and connected for making this first lunch a > huge success! We already need a bigger room! I'll send details same time, > next week. > > Have a great weekend! > > - Nick > > On Fri, Mar 8, 2019 at 11:24 AM Nicholas Gisolfi > wrote: > >> Hi Everyone, >> >> We'll get started at 11:45 today for our first of many Auton Lab >> lunches! We'll be meeting in NSH 3104, and have the space until 1:30pm. >> Stop by or connect any time in between. Remember, *no lunch will be >> provided*, so this is BYO...L (?). There's your neologism for the day! >> >> To join via web browser (I don't believe there is Safari support): >> https://meet.google.com/oho-jjok-ack >> Otherwise, to join by phone, dial +1 515-518-6321 and enter this PIN: 643 >> 666 823# >> >> I'm looking forward to this opportunity to catch up with many of you! >> Even if you can only connect for a second just to say hello, at least you >> can let us know what you think about using Google Meetings for future >> events. >> >> See you soon! >> >> - Nick >> >> On Tue, Mar 5, 2019 at 2:46 PM Nick Gisolfi wrote: >> >>> Hi Everyone, >>> >>> Due to popular demand, we'll be extending our community building >>> activities to include a weekly Auton Lab lunch! We have NSH 3104 reserved >>> from 11:45am to 1:30pm EST for each Friday for the rest of the semester >>> (March 8th will be our first event). The room is small, but we can reserve >>> a larger space as participation grows from week to week. >>> >>> This first point may seem counterintuitive...*lunch will not be >>> provided*. Instead, the purpose of this weekly lab lunch is to give us >>> the opportunity to connect, whether you pack or buy on/off campus. We all >>> eat, so let's take our lunch break together. >>> >>> For those of you off-campus, we'll set up a Google Meeting so you can >>> connect and chat with us over lunch. Time zone permitting, feel free to >>> eat your meal while connected :) >>> >>> While the timing of this event will be consistent week-to-week for the >>> spring semester, I'll still send out a reminder email before each lunch >>> with a url to connect our colleagues outside Pittsburgh. >>> >>> For the first few weeks, we will consider this experimental, and we are >>> open to any and all thoughts/suggestions for how to make it better. Shoot >>> me an email or let us know what you think over lunch this Friday! >>> >>> - Nick >>> >>> >>> >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From eyolcu at andrew.cmu.edu Sat Mar 9 16:00:52 2019 From: eyolcu at andrew.cmu.edu (Emre Yolcu) Date: Sat, 9 Mar 2019 16:00:52 -0500 Subject: gpu10: pytorch and cuda Message-ID: <5c842984.1c69fb81.99dd1.4717@mx.google.com> Hi, Right now on gpu10 `nvcc --version` and `nvidia-smi` seem to work, but `python -c ?import torch; print(torch.cuda.is_available())?` prints False. Is anybody running into the same issue? Emre -------------- next part -------------- An HTML attachment was scrubbed... URL: From yichongx at cs.cmu.edu Sat Mar 9 17:28:29 2019 From: yichongx at cs.cmu.edu (Yichong Xu) Date: Sat, 9 Mar 2019 22:28:29 +0000 Subject: gpu10: pytorch and cuda In-Reply-To: <5c842984.1c69fb81.99dd1.4717@mx.google.com> References: <5c842984.1c69fb81.99dd1.4717@mx.google.com> Message-ID: Same issue here. From my iPhone On Mar 9, 2019, at 4:01 PM, Emre Yolcu > wrote: Hi, Right now on gpu10 `nvcc --version` and `nvidia-smi` seem to work, but `python -c ?import torch; print(torch.cuda.is_available())?` prints False. Is anybody running into the same issue? Emre -------------- next part -------------- An HTML attachment was scrubbed... URL: From predragp at andrew.cmu.edu Sat Mar 9 18:30:29 2019 From: predragp at andrew.cmu.edu (Predrag Punosevac) Date: Sat, 09 Mar 2019 18:30:29 -0500 Subject: gpu10: pytorch and cuda In-Reply-To: Message-ID: <1e5beaaf-0a03-4e21-b474-19340de9ba5c@email.android.com> An HTML attachment was scrubbed... URL: From yhechtli at andrew.cmu.edu Sun Mar 10 09:52:42 2019 From: yhechtli at andrew.cmu.edu (Yotam Hechtlinger) Date: Sun, 10 Mar 2019 09:52:42 -0400 Subject: gpu10: pytorch and cuda In-Reply-To: <1e5beaaf-0a03-4e21-b474-19340de9ba5c@email.android.com> References: <1e5beaaf-0a03-4e21-b474-19340de9ba5c@email.android.com> Message-ID: It's not the same cuda version on GPU 10 and the rest, I think different version of tensorflow has to be installed. Check your tensorflow version and if it supports the cuda version on gpu10. On Saturday, March 9, 2019, Predrag Punosevac wrote: > Try CUDA 10.0 instead of 10.1 > > On Mar 9, 2019 5:28 PM, Yichong Xu wrote: > > Same issue here. > > From my iPhone > > On Mar 9, 2019, at 4:01 PM, Emre Yolcu wrote: > > Hi, > > > > Right now on gpu10 `nvcc --version` and `nvidia-smi` seem to work, but > `python -c ?import torch; print(torch.cuda.is_available())?` prints > False. Is anybody running into the same issue? > > > > Emre > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From yichongx at cs.cmu.edu Sun Mar 10 14:23:59 2019 From: yichongx at cs.cmu.edu (Yichong Xu) Date: Sun, 10 Mar 2019 18:23:59 +0000 Subject: gpu10: pytorch and cuda In-Reply-To: References: <1e5beaaf-0a03-4e21-b474-19340de9ba5c@email.android.com> Message-ID: It seems like tensorflow does not support cuda10 right now - it has to be installed from source. But I?m mainly using pytorch though and the version with cuda10 does not run either. Plus, I tried the original cuda example and it cannot find the gpu either: (base) yichongx at gpu10$ ls Makefile readme.txt simplePrintf simplePrintf.cu simplePrintf.o (base) yichongx at gpu10$ ./simplePrintf CUDA error at ../../common/inc/helper_cuda.h:744 code=999(cudaErrorUnknown) "cudaGetDeviceCount(&device_count)" (base) yichongx at gpu10$ Thanks, Yichong On Mar 10, 2019, at 9:52 AM, Yotam Hechtlinger > wrote: It's not the same cuda version on GPU 10 and the rest, I think different version of tensorflow has to be installed. Check your tensorflow version and if it supports the cuda version on gpu10. On Saturday, March 9, 2019, Predrag Punosevac > wrote: Try CUDA 10.0 instead of 10.1 On Mar 9, 2019 5:28 PM, Yichong Xu > wrote: Same issue here. From my iPhone On Mar 9, 2019, at 4:01 PM, Emre Yolcu > wrote: Hi, Right now on gpu10 `nvcc --version` and `nvidia-smi` seem to work, but `python -c ?import torch; print(torch.cuda.is_available())?` prints False. Is anybody running into the same issue? Emre -------------- next part -------------- An HTML attachment was scrubbed... URL: From yhechtli at andrew.cmu.edu Sun Mar 10 15:05:04 2019 From: yhechtli at andrew.cmu.edu (Yotam Hechtlinger) Date: Sun, 10 Mar 2019 15:05:04 -0400 Subject: gpu10: pytorch and cuda In-Reply-To: References: <1e5beaaf-0a03-4e21-b474-19340de9ba5c@email.android.com> Message-ID: Regarding tensorflow you don't need to compile from source. pip install tf-nightly-gpu Should get it done. I think that's what I've done, but it's been few weeks ago, so try it out and if it doesn't work I'll try to debug it. Notice that you'll have to uninstall it and install the regular version when you switch back to the other GPUs. Not sure regarding pytorch, I haven't tried to install it yet. Yotam. On Sun, Mar 10, 2019 at 2:24 PM Yichong Xu wrote: > It seems like tensorflow does not support cuda10 right now - it has to be > installed from source. > But I?m mainly using pytorch though and the version with cuda10 does not > run either. > Plus, I tried the original cuda example and it cannot find the gpu either: > (base) yichongx at gpu10$ ls > Makefile readme.txt simplePrintf simplePrintf.cu simplePrintf.o > (base) yichongx at gpu10$ ./simplePrintf > CUDA error at ../../common/inc/helper_cuda.h:744 > code=999(cudaErrorUnknown) "cudaGetDeviceCount(&device_count)" > (base) yichongx at gpu10$ > > > > *Thanks,* > *Yichong* > > > > On Mar 10, 2019, at 9:52 AM, Yotam Hechtlinger > wrote: > > It's not the same cuda version on GPU 10 and the rest, I think different > version of tensorflow has to be installed. > > Check your tensorflow version and if it supports the cuda version on gpu10. > > > > On Saturday, March 9, 2019, Predrag Punosevac > wrote: > >> Try CUDA 10.0 instead of 10.1 >> >> On Mar 9, 2019 5:28 PM, Yichong Xu wrote: >> >> Same issue here. >> >> From my iPhone >> >> On Mar 9, 2019, at 4:01 PM, Emre Yolcu wrote: >> >> Hi, >> >> >> Right now on gpu10 `nvcc --version` and `nvidia-smi` seem to work, but >> `python -c ?import torch; print(torch.cuda.is_available())?` prints False. >> Is anybody running into the same issue? >> >> >> Emre >> >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From yichongx at cs.cmu.edu Sun Mar 10 15:13:05 2019 From: yichongx at cs.cmu.edu (Yichong Xu) Date: Sun, 10 Mar 2019 19:13:05 +0000 Subject: gpu10: pytorch and cuda In-Reply-To: References: <1e5beaaf-0a03-4e21-b474-19340de9ba5c@email.android.com> Message-ID: <7EC73AFE-8D59-452E-9444-DF0FBFCDBCE2@cs.cmu.edu> I tried installing the nightly version and the same error appears. I guess it is a recent problem - a few weeks ago I can also run pytorch but now it breaks (at that time there were only 3 gpus available on gpu10). Thanks, Yichong On Mar 10, 2019, at 3:05 PM, Yotam Hechtlinger > wrote: Regarding tensorflow you don't need to compile from source. pip install tf-nightly-gpu Should get it done. I think that's what I've done, but it's been few weeks ago, so try it out and if it doesn't work I'll try to debug it. Notice that you'll have to uninstall it and install the regular version when you switch back to the other GPUs. Not sure regarding pytorch, I haven't tried to install it yet. Yotam. On Sun, Mar 10, 2019 at 2:24 PM Yichong Xu > wrote: It seems like tensorflow does not support cuda10 right now - it has to be installed from source. But I?m mainly using pytorch though and the version with cuda10 does not run either. Plus, I tried the original cuda example and it cannot find the gpu either: (base) yichongx at gpu10$ ls Makefile readme.txt simplePrintf simplePrintf.cu simplePrintf.o (base) yichongx at gpu10$ ./simplePrintf CUDA error at ../../common/inc/helper_cuda.h:744 code=999(cudaErrorUnknown) "cudaGetDeviceCount(&device_count)" (base) yichongx at gpu10$ Thanks, Yichong On Mar 10, 2019, at 9:52 AM, Yotam Hechtlinger > wrote: It's not the same cuda version on GPU 10 and the rest, I think different version of tensorflow has to be installed. Check your tensorflow version and if it supports the cuda version on gpu10. On Saturday, March 9, 2019, Predrag Punosevac > wrote: Try CUDA 10.0 instead of 10.1 On Mar 9, 2019 5:28 PM, Yichong Xu > wrote: Same issue here. From my iPhone On Mar 9, 2019, at 4:01 PM, Emre Yolcu > wrote: Hi, Right now on gpu10 `nvcc --version` and `nvidia-smi` seem to work, but `python -c ?import torch; print(torch.cuda.is_available())?` prints False. Is anybody running into the same issue? Emre -------------- next part -------------- An HTML attachment was scrubbed... URL: From predragp at andrew.cmu.edu Sun Mar 10 22:07:32 2019 From: predragp at andrew.cmu.edu (Predrag Punosevac) Date: Sun, 10 Mar 2019 22:07:32 -0400 Subject: gpu10: pytorch and cuda In-Reply-To: <7EC73AFE-8D59-452E-9444-DF0FBFCDBCE2@cs.cmu.edu> References: <1e5beaaf-0a03-4e21-b474-19340de9ba5c@email.android.com> <7EC73AFE-8D59-452E-9444-DF0FBFCDBCE2@cs.cmu.edu> Message-ID: <20190311020732.zM-zvnLJs%predragp@andrew.cmu.edu> Yichong Xu wrote: > I tried installing the nightly version and the same error appears. I > guess it is a recent problem - a few weeks ago I can also run pytorch > but now it breaks (at that time there were only 3 gpus available on > gpu10). This is likely due to the CUDA upgrade. NVidia is aggressively pushing CUDA 10 branch which we already used on this server. Both pytorch and tensor flow were working fine up until I added another GPU card week ago and upgraded the kernel and CUDA to 10.1. I would suggest that we do a bit of debugging in unison with upstream. In my experience upstream has probably not caught yet with latest changes and this is what we see. Instead of me guessing somebody needs to communicate with pytorch and tensor flow developers (via mailing lists). Cheers, Predrag > > > Thanks, > Yichong > > > > On Mar 10, 2019, at 3:05 PM, Yotam Hechtlinger > wrote: > > Regarding tensorflow you don't need to compile from source. > > pip install tf-nightly-gpu > > Should get it done. I think that's what I've done, but it's been few weeks ago, so try it out and if it doesn't work I'll try to debug it. > Notice that you'll have to uninstall it and install the regular version when you switch back to the other GPUs. > > Not sure regarding pytorch, I haven't tried to install it yet. > > Yotam. > > > On Sun, Mar 10, 2019 at 2:24 PM Yichong Xu > wrote: > It seems like tensorflow does not support cuda10 right now - it has to be installed from source. > But I???m mainly using pytorch though and the version with cuda10 does not run either. > Plus, I tried the original cuda example and it cannot find the gpu either: > (base) yichongx at gpu10$ ls > Makefile readme.txt simplePrintf simplePrintf.cu simplePrintf.o > (base) yichongx at gpu10$ ./simplePrintf > CUDA error at ../../common/inc/helper_cuda.h:744 code=999(cudaErrorUnknown) "cudaGetDeviceCount(&device_count)" > (base) yichongx at gpu10$ > > > > Thanks, > Yichong > > > > On Mar 10, 2019, at 9:52 AM, Yotam Hechtlinger > wrote: > > It's not the same cuda version on GPU 10 and the rest, I think different version of tensorflow has to be installed. > > Check your tensorflow version and if it supports the cuda version on gpu10. > > > > On Saturday, March 9, 2019, Predrag Punosevac > wrote: > Try CUDA 10.0 instead of 10.1 > > On Mar 9, 2019 5:28 PM, Yichong Xu > wrote: > Same issue here. > > From my iPhone > > On Mar 9, 2019, at 4:01 PM, Emre Yolcu > wrote: > > > Hi, > > > > Right now on gpu10 `nvcc --version` and `nvidia-smi` seem to work, but `python -c ???import torch; print(torch.cuda.is_available())???` prints False. Is anybody running into the same issue? > > > > Emre > > > From yichongx at cs.cmu.edu Sun Mar 10 22:13:20 2019 From: yichongx at cs.cmu.edu (Yichong Xu) Date: Mon, 11 Mar 2019 02:13:20 +0000 Subject: gpu10: pytorch and cuda In-Reply-To: <20190311020732.zM-zvnLJs%predragp@andrew.cmu.edu> References: <1e5beaaf-0a03-4e21-b474-19340de9ba5c@email.android.com> <7EC73AFE-8D59-452E-9444-DF0FBFCDBCE2@cs.cmu.edu> <20190311020732.zM-zvnLJs%predragp@andrew.cmu.edu> Message-ID: Thanks for the suggestion Predrag! However it seems like I cannot even run the cuda10.1 examples, as I mentioned previously: yichongx at gpu10$ pwd /home/scratch/yichongx/NVIDIA_CUDA-10.1_Samples/0_Simple/simplePrintf yichongx at gpu10$ ls Makefile readme.txt simplePrintf simplePrintf.cu simplePrintf.o yichongx at gpu10$ ./simplePrintf CUDA error at ../../common/inc/helper_cuda.h:744 code=999(cudaErrorUnknown) "cudaGetDeviceCount(&device_count)? This seems like a problem of CUDA its own. I downloaded the cuda10.1 examples from here: https://docs.nvidia.com/cuda/cuda-samples/index.html Thanks, Yichong On Mar 10, 2019, at 10:07 PM, Predrag Punosevac > wrote: Yichong Xu > wrote: I tried installing the nightly version and the same error appears. I guess it is a recent problem - a few weeks ago I can also run pytorch but now it breaks (at that time there were only 3 gpus available on gpu10). This is likely due to the CUDA upgrade. NVidia is aggressively pushing CUDA 10 branch which we already used on this server. Both pytorch and tensor flow were working fine up until I added another GPU card week ago and upgraded the kernel and CUDA to 10.1. I would suggest that we do a bit of debugging in unison with upstream. In my experience upstream has probably not caught yet with latest changes and this is what we see. Instead of me guessing somebody needs to communicate with pytorch and tensor flow developers (via mailing lists). Cheers, Predrag Thanks, Yichong On Mar 10, 2019, at 3:05 PM, Yotam Hechtlinger > wrote: Regarding tensorflow you don't need to compile from source. pip install tf-nightly-gpu Should get it done. I think that's what I've done, but it's been few weeks ago, so try it out and if it doesn't work I'll try to debug it. Notice that you'll have to uninstall it and install the regular version when you switch back to the other GPUs. Not sure regarding pytorch, I haven't tried to install it yet. Yotam. On Sun, Mar 10, 2019 at 2:24 PM Yichong Xu > wrote: It seems like tensorflow does not support cuda10 right now - it has to be installed from source. But I???m mainly using pytorch though and the version with cuda10 does not run either. Plus, I tried the original cuda example and it cannot find the gpu either: (base) yichongx at gpu10$ ls Makefile readme.txt simplePrintf simplePrintf.cu simplePrintf.o (base) yichongx at gpu10$ ./simplePrintf CUDA error at ../../common/inc/helper_cuda.h:744 code=999(cudaErrorUnknown) "cudaGetDeviceCount(&device_count)" (base) yichongx at gpu10$ Thanks, Yichong On Mar 10, 2019, at 9:52 AM, Yotam Hechtlinger > wrote: It's not the same cuda version on GPU 10 and the rest, I think different version of tensorflow has to be installed. Check your tensorflow version and if it supports the cuda version on gpu10. On Saturday, March 9, 2019, Predrag Punosevac > wrote: Try CUDA 10.0 instead of 10.1 On Mar 9, 2019 5:28 PM, Yichong Xu > wrote: Same issue here. From my iPhone On Mar 9, 2019, at 4:01 PM, Emre Yolcu > wrote: Hi, Right now on gpu10 `nvcc --version` and `nvidia-smi` seem to work, but `python -c ???import torch; print(torch.cuda.is_available())???` prints False. Is anybody running into the same issue? Emre -------------- next part -------------- An HTML attachment was scrubbed... URL: From predragp at andrew.cmu.edu Sun Mar 10 22:19:03 2019 From: predragp at andrew.cmu.edu (Predrag Punosevac) Date: Sun, 10 Mar 2019 22:19:03 -0400 Subject: gpu10: pytorch and cuda In-Reply-To: References: <1e5beaaf-0a03-4e21-b474-19340de9ba5c@email.android.com> <7EC73AFE-8D59-452E-9444-DF0FBFCDBCE2@cs.cmu.edu> <20190311020732.zM-zvnLJs%predragp@andrew.cmu.edu> Message-ID: <20190311021903.0s6INaPIk%predragp@andrew.cmu.edu> Yichong Xu wrote: > Thanks for the suggestion Predrag! However it seems like I cannot even run the cuda10.1 examples, as I mentioned previously: > yichongx at gpu10$ pwd > /home/scratch/yichongx/NVIDIA_CUDA-10.1_Samples/0_Simple/simplePrintf > yichongx at gpu10$ ls > Makefile readme.txt simplePrintf simplePrintf.cu simplePrintf.o > yichongx at gpu10$ ./simplePrintf > CUDA error at ../../common/inc/helper_cuda.h:744 code=999(cudaErrorUnknown) "cudaGetDeviceCount(&device_count)??? > > This seems like a problem of CUDA its own. I downloaded the cuda10.1 examples from here: > https://docs.nvidia.com/cuda/cuda-samples/index.html I can't do anything tonight. Later this week (perhaps Tuesday) I will try to reinstall everything. > > > > > Thanks, > Yichong > > > > On Mar 10, 2019, at 10:07 PM, Predrag Punosevac > wrote: > > Yichong Xu > wrote: > > I tried installing the nightly version and the same error appears. I > guess it is a recent problem - a few weeks ago I can also run pytorch > but now it breaks (at that time there were only 3 gpus available on > gpu10). > > > This is likely due to the CUDA upgrade. NVidia is aggressively pushing > CUDA 10 branch which we already used on this server. Both pytorch and > tensor flow were working fine up until I added another GPU card week ago > and upgraded the kernel and CUDA to 10.1. I would suggest that we do a > bit of debugging in unison with upstream. In my experience upstream has > probably not caught yet with latest changes and this is what we see. > Instead of me guessing somebody needs to communicate with pytorch and > tensor flow developers (via mailing lists). > > Cheers, > Predrag > > > > > Thanks, > Yichong > > > > On Mar 10, 2019, at 3:05 PM, Yotam Hechtlinger > wrote: > > Regarding tensorflow you don't need to compile from source. > > pip install tf-nightly-gpu > > Should get it done. I think that's what I've done, but it's been few weeks ago, so try it out and if it doesn't work I'll try to debug it. > Notice that you'll have to uninstall it and install the regular version when you switch back to the other GPUs. > > Not sure regarding pytorch, I haven't tried to install it yet. > > Yotam. > > > On Sun, Mar 10, 2019 at 2:24 PM Yichong Xu > wrote: > It seems like tensorflow does not support cuda10 right now - it has to be installed from source. > But I???m mainly using pytorch though and the version with cuda10 does not run either. > Plus, I tried the original cuda example and it cannot find the gpu either: > (base) yichongx at gpu10$ ls > Makefile readme.txt simplePrintf simplePrintf.cu simplePrintf.o > (base) yichongx at gpu10$ ./simplePrintf > CUDA error at ../../common/inc/helper_cuda.h:744 code=999(cudaErrorUnknown) "cudaGetDeviceCount(&device_count)" > (base) yichongx at gpu10$ > > > > Thanks, > Yichong > > > > On Mar 10, 2019, at 9:52 AM, Yotam Hechtlinger > wrote: > > It's not the same cuda version on GPU 10 and the rest, I think different version of tensorflow has to be installed. > > Check your tensorflow version and if it supports the cuda version on gpu10. > > > > On Saturday, March 9, 2019, Predrag Punosevac > wrote: > Try CUDA 10.0 instead of 10.1 > > On Mar 9, 2019 5:28 PM, Yichong Xu > wrote: > Same issue here. > > From my iPhone > > On Mar 9, 2019, at 4:01 PM, Emre Yolcu > wrote: > > > Hi, > > > > Right now on gpu10 `nvcc --version` and `nvidia-smi` seem to work, but `python -c ???import torch; print(torch.cuda.is_available())???` prints False. Is anybody running into the same issue? > > > > Emre > > > > From ngisolfi at cs.cmu.edu Thu Mar 14 08:50:34 2019 From: ngisolfi at cs.cmu.edu (Nick Gisolfi) Date: Thu, 14 Mar 2019 08:50:34 -0400 Subject: [Lunch] Friday, March 15 @12:15-1:30pm in NSH 3001 Message-ID: Hi Everyone, We'll do another lab lunch tomorrow afternoon. Bring your own meal and enjoy the company of your colleagues! I'll follow up with a link tomorrow for those connecting from outside Pittsburgh. New details: Where: NSH 3001 When: 12:15-1:30pm Also, there's a delicious loaf of multi-grain bread from Madeleine's bakery in NSH 3111, so feel free to stop by and have a slice (or more!) - Nick -------------- next part -------------- An HTML attachment was scrubbed... URL: From awd at cs.cmu.edu Fri Mar 15 09:23:30 2019 From: awd at cs.cmu.edu (Artur Dubrawski) Date: Fri, 15 Mar 2019 09:23:30 -0400 Subject: Fwd: FW: SPECIAL LECTURE: "Applying Machine Learning to Social Problems: What Happens When We're Wrong?" (March 29 @ 12:30pm) In-Reply-To: <87ce54dfd2f4426a8debfe40059562e3@cs.cmu.edu> References: <87ce54dfd2f4426a8debfe40059562e3@cs.cmu.edu> Message-ID: This could be of interest to many of us. Artur *From:* Jay D. Aronson *Sent:* Thursday, March 14, 2019 3:05 PM *To:* Jay D. Aronson *Subject:* SPECIAL LECTURE: "Applying Machine Learning to Social Problems: What Happens When We're Wrong?" (March 29 @ 12:30pm) Hi All, I am writing to let you know about an upcoming lecture that I think will be of interest to you. Please share with anyone (including your colleagues and students) who you think might be interested. Here are the details: Patrick Ball, Director of Research, Human Rights Data Analysis Group (San Francisco, CA) (https://hrdag.org/people/patrick-ball-phd/) ?Applying Machine Learning to Social Problems: What Happens When We're Wrong?? Abstract: Machine learning (ML) enables us to understand very complicated relationships among variables in observed data, and make predictions about other variables. In this talk, I'll contrast two uses of ML: predicting the probable locations of hidden graves of disappeared people in Mexico, and predicting the probable locations of crime using data from police databases. I'll discuss the benefits of each project, and the potential costs of being wrong. In particular, I'll focus on who bears the cost of being wrong as an example of how we can reason about the likely benefits or harms of a particular application of ML to a social or policy problem. DATE: Friday March 29, 2019 Time: 12:30-2pm LOCATION: 4405 Gates and Hillman Centers (GHC 4405); Carnegie Mellon University LUNCH WILL BE SERVED, BUT AN RSVP IS REQUIRED. PLEASE EMAIL ME ( aronson at andrew.cmu.edu) AND LET ME KNOW IF YOU WILL BE ATTENDING, AND ALSO IF YOU HAVE ANY DIETARY RESTRICTIONS. Thanks so much! Jay P.S. I will send a flyer early next week if you want to post it or pass it along. This is a late-breaking event, so I figured I?d let you know about it as soon as the details were sorted out. Also, please let me know if there are any list serves that I should post this to. -- Jay D. Aronson Director, Center for Human Rights Science Professor of Science, Technology, and Society Department of History Carnegie Mellon University 5000 Forbes Ave. 240 Baker Hall Pittsburgh, PA 15213 USA email: aronson at andrew.cmu.edu office phone: 1.412.268.2887 mobile phone: 1.412.877.0955 web: https://www.cmu.edu/dietrich/history/people/faculty/aronson.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From ngisolfi at andrew.cmu.edu Fri Mar 15 12:14:32 2019 From: ngisolfi at andrew.cmu.edu (Nicholas Gisolfi) Date: Fri, 15 Mar 2019 12:14:32 -0400 Subject: [Lunch] Friday, March 15 @12:15-1:30pm in NSH 3001 In-Reply-To: References: Message-ID: Hi Everyone, We're set up for lunch in NSH 3001. We'll be here until around 1:30. To join the video meeting (no safari support) click this link: https://meet.google.com/xax-dpdi-nta Otherwise, to join by phone, dial +1 725-400-4736 and enter this PIN: 940 076 608# See you soon! - Nick On Thu, Mar 14, 2019 at 8:51 AM Nick Gisolfi wrote: > Hi Everyone, > > We'll do another lab lunch tomorrow afternoon. Bring your own meal and > enjoy the company of your colleagues! I'll follow up with a link tomorrow > for those connecting from outside Pittsburgh. New details: > > Where: NSH 3001 > When: 12:15-1:30pm > > Also, there's a delicious loaf of multi-grain bread from Madeleine's > bakery in NSH 3111, so feel free to stop by and have a slice (or more!) > > - Nick > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ngisolfi at andrew.cmu.edu Fri Mar 22 08:33:25 2019 From: ngisolfi at andrew.cmu.edu (Nicholas Gisolfi) Date: Fri, 22 Mar 2019 08:33:25 -0400 Subject: [Lunch] Today @ 12:15-1:00PM in NSH 3001 Message-ID: Hi Everyone, Lab lunch today is at a slightly different time as we had to squeeze between two other meetings. Bring your own meal, and we'll see you there! Where: NSH 3001 When: 12:15-1:00PM I believe that these Google Meeting links stay active, so here's the same info from last week: https://meet.google.com/xax-dpdi-nta See you at lunch! - Nick -------------- next part -------------- An HTML attachment was scrubbed... URL: From predragp at andrew.cmu.edu Tue Mar 26 09:55:17 2019 From: predragp at andrew.cmu.edu (Predrag Punosevac) Date: Tue, 26 Mar 2019 09:55:17 -0400 Subject: Fwd: Power outage to SMith and Newell-Simon Halls 3/27 AM Message-ID: <20190326135517.SK6oTX41Q%predragp@andrew.cmu.edu> This scheduled powerotrage will affect our machines as we no longer use UPSs for desktops. Please make sure you shutdown your desktop before leaving tonight. On the same note bash.autonlab.org (which is my desktop) will be also powered down. I will try during the day to provision another shell gateway for people who no longer can use lop1.autonlab.org due to autofs issues. Cheers, Predrag -------- Original Message -------- To: "ri-nsh at cs.cmu.edu" , "hcii-members at cs" , "fac-supervisors at cs.cmu.edu" , Heather Jones , Jamie Gregory , Jacob Joseph , Stocky , Megan Hofmann , "ri-edsh at cs.cmu.edu" , Jim Skees , Cheryl Wehrer , From: Paul Stockhausen Subject: Power outage to SMith and Newell-Simon Halls 3/27 AM Date: Tue, 26 Mar 2019 09:47:00 -0400 Folks, ?? FMS has scheduled a power shut down for Newell Simon, Smith Hall, Hamburg and the FMS building in order to connect a high voltage feed to the Tepper Building. The outage will last between 15 and 30 minutes. Currently the Timeline is as follows: 4:30am- Shutdown HVAC and Elevators. 5:30am- Shutdown power. 6am- Bring power back up. 6am-7am- Have all systems back up and running. ?? Please power down any non necessary machines before leaving tonight. ?????? Thanks, ?????? ?????? -Paul ?? Based on feedback from the meeting this morning, the brief power interruptions for Hamburg Hall, Smith Hall, Newell Simon Hall and FMS Building have been scheduled for early morning on March 27.?? Power will be interrupted in each building for 15 to 30 minutes between 5AM and 6AM.?? Building systems not served by the emergency generator or battery backup will be affected.?? Prior to the power outage, FMS crews will secure the elevator and HVAC systems. If you have any questions or concerns about the brief power interruptions, please contact Bob Penn atrpenn at andrew.cmu.edu . *From:*fms-smith on behalf of FMS Announce *Date:*Thursday, March 14, 2019 at 11:56 AM *To:*"fms-shutdown at lists.andrew.cmu.edu" , "fms-hamburg at lists.andrew.cmu.edu" , "fms-smith at lists.andrew.cmu.edu" , "fms-newelsimon at lists.andrew.cmu.edu" , "fms-fmsb at lists.andrew.cmu.edu" *Subject:*FMS -- Power Shutdown Discussion Meeting 3/18 at 10:30AM *Division of Operations* *Facilities Management and Campus Services* ** *TO:?? Shutdown Group, Hamburg Hall, Smith Hall, Newell Simon Hall, FMS Building* *FROM:?? Service Response Center* *DATE: ??March 14, 2019* *SUBJECT:?? Power Shutdown Discussion Meeting 3/18 at 10:30AM* A power shutdown needs to be scheduled for Hamburg Hall, Newell Simon Hall, Smith Hall and the FMS Building in order to connect redundant high voltage power service to the Tepper Quad. *There will be a meeting on Monday, 3/18, at 10:30AM in the FMCS 2^nd Floor Conference Room to discuss details of this shutdown. *The intent of the meeting is to solicit input on the potential impacts to critical equipment that would be affected and determine a date/time that avoids or minimizes the most negative effects. ** Please send an email to Shannon Wetzel atswetzel at andrew.cmu.edu if you plan to attend this meeting. _______________________________________________ To manage your subscription to this mailing list, link to: https://lists.andrew.cmu.edu/mailman/listinfo/fms-hamburg -- Paul Stockhausen 412-268-8223 building.cs.cmu.edu From predragp at andrew.cmu.edu Wed Mar 27 18:10:40 2019 From: predragp at andrew.cmu.edu (Predrag Punosevac) Date: Wed, 27 Mar 2019 18:10:40 -0400 Subject: LOV5 hard rebooted Message-ID: Somebody have pushed lov5 too hard. The server became unresponsive and had to be hard rebooted. Cheers, Predrag From ngisolfi at andrew.cmu.edu Fri Mar 29 09:44:18 2019 From: ngisolfi at andrew.cmu.edu (Nicholas Gisolfi) Date: Fri, 29 Mar 2019 09:44:18 -0400 Subject: [Lunch] Today @ 12:15-1:30PM in NSH 3001 Message-ID: Hi Everyone, This week, lunch is back on schedule. We have NSH 3001 from 12:15-1:30pm. Drop by with your own meal (or morning coffee if you're connecting from the west coast :P) https://meet.google.com/xax-dpdi-nta See you at lunch! - Nick -------------- next part -------------- An HTML attachment was scrubbed... URL: