<div dir="ltr">I actually haven't gotten tensorflow working yet -- the bazel build just hangs on me. I think it maybe has to do with home directories being on NFS, but I can't figure out bazel at all. I'll try some more tonight.<div><br></div><div>Caffe should be workable following the instructions Predrag forwarded.</div><div><br></div><div>- Dougal</div></div><br><div class="gmail_quote"><div dir="ltr">On Thu, Oct 13, 2016 at 6:39 PM Predrag Punosevac <<a href="mailto:predragp@imap.srv.cs.cmu.edu">predragp@imap.srv.cs.cmu.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Dear Autonians,<br class="gmail_msg">
<br class="gmail_msg">
In the case anybody is interested what happens behind the scenes, Doug<br class="gmail_msg">
got Caffe and TensorFlow to work on<br class="gmail_msg">
GPU3. Please see message below. I also got the  very useful feed back<br class="gmail_msg">
from Princeton and Rutgers people. Please check out if you care (you<br class="gmail_msg">
will have to log into Gmail to see the exchange).<br class="gmail_msg">
<br class="gmail_msg">
<a href="https://groups.google.com/forum/#!forum/springdale-users" rel="noreferrer" class="gmail_msg" target="_blank">https://groups.google.com/forum/#!forum/springdale-users</a><br class="gmail_msg">
<br class="gmail_msg">
I need to think how we move forward with this before start pulling<br class="gmail_msg">
triggers. If somebody is itchy and can't wait please build Caffe and<br class="gmail_msg">
TensorFlow in your scratch directory following below howto.<br class="gmail_msg">
<br class="gmail_msg">
Predrag<br class="gmail_msg">
<br class="gmail_msg">
On 2016-10-13 13:24, Dougal Sutherland wrote:<br class="gmail_msg">
> A note about cudnn:<br class="gmail_msg">
><br class="gmail_msg">
> There are a bunch of versions of cudnn. They're not<br class="gmail_msg">
> backwards-compatible, and different versions of<br class="gmail_msg">
> caffe/tensorflow/whatever want different ones.<br class="gmail_msg">
><br class="gmail_msg">
> I currently am using the setup in  ~dsutherl/cudnn_files:<br class="gmail_msg">
><br class="gmail_msg">
>       * I have a bunch of versions of the installer there.<br class="gmail_msg">
>       * The use-cudnn.sh script, intended to be used like "source<br class="gmail_msg">
> use-cudnn.sh 5.1", will untar the appropriate one into a scratch<br class="gmail_msg">
> directory (if it hasn't already been done) and set<br class="gmail_msg">
> CPATH/LIBRARY_PATH/LD_LIBRARY_PATH appropriately. LD_LIBRARY_PATH is<br class="gmail_msg">
> needed for caffe binaries, since they don't link to the absolute path;<br class="gmail_msg">
> the first two (not sure about the the third) are needed for theano.<br class="gmail_msg">
> Dunno about tensorflow yet.<br class="gmail_msg">
><br class="gmail_msg">
> So, here's the Caffe setup:<br class="gmail_msg">
><br class="gmail_msg">
> cd /home/scratch/$USER<br class="gmail_msg">
> git clone <a href="https://github.com/BVLC/caffe" rel="noreferrer" class="gmail_msg" target="_blank">https://github.com/BVLC/caffe</a><br class="gmail_msg">
> cd caffe<br class="gmail_msg">
> cp Makefile.config.example Makefile.config<br class="gmail_msg">
><br class="gmail_msg">
> # tell it to use openblas; using atlas needs some changes to the<br class="gmail_msg">
> Makefile<br class="gmail_msg">
> sed -i 's/BLAS := atlas/BLAS := open/' Makefile.config<br class="gmail_msg">
><br class="gmail_msg">
> # configure to use cudnn (optional)<br class="gmail_msg">
> source ~dsutherl/cudnn-files/use-cudnn.sh 5.1<br class="gmail_msg">
> sed -i 's/# USE_CUDNN := 1/USE_CUDNN := 1/' Makefile.config<br class="gmail_msg">
> perl -i -pe 's|$| '$CUDNN_DIR'/include| if /INCLUDE_DIRS :=/'<br class="gmail_msg">
> Makefile.config<br class="gmail_msg">
> perl -i -pe 's|$| '$CUDNN_DIR'/lib64| if /LIBRARY_DIRS :=/'<br class="gmail_msg">
> Makefile.config<br class="gmail_msg">
><br class="gmail_msg">
> # build the library<br class="gmail_msg">
> make -j23<br class="gmail_msg">
><br class="gmail_msg">
> # to do tests (takes ~10 minutes):<br class="gmail_msg">
> make -j23 test<br class="gmail_msg">
> make runtest<br class="gmail_msg">
><br class="gmail_msg">
> # Now, to run caffe binaries you'll need to remember to source<br class="gmail_msg">
> use-cudnn if you used cudnn before.<br class="gmail_msg">
><br class="gmail_msg">
> # To build the python libary:<br class="gmail_msg">
> make py<br class="gmail_msg">
><br class="gmail_msg">
> # Requirements for the python library:<br class="gmail_msg">
> # Some of the system packages are too old; this installs them in your<br class="gmail_msg">
> scratch directory.<br class="gmail_msg">
> # You'll have to set PYTHONUSERBASE again before running any python<br class="gmail_msg">
> processes that use these libs.<br class="gmail_msg">
> export PYTHONUSERBASE=$HOME/scratch/.local;<br class="gmail_msg">
> export PATH=$PYTHONUSERBASE/bin:"$PATH"  # <- optional<br class="gmail_msg">
> pip install --user -r python/requirements.txt<br class="gmail_msg">
><br class="gmail_msg">
> # Caffe is dumb and doesn't package its python library properly. The<br class="gmail_msg">
> easiest way to use it is:<br class="gmail_msg">
> export PYTHONPATH=/home/scratch/$USER/caffe/python:$PYTHONPATH<br class="gmail_msg">
> python -c 'import caffe'<br class="gmail_msg">
><br class="gmail_msg">
> On Thu, Oct 13, 2016 at 6:01 PM Dougal Sutherland <<a href="mailto:dougal@gmail.com" class="gmail_msg" target="_blank">dougal@gmail.com</a>><br class="gmail_msg">
> wrote:<br class="gmail_msg">
><br class="gmail_msg">
>> Java fix seemed to work. Now tensorflow wants  python-wheel  and<br class="gmail_msg">
>> swig.<br class="gmail_msg">
>><br class="gmail_msg">
>> On Thu, Oct 13, 2016 at 5:08 PM Predrag Punosevac<br class="gmail_msg">
>> <<a href="mailto:predragp@imap.srv.cs.cmu.edu" class="gmail_msg" target="_blank">predragp@imap.srv.cs.cmu.edu</a>> wrote:<br class="gmail_msg">
>><br class="gmail_msg">
>>> On 2016-10-13 11:46, Dougal Sutherland wrote:<br class="gmail_msg">
>>><br class="gmail_msg">
>>>> Having some trouble with tensorflow, because:<br class="gmail_msg">
>>><br class="gmail_msg">
>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>> * it require's Google's bazel build system<br class="gmail_msg">
>>><br class="gmail_msg">
>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>> * The bazel installer says<br class="gmail_msg">
>>><br class="gmail_msg">
>>>> Java version is 1.7.0_111 while at least 1.8 is needed.<br class="gmail_msg">
>>><br class="gmail_msg">
>>>> *<br class="gmail_msg">
>>><br class="gmail_msg">
>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>> * $ java -version<br class="gmail_msg">
>>><br class="gmail_msg">
>>>> openjdk version "1.8.0_102"<br class="gmail_msg">
>>><br class="gmail_msg">
>>>> OpenJDK Runtime Environment (build 1.8.0_102-b14)<br class="gmail_msg">
>>><br class="gmail_msg">
>>>> OpenJDK 64-Bit Server VM (build 25.102-b14, mixed mode)<br class="gmail_msg">
>>><br class="gmail_msg">
>>>> $ javac -version<br class="gmail_msg">
>>><br class="gmail_msg">
>>>> javac 1.7.0_111<br class="gmail_msg">
>>><br class="gmail_msg">
>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>> I just did yum -y install java-1.8.0* which installs openjdk 1.8.<br class="gmail_msg">
>>> Please<br class="gmail_msg">
>>><br class="gmail_msg">
>>> change your java. Let me know if<br class="gmail_msg">
>>><br class="gmail_msg">
>>> you want me to install Oracle JDK 1.8<br class="gmail_msg">
>>><br class="gmail_msg">
>>> Predrag<br class="gmail_msg">
>>><br class="gmail_msg">
>>>> On Thu, Oct 13, 2016 at 4:38 PM Predrag Punosevac<br class="gmail_msg">
>>><br class="gmail_msg">
>>>> <<a href="mailto:predragp@cs.cmu.edu" class="gmail_msg" target="_blank">predragp@cs.cmu.edu</a>> wrote:<br class="gmail_msg">
>>><br class="gmail_msg">
>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> Dougal Sutherland <<a href="mailto:dougal@gmail.com" class="gmail_msg" target="_blank">dougal@gmail.com</a>> wrote:<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>> Also, this seemed to work for me so far for protobuf:<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>> cd /home/scratch/$USER<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>> VER=3.1.0<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>> wget<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>><br class="gmail_msg">
> <a href="https://github.com/google/protobuf/releases/download/v$VER/protobuf-cpp-$VER.tar.gz" rel="noreferrer" class="gmail_msg" target="_blank">https://github.com/google/protobuf/releases/download/v$VER/protobuf-cpp-$VER.tar.gz</a><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>> tar xf protobuf-cpp-$VER.tar.gz<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>> cd protobuf-cpp-$VER<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>> ./configure --prefix=/home/scratch/$USER<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>> make -j12<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>> make -j12 check<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>> make install<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> That is great help!<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>> You could change --prefix=/usr if making an RPM.<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>> On Thu, Oct 13, 2016 at 4:26 PM Dougal Sutherland<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> <<a href="mailto:dougal@gmail.com" class="gmail_msg" target="_blank">dougal@gmail.com</a>> wrote:<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>> Some more packages for caffe:<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>> leveldb-devel snappy-devel opencv-devel boost-devel<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>> hdf5-devel gflags-devel glog-devel lmdb-devel<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>> (Some of those might be installed already, but at least<br class="gmail_msg">
>>> gflags<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> is<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>> definitely missing.)<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>> On Thu, Oct 13, 2016 at 3:44 PM Predrag Punosevac <<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>> <a href="mailto:predragp@imap.srv.cs.cmu.edu" class="gmail_msg" target="_blank">predragp@imap.srv.cs.cmu.edu</a>> wrote:<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>> On 2016-10-12 23:26, Arne Suppe wrote:<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>> Hmm - I don???t use matlab for deep learning, but gpuDevice<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> also hangs<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>> on my computer with R2016a.<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>> We would have to escalate this with MathWorks. I have seen<br class="gmail_msg">
>>> work<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> around<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>> Internet but it looks like a bug in one of Mathworks provided<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> MEX files.<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>> I was able compile the matrixMul example in the CUDA<br class="gmail_msg">
>>> samples<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> and run<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>> it on gpu3, so I think the build environment is probably<br class="gmail_msg">
>>> all<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> set.<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>> As for the openGL, I think its possibly a problem with<br class="gmail_msg">
>>> their<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> build<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>> script <a href="http://findgl.mk" rel="noreferrer" class="gmail_msg" target="_blank">findgl.mk</a> [1] [1] which is not familiar with<br class="gmail_msg">
>>> Springdale OS.<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> The<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>> demo_suite directory has a precompiled nbody binary you may<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> try, but I<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>> suspect most users will not need graphics.<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>> That should not be too hard to fix. Some header files have to<br class="gmail_msg">
>>> be<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>> manually edited. The funny part until 7.2 Princeton people<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> didn't bother<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>> to remove RHEL branding which actually made things easier for<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> us.<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>> Doug is trying right now to compile the latest Caffe,<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> TensorFlow, and<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>> protobuf-3. We will try to create an RPM for that so that we<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> don't have<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>> to go through this again. I also asked Princeton and Rutgers<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> guys if<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>> they<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>> have WIP RPMs to share.<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>> Predrag<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>> Arne<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> On Oct 12, 2016, at 10:23 PM, Predrag Punosevac<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> <<a href="mailto:predragp@cs.cmu.edu" class="gmail_msg" target="_blank">predragp@cs.cmu.edu</a>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> wrote:<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> Arne Suppe <<a href="mailto:suppe@andrew.cmu.edu" class="gmail_msg" target="_blank">suppe@andrew.cmu.edu</a>> wrote:<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>> Hi Predrag,<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>> Don???t know if this applies to you, but I just build a<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> machines with<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>> a GTX1080 which has the same PASCAL architecture as the<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> Titan.  After<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>> installing CUDA 8, I still found I needed to install the<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> latest<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>> driver off of the NVIDIA web site to get the card<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> recognized.  Right<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>> now, I am running 367.44.<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>> Arne<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> Arne,<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> Thank you so much for this e-mail. Yes it is damn PASCAL<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> arhitecture I<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> see lots of people complaining about it on the forums. I<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> downloaded<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> and<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> installed driver from<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>><br class="gmail_msg">
> <a href="http://www.nvidia.com/content/DriverDownload-March2009/confirmation.php?url=/XFree86/Linux-x86_64/367.57/NVIDIA-Linux-x86_64-367.57.run&lang=us&type=GeForce" rel="noreferrer" class="gmail_msg" target="_blank">http://www.nvidia.com/content/DriverDownload-March2009/confirmation.php?url=/XFree86/Linux-x86_64/367.57/NVIDIA-Linux-x86_64-367.57.run&lang=us&type=GeForce</a><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> That seems to made a real difference. Check out this<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> beautiful outputs<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> root@gpu3$ ls nvidia*<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> nvidia0  nvidia1  nvidia2  nvidia3  nvidiactl  nvidia-uvm<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> nvidia-uvm-tools<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> root@gpu3$ lspci | grep -i nvidia<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> 02:00.0 VGA compatible controller: NVIDIA Corporation<br class="gmail_msg">
>>> Device<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> 1b00 (rev<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> a1)<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> 02:00.1 Audio device: NVIDIA Corporation Device 10ef (rev<br class="gmail_msg">
>>> a1)<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> 03:00.0 VGA compatible controller: NVIDIA Corporation<br class="gmail_msg">
>>> Device<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> 1b00 (rev<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> a1)<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> 03:00.1 Audio device: NVIDIA Corporation Device 10ef (rev<br class="gmail_msg">
>>> a1)<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> 82:00.0 VGA compatible controller: NVIDIA Corporation<br class="gmail_msg">
>>> Device<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> 1b00 (rev<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> a1)<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> 82:00.1 Audio device: NVIDIA Corporation Device 10ef (rev<br class="gmail_msg">
>>> a1)<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> 83:00.0 VGA compatible controller: NVIDIA Corporation<br class="gmail_msg">
>>> Device<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> 1b00 (rev<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> a1)<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> 83:00.1 Audio device: NVIDIA Corporation Device 10ef (rev<br class="gmail_msg">
>>> a1)<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> root@gpu3$ ls /proc/driver<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> nvidia  nvidia-uvm  nvram  rtc<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> root@gpu3$ lsmod |grep nvidia<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> nvidia_uvm            738901  0<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> nvidia_drm             43405  0<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> nvidia_modeset        764432  1 nvidia_drm<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> nvidia              11492947  2 nvidia_modeset,nvidia_uvm<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> drm_kms_helper        125056  2 ast,nvidia_drm<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> drm                   349210  5<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> ast,ttm,drm_kms_helper,nvidia_drm<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> i2c_core               40582  7<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> ast,drm,igb,i2c_i801,drm_kms_helper,i2c_algo_bit,nvidia<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> root@gpu3$ nvidia-smi<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> Wed Oct 12 22:03:27 2016<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>><br class="gmail_msg">
> +-----------------------------------------------------------------------------+<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> | NVIDIA-SMI 367.57                 Driver Version: 367.57<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> |<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>><br class="gmail_msg">
> |-------------------------------+----------------------+----------------------+<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> | GPU  Name        Persistence-M| Bus-Id        Disp.A |<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> Volatile<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> Uncorr. ECC |<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage |<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> GPU-Util<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> Compute M. |<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>><br class="gmail_msg">
> |===============================+======================+======================|<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> |   0  TITAN X (Pascal)    Off  | 0000:02:00.0     Off |<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> N/A |<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> | 23%   32C    P0    56W / 250W |      0MiB / 12189MiB |<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> 0%<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> Default |<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>><br class="gmail_msg">
> +-------------------------------+----------------------+----------------------+<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> |   1  TITAN X (Pascal)    Off  | 0000:03:00.0     Off |<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> N/A |<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> | 23%   36C    P0    57W / 250W |      0MiB / 12189MiB |<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> 0%<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> Default |<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>><br class="gmail_msg">
> +-------------------------------+----------------------+----------------------+<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> |   2  TITAN X (Pascal)    Off  | 0000:82:00.0     Off |<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> N/A |<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> | 23%   35C    P0    57W / 250W |      0MiB / 12189MiB |<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> 0%<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> Default |<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>><br class="gmail_msg">
> +-------------------------------+----------------------+----------------------+<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> |   3  TITAN X (Pascal)    Off  | 0000:83:00.0     Off |<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> N/A |<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> |  0%   35C    P0    56W / 250W |      0MiB / 12189MiB |<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> 0%<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> Default |<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>><br class="gmail_msg">
> +-------------------------------+----------------------+----------------------+<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>><br class="gmail_msg">
> +-----------------------------------------------------------------------------+<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> | Processes:<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> GPU<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> Memory |<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> |  GPU       PID  Type  Process name<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> Usage<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> |<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>><br class="gmail_msg">
> |=============================================================================|<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> |  No running processes found<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> |<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>><br class="gmail_msg">
> +-----------------------------------------------------------------------------+<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> /usr/local/cuda/extras/demo_suite/deviceQuery<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> Alignment requirement for Surfaces:            Yes<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> Device has ECC support:                        Disabled<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> Device supports Unified Addressing (UVA):      Yes<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> Device PCI Domain ID / Bus ID / location ID:   0 / 131 /<br class="gmail_msg">
>>> 0<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> Compute Mode:<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> < Default (multiple host threads can use<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> ::cudaSetDevice() with<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> device simultaneously) ><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>> Peer access from TITAN X (Pascal) (GPU0) -> TITAN X<br class="gmail_msg">
>>> (Pascal)<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> (GPU1) :<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> Yes<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>> Peer access from TITAN X (Pascal) (GPU0) -> TITAN X<br class="gmail_msg">
>>> (Pascal)<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> (GPU2) :<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> No<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>> Peer access from TITAN X (Pascal) (GPU0) -> TITAN X<br class="gmail_msg">
>>> (Pascal)<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> (GPU3) :<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> No<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>> Peer access from TITAN X (Pascal) (GPU1) -> TITAN X<br class="gmail_msg">
>>> (Pascal)<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> (GPU0) :<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> Yes<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>> Peer access from TITAN X (Pascal) (GPU1) -> TITAN X<br class="gmail_msg">
>>> (Pascal)<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> (GPU2) :<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> No<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>> Peer access from TITAN X (Pascal) (GPU1) -> TITAN X<br class="gmail_msg">
>>> (Pascal)<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> (GPU3) :<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> No<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>> Peer access from TITAN X (Pascal) (GPU2) -> TITAN X<br class="gmail_msg">
>>> (Pascal)<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> (GPU0) :<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> No<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>> Peer access from TITAN X (Pascal) (GPU2) -> TITAN X<br class="gmail_msg">
>>> (Pascal)<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> (GPU1) :<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> No<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>> Peer access from TITAN X (Pascal) (GPU2) -> TITAN X<br class="gmail_msg">
>>> (Pascal)<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> (GPU3) :<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> Yes<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>> Peer access from TITAN X (Pascal) (GPU3) -> TITAN X<br class="gmail_msg">
>>> (Pascal)<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> (GPU0) :<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> No<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>> Peer access from TITAN X (Pascal) (GPU3) -> TITAN X<br class="gmail_msg">
>>> (Pascal)<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> (GPU1) :<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> No<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>> Peer access from TITAN X (Pascal) (GPU3) -> TITAN X<br class="gmail_msg">
>>> (Pascal)<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> (GPU2) :<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> Yes<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> deviceQuery, CUDA Driver = CUDART, CUDA Driver Version =<br class="gmail_msg">
>>> 8.0,<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> CUDA<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> Runtime Version = 8.0, NumDevs = 4, Device0 = TITAN X<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> (Pascal),<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> Device1<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> = TITAN X (Pascal), Device2 = TITAN X (Pascal), Device3 =<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> TITAN X<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> (Pascal)<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> Result = PASS<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> Now not everything is rosy<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> root@gpu3$ cd<br class="gmail_msg">
>>> ~/NVIDIA_CUDA-8.0_Samples/5_Simulations/nbody<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> root@gpu3$ make<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>>> WARNING - libGL.so not found, refer to CUDA Getting<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> Started Guide<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> for how to find and install them. <<<<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>>> WARNING - libGLU.so not found, refer to CUDA Getting<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> Started Guide<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> for how to find and install them. <<<<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>>> WARNING - libX11.so not found, refer to CUDA Getting<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> Started Guide<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> for how to find and install them. <<<<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> even though those are installed. For example<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> root@gpu3$ yum whatprovides  */libX11.so<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> libX11-devel-1.6.3-2.el7.i686 : Development files for<br class="gmail_msg">
>>> libX11<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> Repo        : core<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> Matched from:<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> Filename    : /usr/lib/libX11.so<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> also<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> mesa-libGLU-devel<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> mesa-libGL-devel<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> xorg-x11-drv-nvidia-devel<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> but<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> root@gpu3$ yum -y install mesa-libGLU-devel<br class="gmail_msg">
>>> mesa-libGL-devel<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> xorg-x11-drv-nvidia-devel<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> Package mesa-libGLU-devel-9.0.0-4.el7.x86_64 already<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> installed and<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> latest version<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> Package mesa-libGL-devel-10.6.5-3.20150824.el7.x86_64<br class="gmail_msg">
>>> already<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> installed<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> and latest version<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> Package 1:xorg-x11-drv-nvidia-devel-367.48-1.el7.x86_64<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> already<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> installed and latest version<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> Also from MATLAB gpuDevice hangs.<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> So we still don't have a working installation. Any help<br class="gmail_msg">
>>> would<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> be<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> appreciated.<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> Best,<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> Predrag<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> P.S. Once we have a working installation we can think of<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> installing<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> Caffe and TensorFlow. For now we have to see why the<br class="gmail_msg">
>>> things<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> are not<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>> working.<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> On Oct 12, 2016, at 6:26 PM, Predrag Punosevac<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> <<a href="mailto:predragp@cs.cmu.edu" class="gmail_msg" target="_blank">predragp@cs.cmu.edu</a>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> wrote:<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> Dear Autonians,<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> GPU3 is "configured". Namely you can log into it and all<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> packages<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> are<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> installed. However I couldn't get NVIDIA provided CUDA<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> driver to<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> recognize GPU cards. They appear to be properly<br class="gmail_msg">
>>> installed<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> from the<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> hardware point of view and you can list them with<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> lshw -class display<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> root@gpu3$ lshw -class display<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> *-display UNCLAIMED<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> description: VGA compatible controller<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> product: NVIDIA Corporation<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> vendor: NVIDIA Corporation<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> physical id: 0<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> bus info: pci@0000:02:00.0<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> version: a1<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> width: 64 bits<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> clock: 33MHz<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> capabilities: pm msi pciexpress vga_controller<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> cap_list<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> configuration: latency=0<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> resources: iomemory:383f0-383ef<br class="gmail_msg">
>>> iomemory:383f0-383ef<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> memory:cf000000-cfffffff<br class="gmail_msg">
>>> memory:383fe0000000-383fefffffff<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> memory:383ff0000000-383ff1ffffff ioport:6000(size=128)<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> memory:d0000000-d007ffff<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> *-display UNCLAIMED<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> description: VGA compatible controller<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> product: NVIDIA Corporation<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> vendor: NVIDIA Corporation<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> physical id: 0<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> bus info: pci@0000:03:00.0<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> version: a1<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> width: 64 bits<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> clock: 33MHz<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> capabilities: pm msi pciexpress vga_controller<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> cap_list<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> configuration: latency=0<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> resources: iomemory:383f0-383ef<br class="gmail_msg">
>>> iomemory:383f0-383ef<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> memory:cd000000-cdffffff<br class="gmail_msg">
>>> memory:383fc0000000-383fcfffffff<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> memory:383fd0000000-383fd1ffffff ioport:5000(size=128)<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> memory:ce000000-ce07ffff<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> *-display<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> description: VGA compatible controller<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> product: ASPEED Graphics Family<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> vendor: ASPEED Technology, Inc.<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> physical id: 0<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> bus info: pci@0000:06:00.0<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> version: 30<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> width: 32 bits<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> clock: 33MHz<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> capabilities: pm msi vga_controller bus_master<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> cap_list rom<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> configuration: driver=ast latency=0<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> resources: irq:19 memory:cb000000-cbffffff<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> memory:cc000000-cc01ffff ioport:4000(size=128)<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> *-display UNCLAIMED<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> description: VGA compatible controller<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> product: NVIDIA Corporation<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> vendor: NVIDIA Corporation<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> physical id: 0<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> bus info: pci@0000:82:00.0<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> version: a1<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> width: 64 bits<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> clock: 33MHz<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> capabilities: pm msi pciexpress vga_controller<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> cap_list<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> configuration: latency=0<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> resources: iomemory:387f0-387ef<br class="gmail_msg">
>>> iomemory:387f0-387ef<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> memory:fa000000-faffffff<br class="gmail_msg">
>>> memory:387fe0000000-387fefffffff<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> memory:387ff0000000-387ff1ffffff ioport:e000(size=128)<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> memory:fb000000-fb07ffff<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> *-display UNCLAIMED<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> description: VGA compatible controller<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> product: NVIDIA Corporation<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> vendor: NVIDIA Corporation<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> physical id: 0<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> bus info: pci@0000:83:00.0<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> version: a1<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> width: 64 bits<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> clock: 33MHz<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> capabilities: pm msi pciexpress vga_controller<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> cap_list<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> configuration: latency=0<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> resources: iomemory:387f0-387ef<br class="gmail_msg">
>>> iomemory:387f0-387ef<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> memory:f8000000-f8ffffff<br class="gmail_msg">
>>> memory:387fc0000000-387fcfffffff<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> memory:387fd0000000-387fd1ffffff ioport:d000(size=128)<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> memory:f9000000-f907ffff<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> However what scares the hell out of me is that I don't<br class="gmail_msg">
>>> see<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> NVIDIA<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> driver<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> loaded<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> lsmod|grep nvidia<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> and the device nodes /dev/nvidia are not created. I am<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> guessing I<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> just<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> missed some trivial step during the CUDA installation<br class="gmail_msg">
>>> which<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> is very<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> involving. I am unfortunately too tired to debug this<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>> tonight.<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>>>> Predrag<br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>><br class="gmail_msg">
>>><br class="gmail_msg">
>>>> Links:<br class="gmail_msg">
>>><br class="gmail_msg">
>>>> ------<br class="gmail_msg">
>>><br class="gmail_msg">
>>>> [1] <a href="http://findgl.mk" rel="noreferrer" class="gmail_msg" target="_blank">http://findgl.mk</a><br class="gmail_msg">
><br class="gmail_msg">
><br class="gmail_msg">
> Links:<br class="gmail_msg">
> ------<br class="gmail_msg">
> [1] <a href="http://findgl.mk" rel="noreferrer" class="gmail_msg" target="_blank">http://findgl.mk</a><br class="gmail_msg">
</blockquote></div>