gpu20 and gpu21 provisioned

[Jeff Schneider]

Thanks for the hard work bringing these online!

On 5/15/2020 11:06 PM, Predrag Punosevac wrote:
> Dear Autonians,
>
> After about 12h of work you can finally log into the newest addition to
> our cluster: GPU20 and GPU21. These are some of the finest GPU nodes on
> the CMU campus. They came with a price tag of 37K each after 8K per unit
> education discount. I have no idea whom Dr. Jeff Schneider shook down
> for money but he surely knows how to do it. Each machine has 4 Tesla
> V100 GPU cards which have 32GB of GPU memory. That is sufficient to
> train 3D neural networks and to answer all the questions Dr. Dubrawski
> might ask you :-) On a serious note, we had really hard time getting
> these monsters onto the campus during the Cov19 and provisioning them
> under current circumstances. We didn't have deep rack space for these
> nor electricity so I am temporary borrowing space and electricity from
> somebody else. You will notice that network is only 1Gigabit as I could
> not get 10Gigabit network working with 30m Cat 5e which was needed to
> plug machines into our switch. That is how far they are actually
> physically located from our cluster.
>
> I will have to take sleep before adding scratch directories and
> installing MATLAB.
>
> Best,
> Predrag
>
> P.S. Dr. Schneider has more surprises but I will need to make a new trip
> to CMU in my hazmat suit before those babes are brought online.
>
>
>
>
>
>
>
> predragp at gpu20$ nvidia-smi
> Fri May 15 22:25:55 2020
> +-----------------------------------------------------------------------------+
> | NVIDIA-SMI 440.64.00    Driver Version: 440.64.00    CUDA Version:
> 10.2     |
> |-------------------------------+----------------------+----------------------+
> | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile
> Uncorr. ECC |
> | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util
> Compute M. |
> |===============================+======================+======================|
> |   0  Tesla V100-SXM2...  Off  | 00000000:61:00.0 Off |
>    0 |
> | N/A   42C    P0    53W / 300W |      0MiB / 32510MiB |      0%
> Default |
> +-------------------------------+----------------------+----------------------+
> |   1  Tesla V100-SXM2...  Off  | 00000000:62:00.0 Off |
>    0 |
> | N/A   41C    P0    53W / 300W |      0MiB / 32510MiB |      0%
> Default |
> +-------------------------------+----------------------+----------------------+
> |   2  Tesla V100-SXM2...  Off  | 00000000:89:00.0 Off |
>    0 |
> | N/A   40C    P0    56W / 300W |      0MiB / 32510MiB |      0%
> Default |
> +-------------------------------+----------------------+----------------------+
> |   3  Tesla V100-SXM2...  Off  | 00000000:8A:00.0 Off |
>    0 |
> | N/A   41C    P0    55W / 300W |      0MiB / 32510MiB |      0%
> Default |
> +-------------------------------+----------------------+----------------------+
>
>       
> +-----------------------------------------------------------------------------+
> | Processes:                                                       GPU
> Memory |
> |  GPU       PID   Type   Process name                             Usage
>      |
> |=============================================================================|
> |  No running processes found
>      |
> +-----------------------------------------------------------------------------+
>
>
>
> egp at gpu21$ pwd
> /zfsauton2/home/predragp
> predragp at gpu21$ nvidia-smi
> Fri May 15 22:52:16 2020
> +-----------------------------------------------------------------------------+
> | NVIDIA-SMI 440.64.00    Driver Version: 440.64.00    CUDA Version:
> 10.2     |
> |-------------------------------+----------------------+----------------------+
> | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile
> Uncorr. ECC |
> | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util
> Compute M. |
> |===============================+======================+======================|
> |   0  Tesla V100-SXM2...  Off  | 00000000:61:00.0 Off |
>    0 |
> | N/A   35C    P0    54W / 300W |      0MiB / 32510MiB |      0%
> Default |
> +-------------------------------+----------------------+----------------------+
> |   1  Tesla V100-SXM2...  Off  | 00000000:62:00.0 Off |
>    0 |
> | N/A   34C    P0    54W / 300W |      0MiB / 32510MiB |      0%
> Default |
> +-------------------------------+----------------------+----------------------+
> |   2  Tesla V100-SXM2...  Off  | 00000000:89:00.0 Off |
>    0 |
> | N/A   34C    P0    54W / 300W |      0MiB / 32510MiB |      0%
> Default |
> +-------------------------------+----------------------+----------------------+
> |   3  Tesla V100-SXM2...  Off  | 00000000:8A:00.0 Off |
>    0 |
> | N/A   34C    P0    57W / 300W |      0MiB / 32510MiB |      0%
> Default |
> +-------------------------------+----------------------+----------------------+
>
>       
> +-----------------------------------------------------------------------------+
> | Processes:                                                       GPU
> Memory |
> |  GPU       PID   Type   Process name                             Usage
>      |
> |=============================================================================|
> |  No running processes found
>      |
> +-----------------------------------------------------------------------------+
> predragp at gpu21$ ounted.