<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<div id="divtagdefaultwrapper" dir="ltr" style="font-size: 11pt; color: rgb(0, 0, 0); font-family: "Trebuchet MS", Trebuchet, sans-serif, "EmojiFont", "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols;">
<p>Hi All,<br>
</p>
<p><br>
</p>
<p>Following is snapshot of sample GPU node usage which is usually run by a single user. This pattern seems to have dramatically increased over the last couple of days.<br>
</p>
<p></p>
<div><br>
| 0 GeForce RTX 208... Off | 00000000:18:00.0 Off | N/A |<br>
| 35% 57C P2 92W / 250W | 7500MiB / 11019MiB | 38% Default |<br>
+-------------------------------+----------------------+----------------------+<br>
| 1 GeForce RTX 208... Off | 00000000:3B:00.0 Off | N/A |<br>
| 38% 63C P2 126W / 250W | 3738MiB / 11019MiB | 64% Default |<br>
+-------------------------------+----------------------+----------------------+<br>
| 2 GeForce RTX 208... Off | 00000000:86:00.0 Off | N/A |<br>
| 41% 66C P2 137W / 250W | 3747MiB / 11019MiB | 55% Default |<br>
+-------------------------------+----------------------+----------------------+<br>
| 3 GeForce RTX 208... Off | 00000000:AF:00.0 Off | N/A |<br>
| 41% 67C P2 140W / 250W | 3705MiB / 11019MiB | 54% Default |<br>
</div>
<div><br>
</div>
<div>I usually need to load model backbone for my experiments which requires around 10GB of RAM, but because of above usage pattern I am unable to really use any GPU. One way around this to increase percentage GPU usage and reduce number of GPUs used.<br>
</div>
<div><br>
</div>
<div>Please be mindful especially if you're running across multiple GPU and your process has GPU usage in bursts. Thanks for your help!<br>
</div>
<br>
<p></p>
<div id="Signature">
<div id="divtagdefaultwrapper" dir="ltr" style="font-size: 12pt; color: rgb(0, 0, 0); font-family: "Trebuchet MS", Trebuchet, sans-serif, "EmojiFont", "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols;">
<span style="font-size:11pt"></span><span style="font-size:11pt"></span><span style="font-size:11pt"></span>
<table class="ms-rteTable-lightlines" style="border-collapse:collapse; table-layout:fixed; border:1px solid transparent" cellspacing="0">
<tbody>
<tr class="ms-rteTableEvenRow-lightlines" style="border-collapse:collapse; border:1px solid transparent">
<td class="ms-rteTableEvenCol-lightlines" style="border-collapse:collapse; width:123px; border-top-color:transparent; border-bottom-color:rgb(146,192,224); border-left-color:transparent; border-right-color:transparent; border-top-width:1px; border-bottom-width:1px; border-top-style:solid; border-bottom-style:solid" align="left">
<a href="https://www.autonlab.org/" class="OWAAutoLink" title="Ctrl+Click or tap to follow the link" id="LPNoLP"><img naturalheight="57" naturalwidth="98" class="EmojiInsert" alt="1562005799537" src="cid:9280266f-c767-4b76-91c0-744e5590d275"></a><br>
<br>
</td>
<td class="ms-rteTableOddCol-lightlines" style="border-collapse:collapse; width:248px; border-top-color:transparent; border-bottom-color:rgb(146,192,224); border-left-color:transparent; border-right-color:transparent; border-top-width:1px; border-bottom-width:1px; border-top-style:solid; border-bottom-style:solid">
<span style="font-size:11pt"></span><a href="https://www.linkedin.com/in/sarveshjayaraman/" class="OWAAutoLink" id="LPNoLP"><span style="font-size:11pt">Sarvesh Jayaraman</span></a><br>
<span style="font-size:11pt"></span><span style="font-size:11pt">Sr. Research Analyst,
</span><span style="font-size:11pt">Auton Lab</span><br>
<span style="font-size:11pt">Carnegie Mellon University</span><br>
<span style="font-size:11pt">Mob: +1-240-893-4287</span><br>
<span style="font-size:11pt"></span></td>
</tr>
</tbody>
</table>
<span style="font-size:11pt"></span><br>
</div>
</div>
</div>
</body>
</html>