<div dir="ltr">THis is an important note for everyone working on some of our healthcare projects with Pitt which utilize their CRC data hosting and computing resources.<div>We are expected to acknowledge in our papers and presentations these two additional sources of funding - see the specific NIH and NSF award info below.</div><div><br></div><div>Cheers</div><div>Artur<br><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">---------- Forwarded message ---------<br>From: <strong class="gmail_sendername" dir="auto">Clermont, Gilles</strong> <span dir="auto"><<a href="mailto:cler@pitt.edu">cler@pitt.edu</a>></span><br>Date: Fri, Jul 28, 2023 at 8:48 AM<br>Subject: This has been brought up by CRC at our last renewal and delayed it...please make every effort to acknowledge them in any work we crank out. Our ability to continue using CRC may depend on this<br>To: Hauskrecht, Milos <<a href="mailto:milos@pitt.edu">milos@pitt.edu</a>>, Parker, Bob <<a href="mailto:rparker@pitt.edu">rparker@pitt.edu</a>>, Kyle Miller <<a href="mailto:mille856@andrew.cmu.edu">mille856@andrew.cmu.edu</a>>, <a href="mailto:awd@cs.cmu.edu">awd@cs.cmu.edu</a> <<a href="mailto:awd@cs.cmu.edu">awd@cs.cmu.edu</a>>, Pinsky, Michael R <<a href="mailto:pinsky@pitt.edu">pinsky@pitt.edu</a>>, Al-Zaiti, Salah Shafiq <<a href="mailto:SSA33@pitt.edu">SSA33@pitt.edu</a>><br></div><br><br><div class="msg1731491148270900756">
<div bgcolor="#E9EAEF" lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="m_1731491148270900756WordSection1">
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div style="border:none;border-top:solid #e1e1e1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> Center for Research Computing <<a href="mailto:bgc14@crc.pitt.edu" target="_blank">bgc14@crc.pitt.edu</a>>
<br>
<b>Sent:</b> Friday, July 28, 2023 8:03 AM<br>
<b>To:</b> Clermont, Gilles <<a href="mailto:cler@pitt.edu" target="_blank">cler@pitt.edu</a>><br>
<b>Subject:</b> CRC News July 2023<u></u><u></u></p>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal"><span style="font-size:1.0pt;font-family:"Arial",sans-serif;color:#1c2957">Acknowledge CRC resources, MPI error alerts, new job efficiency display
<u></u><u></u></span></p>
</div>
<table border="0" cellspacing="0" cellpadding="0" width="100%" style="width:100.0%;background:#e4e4e4;border-spacing:0" id="m_1731491148270900756background-table">
<tbody>
<tr>
<td style="background:#d4d4d4;padding:0in 0in 0in 0in">
<div align="center">
<table border="0" cellspacing="0" cellpadding="0" width="580" style="width:435.0pt;border-spacing:0">
<tbody>
<tr style="height:15.0pt">
<td width="580" style="width:435.0pt;padding:0in 0in 0in 0in;height:15.0pt"></td>
</tr>
<tr>
<td width="580" style="width:435.0pt;padding:0in 0in 0in 0in">
<table border="0" cellspacing="0" cellpadding="0" width="580" style="width:435.0pt;border-spacing:0;border-radius:6px 6px 0px 0px" id="m_1731491148270900756top-bar">
<tbody>
<tr>
<td width="565" style="width:423.75pt;padding:0in 0in 0in 0in">
<table border="0" cellspacing="0" cellpadding="0" width="565" style="width:423.75pt;border-spacing:0">
<tbody>
<tr style="height:6.0pt">
<td width="565" style="width:423.75pt;padding:0in 0in 0in 0in;height:6.0pt"></td>
</tr>
</tbody>
</table>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#878a96"><a href="https://centerforresearchcomputing.cmail20.com/t/j-e-ekuthkk-iuuhvhyjk-r/" target="_blank"><span style="color:#053471">Is this email not displaying correctly? View it in
your browser</span></a> <u></u><u></u></span></p>
</div>
<table border="0" cellspacing="0" cellpadding="0" width="565" style="width:423.75pt;border-spacing:0">
<tbody>
<tr style="height:6.0pt">
<td width="565" style="width:423.75pt;padding:0in 0in 0in 0in;height:6.0pt"></td>
</tr>
</tbody>
</table>
</td>
<td width="15" style="width:11.25pt;padding:0in 0in 0in 0in"></td>
</tr>
</tbody>
</table>
</td>
</tr>
<tr>
<td style="background:#003594;padding:0in 0in 0in 0in">
<table border="0" cellpadding="0" style="border-spacing:0">
<tbody>
<tr style="height:22.5pt">
<td width="160" style="width:120.0pt;background:#003594;padding:18.75pt .75pt 18.75pt 24.0pt;height:22.5pt;border-spacing:0" id="m_1731491148270900756header">
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;color:#1c2957"><img border="0" width="160" height="46" style="width:1.6666in;height:.4791in" id="m_1731491148270900756_x0000_i1038" src="http://i5.cmail20.com/ti/j/CA/631/515/230732/images/pitt_logo_2019_s-new2.png" alt="University of Pittsburgh"></span><span style="font-family:"Arial",sans-serif;color:#1c2957"><u></u><u></u></span></p>
</td>
<td width="360" style="width:3.75in;background:#003594;padding:11.25pt .75pt 11.25pt 11.25pt;height:22.5pt" id="m_1731491148270900756header-title">
<p class="MsoNormal" style="line-height:18.0pt"><b><span style="font-size:13.0pt;font-family:"Arial",sans-serif;color:white">Pitt Research
</span></b><b><span style="font-size:24.0pt;font-family:"Arial",sans-serif;color:#1c2957"><br>
</span></b><b><span style="font-size:16.5pt;font-family:"Arial",sans-serif;color:white">Center for Research Computing
</span></b><b><span style="font-size:24.0pt;font-family:"Arial",sans-serif;color:#1c2957"><u></u><u></u></span></b></p>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
<tr id="m_1731491148270900756main-content">
<td width="580" style="width:435.0pt;background:white;padding:0in 0in 0in 0in">
<table border="0" cellspacing="0" cellpadding="0" width="580" style="width:435.0pt;border-spacing:0">
<tbody>
<tr style="border-spacing:0">
<td width="520" style="width:390.0pt;padding:0in 22.5pt 0in 22.5pt">
<table border="0" cellspacing="0" cellpadding="0" width="520" style="width:390.0pt;border-spacing:0">
<tbody>
<tr style="height:15.0pt">
<td width="520" style="width:390.0pt;padding:0in 0in 0in 0in;height:15.0pt"></td>
</tr>
</tbody>
</table>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;color:#1c2957;display:none"><u></u> <u></u></span></p>
<table border="0" cellspacing="0" cellpadding="0" width="520" style="width:390.0pt;border-spacing:0">
<tbody>
<tr>
<td width="520" style="width:390.0pt;padding:0in 0in 0in 0in">
<h2 style="margin-bottom:9.6pt"><span style="font-size:22.5pt;font-family:"Arial",sans-serif;color:#1c2957;font-weight:normal">CRC News July 2023
<u></u><u></u></span></h2>
</td>
</tr>
<tr style="height:7.5pt">
<td width="520" style="width:390.0pt;padding:0in 0in 0in 0in;height:7.5pt"></td>
</tr>
</tbody>
</table>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;color:#1c2957;display:none"><u></u> <u></u></span></p>
<table border="0" cellspacing="0" cellpadding="0" width="520" style="width:390.0pt;border-spacing:0">
<tbody>
<tr style="height:15.0pt">
<td width="520" style="width:390.0pt;padding:0in 0in 0in 0in;height:15.0pt"></td>
</tr>
</tbody>
</table>
<div class="MsoNormal" align="center" style="margin-right:0in;margin-bottom:15.0pt;margin-left:0in;text-align:center">
<span style="font-family:"Arial",sans-serif;color:#1c2957">
<hr size="2" width="100%" noshade style="color:white" align="center">
</span></div>
<table border="0" cellspacing="0" cellpadding="0" width="520" style="width:390.0pt;border-spacing:0">
<tbody>
<tr>
<td width="125" valign="top" style="width:93.75pt;padding:0in 0in 0in 0in">
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;color:#1c2957"><img border="0" width="125" height="105" style="width:1.302in;height:1.0937in" id="m_1731491148270900756_x0000_i1037" src="http://i1.cmail20.com/ei/j/C9/452/A29/csimport/NSFNIHlogos.111836.jpg"></span><span style="font-family:"Arial",sans-serif;color:#1c2957"><u></u><u></u></span></p>
</td>
<td width="15" style="width:11.25pt;padding:0in 0in 0in 0in"></td>
<td width="380" style="width:285.0pt;padding:0in 0in 0in 0in">
<h2 style="margin:0in">
<span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#1c2957">Researchers required to acknowledge NIH S10 and NSF MRI awards when using CRC resources
<u></u><u></u></span></h2>
<div style="margin-bottom:13.5pt">
<p class="MsoNormal" style="margin-bottom:11.25pt;line-height:16.5pt">
<span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#1c2957">CRC’s recent hardware upgrades are thanks to two very large grants from NIH and NSF. Proper citation plays a crucial part in the University being able to secure more grants of this nature
in the future. For all publications that use research conducted at the CRC after July 2022 please follow the instructions below (also found on the CRC website:
<a href="https://centerforresearchcomputing.cmail20.com/t/j-l-ekuthkk-iuuhvhyjk-j/" target="_blank">
<span style="color:#096ac8">https://crc.pitt.edu/about/acknowledge</span></a>). </span><span style="font-size:12.0pt;font-family:"Arial",sans-serif;color:#1c2957"><u></u><u></u></span></p>
<p class="MsoNormal" style="margin-bottom:11.25pt;line-height:16.5pt">
<span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#1c2957">All research products (publications, conference proceedings, and presentations) resulting from an allocation of computing time or storage on HTC (HTC cluster) should include the following
acknowledgement:</span><span style="font-size:12.0pt;font-family:"Arial",sans-serif;color:#1c2957"><u></u><u></u></span></p>
<p class="MsoNormal" style="margin-bottom:11.25pt;line-height:16.5pt">
<strong><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#1c2957">"This research was supported in part by the University of Pittsburgh Center for Research Computing, RRID:SCR_022735, through the resources provided. Specifically, this work
used the HTC cluster, which is supported by NIH award number S10OD028483."</span></strong><span style="font-size:12.0pt;font-family:"Arial",sans-serif;color:#1c2957"><u></u><u></u></span></p>
<p class="MsoNormal" style="margin-bottom:11.25pt;line-height:16.5pt">
<span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#1c2957">All research products (publications, conference proceedings, and presentations) resulting from an allocation of computing time or storage on H2P (SMP, MPI, and GPU clusters) should include
an acknowledgement:</span><span style="font-size:12.0pt;font-family:"Arial",sans-serif;color:#1c2957"><u></u><u></u></span></p>
<p class="MsoNormal" style="margin-bottom:11.25pt;line-height:16.5pt">
<strong><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#1c2957">"This research was supported in part by the University of Pittsburgh Center for Research Computing, RRID:SCR_022735, through the resources provided. Specifically, this work
used the H2P cluster, which is supported by NSF award number OAC-2117681."</span></strong><span style="font-size:12.0pt;font-family:"Arial",sans-serif;color:#1c2957"><u></u><u></u></span></p>
<p class="MsoNormal" style="margin-bottom:11.25pt;line-height:16.5pt">
<span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#1c2957">You are free to also acknowledge or thank individual members of the CRC team.</span><span style="font-size:12.0pt;font-family:"Arial",sans-serif;color:#1c2957"><u></u><u></u></span></p>
<p class="MsoNormal" style="margin-bottom:11.25pt;line-height:16.5pt">
<span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#1c2957">Please remember to also list any research products (publications, conference proceedings, and presentations) resulting from </span><span style="font-size:12.0pt;font-family:"Arial",sans-serif;color:#1c2957">usage
of CRC resources </span><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#1c2957">in any allocation or renewal proposal.</span><span style="font-size:12.0pt;font-family:"Arial",sans-serif;color:#1c2957"><u></u><u></u></span></p>
</div>
</td>
</tr>
<tr style="height:18.75pt">
<td width="520" colspan="3" style="width:390.0pt;padding:0in 0in 0in 0in;height:18.75pt">
</td>
</tr>
</tbody>
</table>
<div class="MsoNormal" align="center" style="margin-right:0in;margin-bottom:15.0pt;margin-left:0in;text-align:center">
<span style="font-family:"Arial",sans-serif;color:#1c2957">
<hr size="2" width="100%" noshade style="color:white" align="center">
</span></div>
<table border="0" cellspacing="0" cellpadding="0" width="520" style="width:390.0pt;border-spacing:0">
<tbody>
<tr>
<td width="125" valign="top" style="width:93.75pt;padding:0in 0in 0in 0in">
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;color:#1c2957"><img border="0" width="125" height="83" style="width:1.302in;height:.8645in" id="m_1731491148270900756_x0000_i1036" src="http://i2.cmail20.com/ei/j/C9/452/A29/csimport/binary-2372130_1920.105234.jpg"></span><span style="font-family:"Arial",sans-serif;color:#1c2957"><u></u><u></u></span></p>
</td>
<td width="15" style="width:11.25pt;padding:0in 0in 0in 0in"></td>
<td width="380" style="width:285.0pt;padding:0in 0in 0in 0in">
<h2 style="margin:0in">
<span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#1c2957">Some MPI Cluster Users Will See Minimum Node Count Error
<u></u><u></u></span></h2>
<div style="margin-bottom:13.5pt">
<p class="MsoNormal" style="margin-bottom:11.25pt;line-height:16.5pt">
<span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#1c2957">Users of the MPI cluster may notice the following output when attempting to submit a job requesting less than 2 nodes:</span><span style="font-size:12.0pt;font-family:"Arial",sans-serif;color:#1c2957"><u></u><u></u></span></p>
<p class="MsoNormal" style="margin-bottom:11.25pt;line-height:16.5pt">
<strong><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#1c2957">sbatch: error: ERROR: Minimum node count is 2!</span></strong><b><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#1c2957"><br>
<strong><span style="font-family:"Arial",sans-serif">sbatch: error: Batch job submission failed: Node count specification invalid</span></strong></span></b><span style="font-size:12.0pt;font-family:"Arial",sans-serif;color:#1c2957"><u></u><u></u></span></p>
<p class="MsoNormal" style="margin-bottom:11.25pt;line-height:16.5pt">
<span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#1c2957">This is to prevent single-node jobs that should otherwise be run on the SMP cluster from using MPI-centric hardware that were configured specifically to support parallel jobs across
multiple compute nodes.</span><span style="font-size:12.0pt;font-family:"Arial",sans-serif;color:#1c2957"><u></u><u></u></span></p>
<p class="MsoNormal" style="margin-bottom:11.25pt;line-height:16.5pt">
<span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#1c2957">To prevent similar undersubscription at the CPU level and for consistency with accounting of service unit usage for MPI jobs, a warning message is currently in place for jobs that request
CPU cores less than the full amount that the node offers (ntasks-per-node):</span><span style="font-size:12.0pt;font-family:"Arial",sans-serif;color:#1c2957"><u></u><u></u></span></p>
<p class="MsoNormal" style="margin-bottom:11.25pt;line-height:16.5pt">
<span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#1c2957">sbatch: WARNING: Your MPI job is not requesting the full number of cores on its nodes: 48 cores on the mpi partition. MPI jobs will need to request the full number of cores to be successfully
submitted to SLURM after July 15, 2023. Please modify your submission script to request the full core amount or consider submitting your job to the SMP cluster.</span><span style="font-size:12.0pt;font-family:"Arial",sans-serif;color:#1c2957"><u></u><u></u></span></p>
<p class="MsoNormal" style="margin-bottom:11.25pt;line-height:16.5pt">
<span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#1c2957">After July 15, MPI jobs that do not request the full node's resources will no longer enter the queue. You will need to adjust your SLURM submission script to request all of the cores
available. At a minimum, your job submission script will need the following SLURM directives:</span><span style="font-size:12.0pt;font-family:"Arial",sans-serif;color:#1c2957"><u></u><u></u></span></p>
<p class="MsoNormal" style="margin-bottom:11.25pt;line-height:16.5pt">
<strong><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#1c2957">#SBATCH --nodes=2</span></strong><b><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#1c2957"><br>
<strong><span style="font-family:"Arial",sans-serif">#SBATCH --ntasks-per-node=48</span></strong><br>
<strong><span style="font-family:"Arial",sans-serif">#SBATCH --cluster=mpi</span></strong><br>
<strong><span style="font-family:"Arial",sans-serif">#SBATCH --partition=mpi</span></strong></span></b><span style="font-size:12.0pt;font-family:"Arial",sans-serif;color:#1c2957"><u></u><u></u></span></p>
<p class="MsoNormal" style="margin-bottom:11.25pt;line-height:16.5pt">
<span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#1c2957">If your MPI workflow is optimized for a certain number of cores, you will need to make sure that the proper amount is being specified to your software by input argument, configuration
file parameter, etc. Please <a href="https://centerforresearchcomputing.cmail20.com/t/j-l-ekuthkk-iuuhvhyjk-t/" target="_blank">
<span style="color:#096ac8">submit a help ticket</span></a> if you need guidance in updating your job submission script.</span><span style="font-size:12.0pt;font-family:"Arial",sans-serif;color:#1c2957"><u></u><u></u></span></p>
</div>
</td>
</tr>
<tr style="height:18.75pt">
<td width="520" colspan="3" style="width:390.0pt;padding:0in 0in 0in 0in;height:18.75pt">
</td>
</tr>
</tbody>
</table>
<div class="MsoNormal" align="center" style="margin-right:0in;margin-bottom:15.0pt;margin-left:0in;text-align:center">
<span style="font-family:"Arial",sans-serif;color:#1c2957">
<hr size="2" width="100%" noshade style="color:white" align="center">
</span></div>
<table border="0" cellspacing="0" cellpadding="0" width="520" style="width:390.0pt;border-spacing:0">
<tbody>
<tr>
<td width="125" valign="top" style="width:93.75pt;padding:0in 0in 0in 0in">
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;color:#1c2957"><img border="0" width="125" height="166" style="width:1.302in;height:1.7291in" id="m_1731491148270900756_x0000_i1035" src="http://i3.cmail20.com/ei/j/C9/452/A29/csimport/Nickolas_Comeau_112.142557.jpg"></span><span style="font-family:"Arial",sans-serif;color:#1c2957"><u></u><u></u></span></p>
</td>
<td width="15" style="width:11.25pt;padding:0in 0in 0in 0in"></td>
<td width="380" style="width:285.0pt;padding:0in 0in 0in 0in">
<h2 style="margin:0in">
<span style="font-size:16.0pt;font-family:"Arial",sans-serif;color:#1c2957">Ask Nick: Can I access information on my job’s efficiency?
<u></u><u></u></span></h2>
<div style="margin-bottom:13.5pt">
<p class="MsoNormal" style="margin-bottom:11.25pt;line-height:16.5pt">
<span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#1c2957">After CRC made a policy change on the lower limit of requested CPUs on MPI (see details above), some users were still submitting and being allocated multiple full nodes worth of CPUs,
but only a small number were being utilized for the duration of the job. We used a script provided in the SLURM installation called "seff" to display this for an individual job, and I created a wrapper around that functionality to show output more easily for
multiple jobs.</span><span style="font-size:12.0pt;font-family:"Arial",sans-serif;color:#1c2957"><u></u><u></u></span></p>
<p class="MsoNormal" style="margin-bottom:11.25pt;line-height:16.5pt">
<span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#1c2957">The idea is to equip users with information about just how much of their job's resources are being used over the duration of the job, given their submission script configuration. This
output could be used by PIs to demonstrate the efficiency of the submitted jobs when they request a yearly allocation of service units. This functionality will be available in mid-August.</span><span style="font-size:12.0pt;font-family:"Arial",sans-serif;color:#1c2957"><u></u><u></u></span></p>
<p class="MsoNormal" style="margin-bottom:11.25pt;line-height:16.5pt">
<span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#1c2957">Users will be able to specify which cluster, which user, and a date range to look for completed jobs. The most valuable information is in the CPU Efficiency and Memory Efficiency items.
When these are high, the full allocation of resources for the job is being used. Low values indicate that the user is requesting more resources than necessary for their processing, and that they should adjust their submission script. The function currently
only shows CPU output, but we hope to add efficiency for GPU usage in the future.</span><span style="font-size:12.0pt;font-family:"Arial",sans-serif;color:#1c2957"><u></u><u></u></span></p>
<p class="MsoNormal" style="margin-bottom:11.25pt;line-height:16.5pt">
<span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#1c2957">In the example below of output for multiple jobs, the top line is the input command, followed by what the user sees regarding</span><span style="font-size:12.0pt;font-family:"Arial",sans-serif;color:#1c2957">
</span><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#1c2957">their completed jobs. Note that one job failed, and therefore the efficiency could not be calculated.</span><span style="font-size:12.0pt;font-family:"Arial",sans-serif;color:#1c2957"><u></u><u></u></span></p>
<p class="MsoNormal" style="margin-bottom:11.25pt;line-height:16.5pt">
<strong><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#1c2957">crc-effective -M mpi -u leb140 -S 06/01/23</span></strong><b><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#1c2957"><br>
<strong><span style="font-family:"Arial",sans-serif">Showing jobs on mpi that completed between 06/01/23 and now for leb140:</span></strong></span></b><span style="font-size:12.0pt;font-family:"Arial",sans-serif;color:#1c2957"><u></u><u></u></span></p>
<p class="MsoNormal" style="margin-bottom:11.25pt;line-height:16.5pt">
<strong><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#1c2957">cluster name: mpi</span></strong><b><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#1c2957"><br>
<strong><span style="font-family:"Arial",sans-serif">Job ID: 1489891</span></strong><br>
<strong><span style="font-family:"Arial",sans-serif">Cluster: mpi</span></strong><br>
<strong><span style="font-family:"Arial",sans-serif">User/Group: leb140/sam</span></strong><br>
<strong><span style="font-family:"Arial",sans-serif">State: COMPLETED (exit code 0)</span></strong><br>
<strong><span style="font-family:"Arial",sans-serif">Nodes: 2</span></strong><br>
<strong><span style="font-family:"Arial",sans-serif">Cores per node: 48</span></strong><br>
<strong><span style="font-family:"Arial",sans-serif">CPU Utilized: 2-08:55:19</span></strong><br>
<strong><span style="font-family:"Arial",sans-serif">CPU Efficiency: 99.10% of 2-09:26:24 core-walltime</span></strong><br>
<strong><span style="font-family:"Arial",sans-serif">Job Wall-clock time: 00:35:54</span></strong><br>
<strong><span style="font-family:"Arial",sans-serif">Memory Utilized: 4.45 MB</span></strong><br>
<strong><span style="font-family:"Arial",sans-serif">Memory Efficiency: 0.00% of 187.50 GB</span></strong></span></b><span style="font-size:12.0pt;font-family:"Arial",sans-serif;color:#1c2957"><u></u><u></u></span></p>
<p class="MsoNormal" style="margin-bottom:11.25pt;line-height:16.5pt">
<strong><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#1c2957">cluster name: mpi</span></strong><b><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#1c2957"><br>
<strong><span style="font-family:"Arial",sans-serif">Job ID: 1490283</span></strong><br>
<strong><span style="font-family:"Arial",sans-serif">Cluster: mpi</span></strong><br>
<strong><span style="font-family:"Arial",sans-serif">User/Group: leb140/sam</span></strong><br>
<strong><span style="font-family:"Arial",sans-serif">State: FAILED (exit code 1)</span></strong><br>
<strong><span style="font-family:"Arial",sans-serif">Cores: 1</span></strong><br>
<strong><span style="font-family:"Arial",sans-serif">CPU Utilized: 00:00:00</span></strong><br>
<strong><span style="font-family:"Arial",sans-serif">CPU Efficiency: 0.00% of 00:00:00 core-walltime</span></strong><br>
<strong><span style="font-family:"Arial",sans-serif">Job Wall-clock time: 00:00:00</span></strong><br>
<strong><span style="font-family:"Arial",sans-serif">Memory Utilized: 0.00 MB (estimated maximum)</span></strong><br>
<strong><span style="font-family:"Arial",sans-serif">Memory Efficiency: 0.00% of 1.95 GB (1.95 GB/core)</span></strong></span></b><span style="font-size:12.0pt;font-family:"Arial",sans-serif;color:#1c2957"><u></u><u></u></span></p>
<p class="MsoNormal" style="margin-bottom:11.25pt;line-height:16.5pt">
<em><span style="font-size:9.0pt;font-family:"Arial",sans-serif;color:#1c2957">Nickolas Comeau is a CRC Research Computing Specialist</span></em><span style="font-size:12.0pt;font-family:"Arial",sans-serif;color:#1c2957"><u></u><u></u></span></p>
</div>
</td>
</tr>
<tr style="height:18.75pt">
<td width="520" colspan="3" style="width:390.0pt;padding:0in 0in 0in 0in;height:18.75pt">
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;color:#1c2957;display:none"><u></u> <u></u></span></p>
<table border="0" cellspacing="0" cellpadding="0" style="border-collapse:collapse;border-spacing:0">
<tbody>
<tr>
<td style="padding:0in 0in 0in 0in">
<div class="MsoNormal" align="center" style="text-align:center"><span style="font-family:"Arial",sans-serif;color:#1c2957">
<hr size="2" width="100%" noshade style="color:white" align="center">
</span></div>
<table border="0" cellpadding="0" style="border-spacing:0">
<tbody>
<tr>
<td width="580" style="width:435.0pt;background:white;padding:7.5pt 22.5pt 11.25pt 22.5pt">
<p class="MsoNormal"><span style="color:black"><a href="https://centerforresearchcomputing.cmail20.com/t/j-l-ekuthkk-iuuhvhyjk-h/" target="_blank"><span style="font-family:"Arial",sans-serif"><img border="0" width="270" height="84" style="width:2.8125in;height:.875in" id="m_1731491148270900756_x0000_i1034" src="http://i6.cmail20.com/ti/j/CA/631/515/230732/images/pitt_logo_2019_blue_s.png" alt="University of Pittsburgh"></span></a></span><span style="font-family:"Arial",sans-serif;color:#1c2957"><u></u><u></u></span></p>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
<tr>
<td style="padding:0in 22.5pt 0in 22.5pt" id="m_1731491148270900756footer-section">
<table border="0" cellpadding="0" style="border-spacing:0">
<tbody>
<tr>
<td width="290" style="width:217.5pt;padding:.75pt .75pt .75pt .75pt;border-spacing:0">
<p><span style="font-family:"Arial",sans-serif;color:#1c2957">University of Pittsburgh<br>
Center for Research Computing<u></u><u></u></span></p>
<p><span style="font-family:"Arial",sans-serif;color:#1c2957">312 Schenley Place<br>
4420 Bayard Street<br>
Pittsburgh, PA 15260<br>
412-648-3094<br>
<a href="https://centerforresearchcomputing.cmail20.com/t/j-l-ekuthkk-iuuhvhyjk-k/" target="_blank">crc.pitt.edu</a><u></u><u></u></span></p>
</td>
<td width="290" valign="top" style="width:217.5pt;padding:.75pt .75pt .75pt .75pt">
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Arial",sans-serif;color:#1c2957;display:none"><u></u> <u></u></span></p>
<table border="0" cellpadding="0" style="border-spacing:0">
<tbody>
<tr>
<td width="580" style="width:435.0pt;padding:7.5pt .75pt .75pt .75pt">
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;color:#1c2957">You are receiving this email as a user of the Center for Research Computing
<u></u><u></u></span></p>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Arial",sans-serif;color:#1c2957;display:none"><u></u> <u></u></span></p>
<table border="0" cellpadding="0" style="border-spacing:0">
<tbody>
<tr style="height:15.0pt">
<td style="padding:.75pt .75pt .75pt .75pt;height:15.0pt"></td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
<tr>
<td width="580" style="width:435.0pt;padding:15.0pt 22.5pt 15.0pt 22.5pt">
<div id="m_1731491148270900756unsubscribe-link">
<p class="MsoNormal"><span style="font-size:9.5pt;font-family:"Arial",sans-serif;color:#1c2957"><a href="https://centerforresearchcomputing.cmail20.com/t/j-u-ekuthkk-iuuhvhyjk-y/" target="_blank"><span style="color:#096ac8">Unsubscribe</span></a>
<u></u><u></u></span></p>
</div>
</td>
</tr>
</tbody>
</table>
</div>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal"><span style="font-family:"Arial",sans-serif;color:#1c2957"><img border="0" width="1" height="1" style="width:.0104in;height:.0104in" id="m_1731491148270900756_x0000_i1033" src="https://centerforresearchcomputing.cmail20.com/t/j-o-ekuthkk-iuuhvhyjk/o.gif"></span><span style="font-family:"Arial",sans-serif;color:#1c2957"><u></u><u></u></span></p>
</div>
</div>
</div></div></div></div>