Discussion:
How to interpret sacct output
Robert Stober
2014-08-05 16:19:40 UTC
Permalink
Please consider this sacct output. My question is how should this output
be understood?

59|usera|defq|2|162|
59.batch|||1|81|

Here's how I'm reading it:

1. usera submitted jobid 59 and requested 2 CPUs.
2. He actually used only one CPU for 81 seconds.
3. As a result, 2 CPUs were reserved and unavailable to anyone during
the 81 second run time, so 162 CPU seconds were effectively used (by
virtue of both CPUs being unavailable during the entire job run time).

Do I have an accurate understanding of this? If so, then it appears that
if I want the total "chargeable" resource usage I should just look at
the primary job record (59 in this case) and skip the records for each
of the job steps since they're included in the primary. Would you concur?

Thank you,

Robert
--
brightcomputing_logo.png

Mr. Robert Stober
Senior Systems Architect


Mob: +1 209 986 9298
Skype: rstober
robert.stober-CnyEIyB4/***@public.gmane.org <mailto:robert.stober-CnyEIyB4/***@public.gmane.org>


Bright Computing, Inc.
2880 Zanker Road
Suite 203
San Jose, CA 95134
Tel: +1 408 300 9448
Fax: +1 408 715 0102
www.BrightComputing.com <http://www.brightcomputing.com>
David Bigagli
2014-08-05 16:52:33 UTC
Permalink
Hi Robert,
the first line is the allocation and the second the batch
step, the batch step runs on one cpu. I am not sure what are the columns
with values 162 and 81. The elapsed time of the allocation should be the
sum of elapsed time of all steps, so if the step ran for 81 seconds I
would expect the elapsed time of the allocation to be 81.

For exaple:

***@prometeo ~/slurm/work>\sacct -o jobid,ncpus,state,elapsed -j 78475
JobID NCPUS State Elapsed
------------ ---------- ---------- ----------
78475 2 COMPLETED 00:00:20
78475.batch 1 COMPLETED 00:00:20
78475.0 2 COMPLETED 00:00:10
78475.1 2 COMPLETED 00:00:10
Post by Robert Stober
Please consider this sacct output. My question is how should this output
be understood?
59|usera|defq|2|162|
59.batch|||1|81|
1. usera submitted jobid 59 and requested 2 CPUs.
2. He actually used only one CPU for 81 seconds.
3. As a result, 2 CPUs were reserved and unavailable to anyone during
the 81 second run time, so 162 CPU seconds were effectively used (by
virtue of both CPUs being unavailable during the entire job run time).
Do I have an accurate understanding of this? If so, then it appears that
if I want the total "chargeable" resource usage I should just look at
the primary job record (59 in this case) and skip the records for each
of the job steps since they're included in the primary. Would you concur?
Thank you,
Robert
--
brightcomputing_logo.png
Mr. Robert Stober
Senior Systems Architect
Mob: +1 209 986 9298
Skype: rstober
Bright Computing, Inc.
2880 Zanker Road
Suite 203
San Jose, CA 95134
Tel: +1 408 300 9448
Fax: +1 408 715 0102
www.BrightComputing.com <http://www.brightcomputing.com>
--
Thanks,
/David/Bigagli

Slurm User Group Meeting
September 23-24, Lugano, Switzerland
Find out more http://slurm.schedmd.com/slurm_ug_agenda.html
www.schedmd.com
Robert Stober
2014-08-05 18:48:50 UTC
Permalink
Hi David,

That's because I forgot to include my sacct command:

sacct -ap -S 2014-01-01 -E now
--format=jobid,user,partition,ncpus,cputimeraw,nodelist

So in my example, the single job step (59.batch) ran on one core and
took 81 seconds. The first entry (which I'm referring to as the "primary
job") shows that he reserved (or requested) two cpus. I'm assuming that
since the two cpus he requested were unavailable to anyone else, Slurm
is just showing that he effectively used 162 cpu seconds (81 x 2).

Thank you,

Robert
Post by David Bigagli
Hi Robert,
the first line is the allocation and the second the batch
step, the batch step runs on one cpu. I am not sure what are the
columns with values 162 and 81. The elapsed time of the allocation
should be the sum of elapsed time of all steps, so if the step ran for
81 seconds I would expect the elapsed time of the allocation to be 81.
JobID NCPUS State Elapsed
------------ ---------- ---------- ----------
78475 2 COMPLETED 00:00:20
78475.batch 1 COMPLETED 00:00:20
78475.0 2 COMPLETED 00:00:10
78475.1 2 COMPLETED 00:00:10
Post by Robert Stober
Please consider this sacct output. My question is how should this output
be understood?
59|usera|defq|2|162|
59.batch|||1|81|
1. usera submitted jobid 59 and requested 2 CPUs.
2. He actually used only one CPU for 81 seconds.
3. As a result, 2 CPUs were reserved and unavailable to anyone during
the 81 second run time, so 162 CPU seconds were effectively used (by
virtue of both CPUs being unavailable during the entire job run time).
Do I have an accurate understanding of this? If so, then it appears that
if I want the total "chargeable" resource usage I should just look at
the primary job record (59 in this case) and skip the records for each
of the job steps since they're included in the primary. Would you concur?
Thank you,
Robert
--
brightcomputing_logo.png
Mr. Robert Stober
Senior Systems Architect
Mob: +1 209 986 9298
Skype: rstober
Bright Computing, Inc.
2880 Zanker Road
Suite 203
San Jose, CA 95134
Tel: +1 408 300 9448
Fax: +1 408 715 0102
www.BrightComputing.com <http://www.brightcomputing.com>
--
brightcomputing_logo.png

Mr. Robert Stober
Senior Systems Architect


Mob: +1 209 986 9298
Skype: rstober
robert.stober-CnyEIyB4/***@public.gmane.org <mailto:robert.stober-CnyEIyB4/***@public.gmane.org>


Bright Computing, Inc.
2880 Zanker Road
Suite 203
San Jose, CA 95134
Tel: +1 408 300 9448
Fax: +1 408 715 0102
www.BrightComputing.com <http://www.brightcomputing.com>
David Bigagli
2014-08-05 20:40:30 UTC
Permalink
Yes that is correct. The first entry is the allocation which has 2 cpus,
-n 2 was specified, the second entry is the batch step that run for 81
seconds, so the total cpu time used was 81 8*2.

Form 'man sacct':

CPUTime Formatted (Elapsed time * CPU) count used by a job or step.

CPUTimeRaw Unlike above non formatted (Elapsed time * CPU) count for
a job or step. Units are cpu-seconds
Post by Robert Stober
Hi David,
sacct -ap -S 2014-01-01 -E now
--format=jobid,user,partition,ncpus,cputimeraw,nodelist
So in my example, the single job step (59.batch) ran on one core and
took 81 seconds. The first entry (which I'm referring to as the "primary
job") shows that he reserved (or requested) two cpus. I'm assuming that
since the two cpus he requested were unavailable to anyone else, Slurm
is just showing that he effectively used 162 cpu seconds (81 x 2).
Thank you,
Robert
Post by David Bigagli
Hi Robert,
the first line is the allocation and the second the batch
step, the batch step runs on one cpu. I am not sure what are the
columns with values 162 and 81. The elapsed time of the allocation
should be the sum of elapsed time of all steps, so if the step ran for
81 seconds I would expect the elapsed time of the allocation to be 81.
JobID NCPUS State Elapsed
------------ ---------- ---------- ----------
78475 2 COMPLETED 00:00:20
78475.batch 1 COMPLETED 00:00:20
78475.0 2 COMPLETED 00:00:10
78475.1 2 COMPLETED 00:00:10
Post by Robert Stober
Please consider this sacct output. My question is how should this output
be understood?
59|usera|defq|2|162|
59.batch|||1|81|
1. usera submitted jobid 59 and requested 2 CPUs.
2. He actually used only one CPU for 81 seconds.
3. As a result, 2 CPUs were reserved and unavailable to anyone during
the 81 second run time, so 162 CPU seconds were effectively used (by
virtue of both CPUs being unavailable during the entire job run time).
Do I have an accurate understanding of this? If so, then it appears that
if I want the total "chargeable" resource usage I should just look at
the primary job record (59 in this case) and skip the records for each
of the job steps since they're included in the primary. Would you concur?
Thank you,
Robert
--
brightcomputing_logo.png
Mr. Robert Stober
Senior Systems Architect
Mob: +1 209 986 9298
Skype: rstober
Bright Computing, Inc.
2880 Zanker Road
Suite 203
San Jose, CA 95134
Tel: +1 408 300 9448
Fax: +1 408 715 0102
www.BrightComputing.com <http://www.brightcomputing.com>
--
brightcomputing_logo.png
Mr. Robert Stober
Senior Systems Architect
Mob: +1 209 986 9298
Skype: rstober
Bright Computing, Inc.
2880 Zanker Road
Suite 203
San Jose, CA 95134
Tel: +1 408 300 9448
Fax: +1 408 715 0102
www.BrightComputing.com <http://www.brightcomputing.com>
--
Thanks,
/David/Bigagli

Slurm User Group Meeting
September 23-24, Lugano, Switzerland
Find out more http://slurm.schedmd.com/slurm_ug_agenda.html
www.schedmd.com
Robert Stober
2014-08-11 18:49:01 UTC
Permalink
Hi David,

One more question with regard to the sacct output:

sacct -o jobid,ncpus,state,cputimeraw,maxrss -j 72

JobID NCPUS State CPUTimeRAW MaxRSS
------------ ---------- ---------- ---------- ----------
72 2 COMPLETED 542
72.batch 1 COMPLETED 271 6708K
72.0 1 FAILED 122 4228K
72.1 1 FAILED 121 4228K

I don't understand why the cputimeraw of the batch step doesn't equal
the sum of the two job steps, which should be 243? Is this because the
batch step cputimeraw also includes system time that was used to run the
job but not used by the job itself?

Also, how should memory be interpreted? Why is the memory consumption of
the batch step higher than either of the two job steps?

Thank you,

Robert
Post by David Bigagli
Yes that is correct. The first entry is the allocation which has 2
cpus, -n 2 was specified, the second entry is the batch step that run
for 81 seconds, so the total cpu time used was 81 8*2.
CPUTime Formatted (Elapsed time * CPU) count used by a job or step.
CPUTimeRaw Unlike above non formatted (Elapsed time * CPU) count for
a job or step. Units are cpu-seconds
Post by Robert Stober
Hi David,
sacct -ap -S 2014-01-01 -E now
--format=jobid,user,partition,ncpus,cputimeraw,nodelist
So in my example, the single job step (59.batch) ran on one core and
took 81 seconds. The first entry (which I'm referring to as the "primary
job") shows that he reserved (or requested) two cpus. I'm assuming that
since the two cpus he requested were unavailable to anyone else, Slurm
is just showing that he effectively used 162 cpu seconds (81 x 2).
Thank you,
Robert
Post by David Bigagli
Hi Robert,
the first line is the allocation and the second the batch
step, the batch step runs on one cpu. I am not sure what are the
columns with values 162 and 81. The elapsed time of the allocation
should be the sum of elapsed time of all steps, so if the step ran for
81 seconds I would expect the elapsed time of the allocation to be 81.
JobID NCPUS State Elapsed
------------ ---------- ---------- ----------
78475 2 COMPLETED 00:00:20
78475.batch 1 COMPLETED 00:00:20
78475.0 2 COMPLETED 00:00:10
78475.1 2 COMPLETED 00:00:10
Post by Robert Stober
Please consider this sacct output. My question is how should this output
be understood?
59|usera|defq|2|162|
59.batch|||1|81|
1. usera submitted jobid 59 and requested 2 CPUs.
2. He actually used only one CPU for 81 seconds.
3. As a result, 2 CPUs were reserved and unavailable to anyone during
the 81 second run time, so 162 CPU seconds were effectively used (by
virtue of both CPUs being unavailable during the entire job run time).
Do I have an accurate understanding of this? If so, then it appears that
if I want the total "chargeable" resource usage I should just look at
the primary job record (59 in this case) and skip the records for each
of the job steps since they're included in the primary. Would you concur?
Thank you,
Robert
--
brightcomputing_logo.png
Mr. Robert Stober
Senior Systems Architect
Mob: +1 209 986 9298
Skype: rstober
Bright Computing, Inc.
2880 Zanker Road
Suite 203
San Jose, CA 95134
Tel: +1 408 300 9448
Fax: +1 408 715 0102
www.BrightComputing.com <http://www.brightcomputing.com>
--
brightcomputing_logo.png
Mr. Robert Stober
Senior Systems Architect
Mob: +1 209 986 9298
Skype: rstober
Bright Computing, Inc.
2880 Zanker Road
Suite 203
San Jose, CA 95134
Tel: +1 408 300 9448
Fax: +1 408 715 0102
www.BrightComputing.com <http://www.brightcomputing.com>
--
brightcomputing_logo.png

Mr. Robert Stober
Senior Systems Architect


Mob: +1 209 986 9298
Skype: rstober
robert.stober-CnyEIyB4/***@public.gmane.org <mailto:robert.stober-CnyEIyB4/***@public.gmane.org>


Bright Computing, Inc.
2880 Zanker Road
Suite 203
San Jose, CA 95134
Tel: +1 408 300 9448
Fax: +1 408 715 0102
www.BrightComputing.com <http://www.brightcomputing.com>
Robert Stober
2014-08-11 19:06:14 UTC
Permalink
Dear Slurm Gurus,

Let's I run a job that requires three CPUs, and each of my nodes has two
CPUs. How can tell if this job used two slots on atom01 and one slot on
atom03, or if it used one slot on atom01 and two slots on atom03?

[***@atom-head ~]# sacct
--format=jobid,user,partition,ncpus,cputimeraw,nodelist,MaxRSS -j 91
JobID User Partition NCPUS CPUTimeRAW
NodeList MaxRSS
------------ --------- ---------- ---------- ---------- ---------------
----------
91 rstober defq 3 84 atom[01,03]
91.batch 1 28 atom01 12272K
91.0 3 48 atom[01,03] 4064K
91.1 3 219 atom[01,03] 4064K

Thank you,

Robert
Post by Robert Stober
Hi David,
sacct -o jobid,ncpus,state,cputimeraw,maxrss -j 72
JobID NCPUS State CPUTimeRAW MaxRSS
------------ ---------- ---------- ---------- ----------
72 2 COMPLETED 542
72.batch 1 COMPLETED 271 6708K
72.0 1 FAILED 122 4228K
72.1 1 FAILED 121 4228K
I don't understand why the cputimeraw of the batch step doesn't equal
the sum of the two job steps, which should be 243? Is this because the
batch step cputimeraw also includes system time that was used to run
the job but not used by the job itself?
Also, how should memory be interpreted? Why is the memory consumption
of the batch step higher than either of the two job steps?
Thank you,
Robert
Post by David Bigagli
Yes that is correct. The first entry is the allocation which has 2
cpus, -n 2 was specified, the second entry is the batch step that run
for 81 seconds, so the total cpu time used was 81 8*2.
CPUTime Formatted (Elapsed time * CPU) count used by a job or step.
CPUTimeRaw Unlike above non formatted (Elapsed time * CPU) count for
a job or step. Units are cpu-seconds
Post by Robert Stober
Hi David,
sacct -ap -S 2014-01-01 -E now
--format=jobid,user,partition,ncpus,cputimeraw,nodelist
So in my example, the single job step (59.batch) ran on one core and
took 81 seconds. The first entry (which I'm referring to as the "primary
job") shows that he reserved (or requested) two cpus. I'm assuming that
since the two cpus he requested were unavailable to anyone else, Slurm
is just showing that he effectively used 162 cpu seconds (81 x 2).
Thank you,
Robert
Post by David Bigagli
Hi Robert,
the first line is the allocation and the second the batch
step, the batch step runs on one cpu. I am not sure what are the
columns with values 162 and 81. The elapsed time of the allocation
should be the sum of elapsed time of all steps, so if the step ran for
81 seconds I would expect the elapsed time of the allocation to be 81.
JobID NCPUS State Elapsed
------------ ---------- ---------- ----------
78475 2 COMPLETED 00:00:20
78475.batch 1 COMPLETED 00:00:20
78475.0 2 COMPLETED 00:00:10
78475.1 2 COMPLETED 00:00:10
Post by Robert Stober
Please consider this sacct output. My question is how should this output
be understood?
59|usera|defq|2|162|
59.batch|||1|81|
1. usera submitted jobid 59 and requested 2 CPUs.
2. He actually used only one CPU for 81 seconds.
3. As a result, 2 CPUs were reserved and unavailable to anyone during
the 81 second run time, so 162 CPU seconds were effectively used (by
virtue of both CPUs being unavailable during the entire job run time).
Do I have an accurate understanding of this? If so, then it appears that
if I want the total "chargeable" resource usage I should just look at
the primary job record (59 in this case) and skip the records for each
of the job steps since they're included in the primary. Would you concur?
Thank you,
Robert
--
brightcomputing_logo.png
Mr. Robert Stober
Senior Systems Architect
Mob: +1 209 986 9298
Skype: rstober
Bright Computing, Inc.
2880 Zanker Road
Suite 203
San Jose, CA 95134
Tel: +1 408 300 9448
Fax: +1 408 715 0102
www.BrightComputing.com <http://www.brightcomputing.com>
--
brightcomputing_logo.png
Mr. Robert Stober
Senior Systems Architect
Mob: +1 209 986 9298
Skype: rstober
Bright Computing, Inc.
2880 Zanker Road
Suite 203
San Jose, CA 95134
Tel: +1 408 300 9448
Fax: +1 408 715 0102
www.BrightComputing.com <http://www.brightcomputing.com>
--
brightcomputing_logo.png
Mr. Robert Stober
Senior Systems Architect
Mob: +1 209 986 9298
Skype: rstober
Bright Computing, Inc.
2880 Zanker Road
Suite 203
San Jose, CA 95134
Tel: +1 408 300 9448
Fax: +1 408 715 0102
www.BrightComputing.com <http://www.brightcomputing.com>
--
brightcomputing_logo.png

Mr. Robert Stober
Senior Systems Architect


Mob: +1 209 986 9298
Skype: rstober
robert.stober-CnyEIyB4/***@public.gmane.org <mailto:robert.stober-CnyEIyB4/***@public.gmane.org>


Bright Computing, Inc.
2880 Zanker Road
Suite 203
San Jose, CA 95134
Tel: +1 408 300 9448
Fax: +1 408 715 0102
www.BrightComputing.com <http://www.brightcomputing.com>
David Bigagli
2014-08-11 20:26:40 UTC
Permalink
Hi,
the cpu time should indeed add up, it looks like in your case 28
seconds are missing.

The MaxRSS is the maximum resident size of the batch step and the
whatever the batch step ran. They don't have to add up. Typically the
batch step includes the shell and the srun while the other steps the
user application and they usually have different memory profile.
Post by Robert Stober
Hi David,
sacct -o jobid,ncpus,state,cputimeraw,maxrss -j 72
JobID NCPUS State CPUTimeRAW MaxRSS
------------ ---------- ---------- ---------- ----------
72 2 COMPLETED 542
72.batch 1 COMPLETED 271 6708K
72.0 1 FAILED 122 4228K
72.1 1 FAILED 121 4228K
I don't understand why the cputimeraw of the batch step doesn't equal
the sum of the two job steps, which should be 243? Is this because the
batch step cputimeraw also includes system time that was used to run the
job but not used by the job itself?
Also, how should memory be interpreted? Why is the memory consumption of
the batch step higher than either of the two job steps?
Thank you,
Robert
Post by David Bigagli
Yes that is correct. The first entry is the allocation which has 2
cpus, -n 2 was specified, the second entry is the batch step that run
for 81 seconds, so the total cpu time used was 81 8*2.
CPUTime Formatted (Elapsed time * CPU) count used by a job or step.
CPUTimeRaw Unlike above non formatted (Elapsed time * CPU) count for
a job or step. Units are cpu-seconds
Post by Robert Stober
Hi David,
sacct -ap -S 2014-01-01 -E now
--format=jobid,user,partition,ncpus,cputimeraw,nodelist
So in my example, the single job step (59.batch) ran on one core and
took 81 seconds. The first entry (which I'm referring to as the "primary
job") shows that he reserved (or requested) two cpus. I'm assuming that
since the two cpus he requested were unavailable to anyone else, Slurm
is just showing that he effectively used 162 cpu seconds (81 x 2).
Thank you,
Robert
Post by David Bigagli
Hi Robert,
the first line is the allocation and the second the batch
step, the batch step runs on one cpu. I am not sure what are the
columns with values 162 and 81. The elapsed time of the allocation
should be the sum of elapsed time of all steps, so if the step ran for
81 seconds I would expect the elapsed time of the allocation to be 81.
JobID NCPUS State Elapsed
------------ ---------- ---------- ----------
78475 2 COMPLETED 00:00:20
78475.batch 1 COMPLETED 00:00:20
78475.0 2 COMPLETED 00:00:10
78475.1 2 COMPLETED 00:00:10
Post by Robert Stober
Please consider this sacct output. My question is how should this output
be understood?
59|usera|defq|2|162|
59.batch|||1|81|
1. usera submitted jobid 59 and requested 2 CPUs.
2. He actually used only one CPU for 81 seconds.
3. As a result, 2 CPUs were reserved and unavailable to anyone during
the 81 second run time, so 162 CPU seconds were effectively used (by
virtue of both CPUs being unavailable during the entire job run time).
Do I have an accurate understanding of this? If so, then it appears that
if I want the total "chargeable" resource usage I should just look at
the primary job record (59 in this case) and skip the records for each
of the job steps since they're included in the primary. Would you concur?
Thank you,
Robert
--
brightcomputing_logo.png
Mr. Robert Stober
Senior Systems Architect
Mob: +1 209 986 9298
Skype: rstober
Bright Computing, Inc.
2880 Zanker Road
Suite 203
San Jose, CA 95134
Tel: +1 408 300 9448
Fax: +1 408 715 0102
www.BrightComputing.com <http://www.brightcomputing.com>
--
brightcomputing_logo.png
Mr. Robert Stober
Senior Systems Architect
Mob: +1 209 986 9298
Skype: rstober
Bright Computing, Inc.
2880 Zanker Road
Suite 203
San Jose, CA 95134
Tel: +1 408 300 9448
Fax: +1 408 715 0102
www.BrightComputing.com <http://www.brightcomputing.com>
--
brightcomputing_logo.png
Mr. Robert Stober
Senior Systems Architect
Mob: +1 209 986 9298
Skype: rstober
Bright Computing, Inc.
2880 Zanker Road
Suite 203
San Jose, CA 95134
Tel: +1 408 300 9448
Fax: +1 408 715 0102
www.BrightComputing.com <http://www.brightcomputing.com>
--
Thanks,
/David/Bigagli

Slurm User Group Meeting
September 23-24, Lugano, Switzerland
Find out more http://slurm.schedmd.com/slurm_ug_agenda.html
www.schedmd.com
Eckert, Phil
2014-08-11 21:37:00 UTC
Permalink
I think that what you are seeing is that since the whole node is being allocated for the job, the 542 is the charge for all cpus on the node for length of time the job ran, even the ones left idle. The step usage is based on the cpu usage for the number of cpus actually used over the same length of time.

Phil Eckert
LLNL

From: Robert Stober <robert.stober-CnyEIyB4/***@public.gmane.org<mailto:***@brightcomputing.com>>
Reply-To: slurm-dev <slurm-dev-***@public.gmane.org<mailto:slurm-dev-***@public.gmane.org>>
Date: Monday, August 11, 2014 at 11:48 AM
To: slurm-dev <slurm-dev-***@public.gmane.org<mailto:slurm-dev-***@public.gmane.org>>
Subject: [slurm-dev] Re: How to interpret sacct output

Hi David,

One more question with regard to the sacct output:

sacct -o jobid,ncpus,state,cputimeraw,maxrss -j 72

JobID NCPUS State CPUTimeRAW MaxRSS
------------ ---------- ---------- ---------- ----------
72 2 COMPLETED 542
72.batch 1 COMPLETED 271 6708K
72.0 1 FAILED 122 4228K
72.1 1 FAILED 121 4228K

I don't understand why the cputimeraw of the batch step doesn't equal the sum of the two job steps, which should be 243? Is this because the batch step cputimeraw also includes system time that was used to run the job but not used by the job itself?

Also, how should memory be interpreted? Why is the memory consumption of the batch step higher than either of the two job steps?

Thank you,

Robert


On 8/5/2014 1:41 PM, David Bigagli wrote:


Yes that is correct. The first entry is the allocation which has 2 cpus, -n 2 was specified, the second entry is the batch step that run for 81 seconds, so the total cpu time used was 81 8*2.

Form 'man sacct':

CPUTime Formatted (Elapsed time * CPU) count used by a job or step.

CPUTimeRaw Unlike above non formatted (Elapsed time * CPU) count for
a job or step. Units are cpu-seconds

On 08/05/2014 11:48 AM, Robert Stober wrote:
Hi David,

That's because I forgot to include my sacct command:

sacct -ap -S 2014-01-01 -E now
--format=jobid,user,partition,ncpus,cputimeraw,nodelist

So in my example, the single job step (59.batch) ran on one core and
took 81 seconds. The first entry (which I'm referring to as the "primary
job") shows that he reserved (or requested) two cpus. I'm assuming that
since the two cpus he requested were unavailable to anyone else, Slurm
is just showing that he effectively used 162 cpu seconds (81 x 2).

Thank you,

Robert

On 8/5/2014 9:53 AM, David Bigagli wrote:

Hi Robert,
the first line is the allocation and the second the batch
step, the batch step runs on one cpu. I am not sure what are the
columns with values 162 and 81. The elapsed time of the allocation
should be the sum of elapsed time of all steps, so if the step ran for
81 seconds I would expect the elapsed time of the allocation to be 81.

For exaple:

***@prometeo ~/slurm/work>\sacct -o jobid,ncpus,state,elapsed -j 78475
JobID NCPUS State Elapsed
------------ ---------- ---------- ----------
78475 2 COMPLETED 00:00:20
78475.batch 1 COMPLETED 00:00:20
78475.0 2 COMPLETED 00:00:10
78475.1 2 COMPLETED 00:00:10


On 08/05/2014 09:19 AM, Robert Stober wrote:
Please consider this sacct output. My question is how should this output
be understood?

59|usera|defq|2|162|
59.batch|||1|81|

Here's how I'm reading it:

1. usera submitted jobid 59 and requested 2 CPUs.
2. He actually used only one CPU for 81 seconds.
3. As a result, 2 CPUs were reserved and unavailable to anyone during
the 81 second run time, so 162 CPU seconds were effectively used (by
virtue of both CPUs being unavailable during the entire job run time).

Do I have an accurate understanding of this? If so, then it appears that
if I want the total "chargeable" resource usage I should just look at
the primary job record (59 in this case) and skip the records for each
of the job steps since they're included in the primary. Would you
concur?

Thank you,

Robert
--

brightcomputing_logo.png

Mr. Robert Stober
Senior Systems Architect


Mob: +1 209 986 9298
Skype: rstober
robert.stober-CnyEIyB4/***@public.gmane.org<mailto:robert.stober-CnyEIyB4/***@public.gmane.org>
<mailto:robert.stober-CnyEIyB4/***@public.gmane.org><mailto:***@brightcomputing.com>


Bright Computing, Inc.
2880 Zanker Road
Suite 203
San Jose, CA 95134
Tel: +1 408 300 9448
Fax: +1 408 715 0102
www.BrightComputing.com<http://www.BrightComputing.com><http://www.brightcomputing.com><http://www.brightcomputing.com>







--

brightcomputing_logo.png

Mr. Robert Stober
Senior Systems Architect


Mob: +1 209 986 9298
Skype: rstober
robert.stober-CnyEIyB4/***@public.gmane.org<mailto:robert.stober-CnyEIyB4/***@public.gmane.org> <mailto:robert.stober-CnyEIyB4/***@public.gmane.org><mailto:***@brightcomputing.com>


Bright Computing, Inc.
2880 Zanker Road
Suite 203
San Jose, CA 95134
Tel: +1 408 300 9448
Fax: +1 408 715 0102
www.BrightComputing.com<http://www.BrightComputing.com><http://www.brightcomputing.com><http://www.brightcomputing.com>







--

[brightcomputing_logo.png]

Mr. Robert Stober
Senior Systems Architect


Mob: +1 209 986 9298
Skype: rstober
robert.stober-CnyEIyB4/***@public.gmane.org<mailto:robert.stober-CnyEIyB4/***@public.gmane.org>


Bright Computing, Inc.
2880 Zanker Road
Suite 203
San Jose, CA 95134
Tel: +1 408 300 9448
Fax: +1 408 715 0102
www.BrightComputing.com<http://www.brightcomputing.com>

Loading...