Discussion:
Overcommit and Backfilling Policies
Valentin Reis
2013-02-11 10:40:03 UTC
Permalink
Hello,

I have a question regarding the --overcommit option.
Are there currently schedulers that use this option in a "smart" way?
For instance, a backfilling policy with certain rules on choosing
whether to overcommit or not?
Maybe in maui? moab? in LSF?

Thank you for your time,
Valentin Reis
David Bigagli
2013-02-12 12:07:06 UTC
Permalink
Hi, I am not sure what you mean "smart" way but in openlava/lsf for example
you can configure the MXJ parameter which controls the number of slots on
a host to be numCores+N, or you can set the dynamic scheduling thresholds
such that they may overcommit the host resources.


/David
Post by Valentin Reis
Hello,
I have a question regarding the --overcommit option.
Are there currently schedulers that use this option in a "smart" way?
For instance, a backfilling policy with certain rules on choosing
whether to overcommit or not?
Maybe in maui? moab? in LSF?
Thank you for your time,
Valentin Reis
Moe Jette
2013-02-12 16:40:06 UTC
Permalink
With Slurm you can oversubscribe the CPUs to some level, gang
scheduling if you wish. See
http://www.schedmd.com/slurmdocs/gang_scheduling.html
http://www.schedmd.com/slurmdocs/slurm.conf.html (see "Shared" option)
Post by David Bigagli
Hi, I am not sure what you mean "smart" way but in openlava/lsf for example
you can configure the MXJ parameter which controls the number of slots on
a host to be numCores+N, or you can set the dynamic scheduling thresholds
such that they may overcommit the host resources.
/David
Post by Valentin Reis
Hello,
I have a question regarding the --overcommit option.
Are there currently schedulers that use this option in a "smart" way?
For instance, a backfilling policy with certain rules on choosing
whether to overcommit or not?
Maybe in maui? moab? in LSF?
Thank you for your time,
Valentin Reis
Valentin Reis
2013-02-13 14:19:03 UTC
Permalink
Hello,
Thanks for replying!
My question was not as to assert whether it is indeed possible to
overcommit resources or share them.

I am interested in the actual scheduling decisions in pre-existing
schedulers regarding the overcommit feature.The ability to "reshape" a
parallel job is interesting for a scheduler..

Thank you for your time,
Valentin Reis
Post by Moe Jette
With Slurm you can oversubscribe the CPUs to some level, gang
scheduling if you wish. See
http://www.schedmd.com/slurmdocs/gang_scheduling.html
http://www.schedmd.com/slurmdocs/slurm.conf.html (see "Shared" option)
Post by David Bigagli
Hi, I am not sure what you mean "smart" way but in openlava/lsf for example
you can configure the MXJ parameter which controls the number of slots on
a host to be numCores+N, or you can set the dynamic scheduling thresholds
such that they may overcommit the host resources.
/David
Post by Valentin Reis
Hello,
I have a question regarding the --overcommit option.
Are there currently schedulers that use this option in a "smart" way?
For instance, a backfilling policy with certain rules on choosing
whether to overcommit or not?
Maybe in maui? moab? in LSF?
Thank you for your time,
Valentin Reis
Loading...