pbsdsh -u equivalent

[Hartley Greenwald]

Hi,

Is there an equivalent command on slurm for the pbs command pbsdsh -u?
That is to say, is there some command which will give one copy of a command
to each node in a given allocation? I've combed through the documentation
and there doesn't seem to be, but that struck me as odd that there
wouldn't, so that's why I'm asking

Thank you,
Hartley

[Eckert, Phil]

Hartley,

Sounds like you might be wanting srun.

salloc -p pdebug -N 5

salloc: Granted job allocation 1966117

srun hostname

rzmerl1
rzmerl2
rzmerl4
rzmerl3
rzmerl5

Phil Eckert
LLNL

From: Hartley Greenwald <jhgreenwald-***@public.gmane.org<mailto:jhgreenwald-***@public.gmane.org>>
Reply-To: slurm-dev <slurm-dev-***@public.gmane.org<mailto:slurm-dev-***@public.gmane.org>>
Date: Monday, June 30, 2014 at 2:23 PM
To: slurm-dev <slurm-dev-***@public.gmane.org<mailto:slurm-dev-***@public.gmane.org>>
Subject: [slurm-dev] pbsdsh -u equivalent

Hi,

Is there an equivalent command on slurm for the pbs command pbsdsh -u? That is to say, is there some command which will give one copy of a command to each node in a given allocation? I've combed through the documentation and there doesn't seem to be, but that struck me as odd that there wouldn't, so that's why I'm asking

Thank you,
Hartley

[Uwe Sauter]

You could also use pdsh (https://code.google.com/p/pdsh/)

Uwe

Post by Eckert, Phil
Hartley,
Sounds like you might be wanting srun.

salloc -p pdebug -N 5

salloc: Granted job allocation 1966117

srun hostname

rzmerl1
rzmerl2
rzmerl4
rzmerl3
rzmerl5
Phil Eckert
LLNL
Date: Monday, June 30, 2014 at 2:23 PM
Subject: [slurm-dev] pbsdsh -u equivalent
pbsdsh -u equivalent
Hi,
Is there an equivalent command on slurm for the pbs command pbsdsh
-u? That is to say, is there some command which will give one copy of
a command to each node in a given allocation? I've combed through the
documentation and there doesn't seem to be, but that struck me as odd
that there wouldn't, so that's why I'm asking
Thank you,
Hartley

[Hartley Greenwald]

Hi Phil,

Won't that just distribute the work among the nodes? I need a command that
will give one copy of the command to each node rather than distributing it.

Hartley

Post by Uwe Sauter
You could also use pdsh (https://code.google.com/p/pdsh/)
Uwe
Hartley,
Sounds like you might be wanting srun.

salloc -p pdebug -N 5

salloc: Granted job allocation 1966117

srun hostname

rzmerl1
rzmerl2
rzmerl4
rzmerl3
rzmerl5
Phil Eckert
LLNL
Date: Monday, June 30, 2014 at 2:23 PM
Subject: [slurm-dev] pbsdsh -u equivalent
Hi,
Is there an equivalent command on slurm for the pbs command pbsdsh -u?
That is to say, is there some command which will give one copy of a command
to each node in a given allocation? I've combed through the documentation
and there doesn't seem to be, but that struck me as odd that there
wouldn't, so that's why I'm asking
Thank you,
Hartley

[Gary Brown]

Hartley,
To clarify, you want to run a command on every node in the cluster, not
just on every node allocated to a job, correct?

Gary Brown

Post by Hartley Greenwald
Hi Phil,
Won't that just distribute the work among the nodes? I need a command
that will give one copy of the command to each node rather than
distributing it.
Hartley

Post by Uwe Sauter
You could also use pdsh (https://code.google.com/p/pdsh/)
Uwe
Hartley,
Sounds like you might be wanting srun.

salloc -p pdebug -N 5

salloc: Granted job allocation 1966117

srun hostname

rzmerl1
rzmerl2
rzmerl4
rzmerl3
rzmerl5
Phil Eckert
LLNL
Date: Monday, June 30, 2014 at 2:23 PM
Subject: [slurm-dev] pbsdsh -u equivalent
Hi,
Is there an equivalent command on slurm for the pbs command pbsdsh -u?
That is to say, is there some command which will give one copy of a command
to each node in a given allocation? I've combed through the documentation
and there doesn't seem to be, but that struck me as odd that there
wouldn't, so that's why I'm asking
Thank you,
Hartley

[Hartley Greenwald]

I may be wrong about this because I'm pretty new to all this stuff, but I
think that I want to give a copy to every node allocated for the job.

Post by Eckert, Phil
Hartley,
To clarify, you want to run a command on every node in the cluster, not
just on every node allocated to a job, correct?
Gary Brown

Post by Hartley Greenwald
Hi Phil,
Won't that just distribute the work among the nodes? I need a command
that will give one copy of the command to each node rather than
distributing it.
Hartley

Post by Uwe Sauter
You could also use pdsh (https://code.google.com/p/pdsh/)
Uwe
Hartley,
Sounds like you might be wanting srun.

salloc -p pdebug -N 5

salloc: Granted job allocation 1966117

srun hostname

rzmerl1
rzmerl2
rzmerl4
rzmerl3
rzmerl5
Phil Eckert
LLNL
Date: Monday, June 30, 2014 at 2:23 PM
Subject: [slurm-dev] pbsdsh -u equivalent
Hi,
Is there an equivalent command on slurm for the pbs command pbsdsh -u?
That is to say, is there some command which will give one copy of a command
to each node in a given allocation? I've combed through the documentation
and there doesn't seem to be, but that struck me as odd that there
wouldn't, so that's why I'm asking
Thank you,
Hartley

[Christopher Samuel]

Post by Hartley Greenwald
I may be wrong about this because I'm pretty new to all this stuff,
but I think that I want to give a copy to every node allocated for
the job.

To emulate pbsdsh you are quite correct.

According to the manual page the --ntasks-per-node=1 option for srun
should do what you want.

cheers,
Chris
- --
Christopher Samuel Senior Systems Administrator
VLSCI - Victorian Life Sciences Computation Initiative
Email: samuel-***@public.gmane.org Phone: [phone]
http://www.vlsci.org.au/ http://twitter.com/vlsci

[Hartley Greenwald]

I may be wrong about this, but doesn't this not necessarily solve the
problem?

Let's say we have one task and two nodes allocated. In PBS using pbsdsh
-u, both of the nodes will get a copy of the task. However, according to
the documentation --ntasks-per-node=1 only means that each node can get a
maximum of one task. This does not seem to entail that multiple copies of
the tasks will be produced and given to all the nodes only that a maximum
of one task is performed by each node.

Hartley

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Post by Hartley Greenwald
I may be wrong about this because I'm pretty new to all this stuff,
but I think that I want to give a copy to every node allocated for
the job.

To emulate pbsdsh you are quite correct.
According to the manual page the --ntasks-per-node=1 option for srun
should do what you want.
cheers,
Chris
- --
Christopher Samuel Senior Systems Administrator
VLSCI - Victorian Life Sciences Computation Initiative
http://www.vlsci.org.au/ http://twitter.com/vlsci
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iEYEARECAAYFAlOx+JUACgkQO2KABBYQAh/+uQCdHWQEQ/H+aJMJ8ppeMD+C/r88
jb0An2qJT4FZxloNNOqP2owAC2N3W7eZ
=7BJX
-----END PGP SIGNATURE-----

[Martin Perry]

I don't think Slurm will let you allocate more nodes than tasks. You need to request at least as many tasks as nodes. If you want to run exactly one copy of your command on each node in your allocation, just request the same number of tasks and nodes. Example:

[bones] (slurm) slurm> srun -N5 -n5 hostname
trek3
trek0
trek6
trek5
trek7



From: Hartley Greenwald [mailto:***@gmail.com]
Sent: Wednesday, July 02, 2014 12:31 PM
To: slurm-dev
Subject: [slurm-dev] Re: pbsdsh -u equivalent

I may be wrong about this, but doesn't this not necessarily solve the problem?
Let's say we have one task and two nodes allocated. In PBS using pbsdsh -u, both of the nodes will get a copy of the task. However, according to the documentation --ntasks-per-node=1 only means that each node can get a maximum of one task. This does not seem to entail that multiple copies of the tasks will be produced and given to all the nodes only that a maximum of one task is performed by each node.
Hartley

On Mon, Jun 30, 2014 at 6:04 PM, Christopher Samuel <***@unimelb.edu.au<mailto:***@unimelb.edu.au>> wrote:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Post by Hartley Greenwald
I may be wrong about this because I'm pretty new to all this stuff,
but I think that I want to give a copy to every node allocated for
the job.

To emulate pbsdsh you are quite correct.

According to the manual page the --ntasks-per-node=1 option for srun
should do what you want.

cheers,
Chris
- --
Christopher Samuel Senior Systems Administrator
VLSCI - Victorian Life Sciences Computation Initiative
Email: ***@unimelb.edu.au<mailto:***@unimelb.edu.au> Phone: [phone]<tel:%2B61%20%280%293%20903%2055545>
http://www.vlsci.org.au/ http://twitter.com/vlsci

[Christopher Samuel]

Post by Hartley Greenwald
Let's say we have one task and two nodes allocated.

Er, how are you going to do that?

$ sbatch --nodes=2 --ntasks=1 --wrap /bin/true
sbatch: Warning: can't run 1 processes on 2 nodes, setting nnodes to 1
Submitted batch job 1856638

A distributed job (MPI for instance) must have at least
one task on every node for this to make sense.

All the best,
Chris
- --
Christopher Samuel Senior Systems Administrator
VLSCI - Victorian Life Sciences Computation Initiative
Email: samuel-***@public.gmane.org Phone: [phone]
http://www.vlsci.org.au/ http://twitter.com/vlsci