Stories
Slash Boxes
Comments
NOTE: use Perl; is on undef hiatus. You can read content, but you can't post it. More info will be forthcoming forthcomingly.

All the Perl that's Practical to Extract and Report

use Perl Log In

Log In

[ Create a new account ]

agent (5836)

agent
  agentzh@yahoo.cn
http://agentzh.spaces.live.com/

Agent Zhang (章亦春) is a happy Yahoo! China guy who loves Perl more than anything else.

Journal of agent (5836)

Wednesday April 22, 2009
09:56 PM

SSH::Batch: Treating clusters as maths sets and intervals

[ #38851 ]
System administration is also part of my $work. Playing with a (big) bunch of  machines without a handy tool is painful. So I refactored some of our old scripts and released SSH::Batch, a collection of useful parallel ssh scripts, to CPAN:

    http://search.cpan.org/dist/SSH-Batch/

SSH::Batch allows you to name your clusters using variables and interval/set syntax in your ~/.fornodesrc config file. For instance:

    $ cat ~/.fornodesrc
    A=foo[01-03].com bar.org
    B=bar.org baz[a-b,d,e-g].cn foo02.com
    C={A} * {B}
    D={A} - {B}

where cluster C is the intersection set of cluster A and B while D is those machines in A but not in B.

And then you can query machine host list by using SSH::Batch's fornodes script:

   $ fornodes '{C}'
   bar.org
   foo02.com

   $ fornodes '{D}'
   foo01.com
   foo03.com

Furthermore, to run a command on a cluster by the concurrency of 6:

   atnodes 'ls -lh' '{A} + {B}' my.more.com -c 6

Or upload a local file to the remote cluster:

  tonodes ~/my.tar.gz '{A} / {B}' :/tmp/

There's also a key2nodes script to push SSH public keys to remote machines ;)

A colleague in Alibaba B2B is already using it. And one of my teammates is going to use it to operate on those thousands of machines in our instance of the YST (Yahoo! Search Technology) cluster and I'm ready to receive more feedback from him ;)

Have fun :)

P.S. This entry was originally posted to my own blog site as http://blog.agentzh.org/#post-105

The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
 Full
 Abbreviated
 Hidden
More | Login | Reply
Loading... please wait.