parallel processing - Using maximum remote servers -
im trying distribute commands 100 remote computers, noticed commands being sent 16 remote computers. local machine has 16 cores. why parallel using 16 remote computers instead of 100?
parallel --eta --sshloginfile list_of_100_remote_computers.txt < list_of_commands.txt
i believe need specify number of parallel jobs executed.
according parallel man:
--jobs n
-j n
--max-procs n
-p n
number of jobslots. run n jobs in parallel. 0 means many possible. default 100% run 1 job per cpu core.
and keep in mind:
when start more 1 job -j option, reasonable assume each job might not take same amount of time complete. if care seeing output in order file names presented parallel (instead of when completed), use --keeporder option.
parallel multicore @ command line gnu parallel, admin magazine
Comments
Post a Comment