Skip to content

Commit

Permalink
Issue #1: Generalize ssh play, up forks default to 25.
Browse files Browse the repository at this point in the history
  • Loading branch information
geerlingguy committed Nov 15, 2022
1 parent 62987fe commit e227aa2
Show file tree
Hide file tree
Showing 5 changed files with 28 additions and 9 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
config.yml
hosts.ini
7 changes: 5 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,14 @@ Other OSes may need a few tweaks to work correctly. You can also run the playboo

## Benchmarking - Cluster

Make sure you have Ansible installed (`pip3 install ansible`), then set up a `hosts.ini` file in this directory based on the `example.hosts.ini` file.
Make sure you have Ansible installed (`pip3 install ansible`), then copy the following files:

- `cp example.hosts.ini hosts.ini`: This is an inventory of all the hosts in your cluster (or just a single computer).
- `cp example.config.yml config.yml`: This has some configuration options you may need to override, especially the `ssh_*` and `ram_in_gb` options (depending on your cluster layout)

Each host should be reachable via SSH using the username set in `ansible_user`. Other Ansible options can be set under `[cluster:vars]` to connect in more exotic clustering scenarios (e.g. via bastion/jump-host).

Tweak any settings inside `config.yml` as desired (the most important being `hpl_root`—this is where the compiled MPI, ATLAS, and HPL benchmarking code will live).
Tweak other settings inside `config.yml` as desired (the most important being `hpl_root`—this is where the compiled MPI, ATLAS, and HPL benchmarking code will live).

Then run the benchmarking playbook inside this directory:

Expand Down
1 change: 1 addition & 0 deletions ansible.cfg
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
[defaults]
nocows = true
forks = 25
inventory = hosts.ini
interpreter_python = /usr/bin/python3
stdout_callback = yaml
17 changes: 14 additions & 3 deletions config.yml → example.config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,25 @@
# Working directory where HPL and associated applications will be compiled.
hpl_root: /opt/top500

# Home directory of the user for whom SSH keys will be configured.
ssh_user: pi
ssh_user_home: /home/pi

# Use this option if running on a single node.
ram_in_gb: "{{ ( ansible_memtotal_mb / 1024 * 0.75 ) | int | abs }}"

# Use this option if running on multiple nodes in a cluster; calculate manually.
# ram_in_gb: "44"

# Count the nodes for accurate HPL.dat calculations.
nodecount: "{{ ansible_play_hosts | length | int }}"

# HPL.dat configuration options.
# See: https://www.advancedclustering.com/act_kb/tune-hpl-dat-file/
# See also: https://hpl-calculator.sourceforge.net/HPL-HowTo.pdf
nodecount: "{{ ansible_play_hosts | length | int }}"
ram_in_gb: "{{ ( ansible_memtotal_mb / 1024 * 0.75 ) | int | abs }}"
hpl_dat_opts:
# sqrt((Memory in GB * 1024 * 1024 * 1024 * Node count) / 8) * 0.9
Ns: "{{ (((((ram_in_gb | int) * 1024 * 1024 * 1024 * (nodecount|int)) / 8) | root) * 0.90) | int }}"
Ns: "{{ (((((ram_in_gb | int) * 1024 * 1024 * 1024 * (nodecount | int)) / 8) | root) * 0.90) | int }}"
NBs: 192
Ps: 2
Qs: 2
11 changes: 7 additions & 4 deletions main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,10 +31,13 @@
ansible.builtin.file:
path: "{{ item }}"
state: directory
owner: "{{ ansible_user }}"
group: "{{ ansible_user }}"
mode: 0755
loop:
- "{{ hpl_root }}/tmp"
- "{{ hpl_root }}/tmp/atlas-build"
become: true

- name: Download MPI (Message Passing Interface).
ansible.builtin.unarchive:
Expand Down Expand Up @@ -145,11 +148,11 @@
tasks:
- name: Generate an OpenSSH keypair.
community.crypto.openssh_keypair:
path: /home/pi/.ssh/id_rsa
path: "{{ ssh_user_home }}/.ssh/id_rsa"
size: 2048

- name: Read out ssh pubkey from each host.
ansible.builtin.command: cat /home/pi/.ssh/id_rsa.pub
ansible.builtin.command: cat "{{ ssh_user_home }}/.ssh/id_rsa.pub"
changed_when: false
register: ssh_pubkey

Expand All @@ -160,7 +163,7 @@

- name: Write all pubkeys to each host.
ansible.posix.authorized_key:
user: pi
user: "{{ ssh_user }}"
state: present
key: "{{ item }}"
loop: "{{ combined_ssh_pubkeys }}"
Expand All @@ -172,7 +175,7 @@

- name: Accept hostkeys for each host on each host.
ansible.builtin.command: >-
ssh pi@{{ item }} -o StrictHostKeyChecking=accept-new date
ssh {{ ssh_user }}@{{ item }} -o StrictHostKeyChecking=accept-new date
loop: "{{ host_ips }}"

- name: Run linpack benchmark.
Expand Down

0 comments on commit e227aa2

Please sign in to comment.