I now have a GUI program that runs on Marvin, which takes a program developed on Marvin, deploys it on the four Pi’s that make up the oyster cluster, then uses mpirun to run it in parallel on 16 cores, with the results appearing on Marvin.
It’s clearly time to knock off and celebrate…
I’m just testing my Raspberry Pi cluster, to see if I have sorted out the setup properly this time. Finding the primes up to 10,000 with one core, and then sixteen cores, followed by using 16 cores to find primes up to 100,000 gave these results…
pi@oyster0:~ $ mpirun -hostfile myhostfile -np 1 python3 Programs/prime.py 10000
Find all primes up to: 10000
Nodes: 1
Time elapsed: 4.34 seconds
pi@oyster0:~ $ mpirun -hostfile myhostfile -np 16 python3 Programs/prime.py 10000
Find all primes up to: 10000
Nodes: 16
Time elapsed: 0.34 seconds
pi@oyster0:~ $ mpirun -hostfile myhostfile -np 16 python3 Programs/prime.py 100000
Find all primes up to: 100000
Nodes: 16
Time elapsed: 23.78 seconds
So, it is all working as it should now. Next step is to add blinkenlights on the supervising machine, Marvin, which has a UnicornHD HAT. After that, I want to get my GUI based supervisor working.
This leaves far too little time to flog stuff on eBay! I shall have to write a program to do that…
Update:
I ran it for the primes under a million, and it was disturbingly slow. I’d hope for something less than ten times as long as for a hundred thousand, but no!
pi@oyster0:~ $ mpirun -hostfile myhostfile -np 16 python3 Programs/prime.py 1000000
Find all primes up to: 1000000
Nodes: 16
Time elapsed: 2279.37 seconds
Almost ten minutes. I’m assuming things ended up swapping memory in and out, or Python doesn’t handle big integers very well. It’s not a problem, but it is one of the reasons I want blinkenlights…
For a while, I have been turning the camera on the Pi in the greenhouse on and off manually. By that I mean…
That’s clearly a huge faff, so I decided I’d finally make a GUI based program to do the job, using guizero. Other GUI libraries were either immensely complex or mysteriously impossible to install. I looked at a couple of example programs, and cobbled this together…
import paramiko
from guizero import App, Text, PushButton
def switch_camera_on():
ssh_stdin, ssh_stdout, ssh_stderr = ssh.exec_command("sudo rm /var/tmp/blind")
def switch_camera_off():
ssh_stdin, ssh_stdout, ssh_stderr = ssh.exec_command("touch /var/tmp/blind")
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect("server-name-here", username = "user-name-here", password = "password-here")
app = App(title="Greenhouse Camera Control", bg = "lightblue", width = 400, height = 200)
welcome_message = Text(app, text = "Click button to switch camera state.", size = 15)
onswitch = PushButton(app, command = switch_camera_on, text = "Camera on.")
offswitch = PushButton(app, command = switch_camera_off, text = "Camera off.")
app.display()It does need you to have used ssh-keygen on your systems, so they can communicate. I can’t believe how easy simple stuff like this is to hack out!
My Raspberry Pi cluster, named “Oyster” because of something to do with the Walrus and the Carpenter, had been out of commission for months, so I eventually got to work and tried to set it up from scratch, as an alternative to checking everything over and over again, and failing to find anything wrong. Oyster used to look like this, but the Pi 3s in Lego compatible cases ran too hot.
At first, I attempted to use my only Pi 4, “Marvin”, as the control machine for the four Pi 3s in Oyster, but I got something wrong in the setup, and it didn’t work. I had been thinking 20 cores would obviously be better than 16, but suspected the two different user names in use might be causing the problem. It probably wasn’t, as I now think the ssh communication for the cluster is done anonymously. Possibly. Anyway, I changed my mind about 20 cores, when I thought about the other tasks Marvin runs, and how I didn’t want them slowed down. Sixteen will do. Unless I get some more Pi 3s and add them to the cluster…
Anyway, I went through all the setup described in Ashwin Pajankar’s e-book* about Raspberry Pi “supercomputers”, again. Twice. I found some online guides, and checked the setup with those, too. I could run programs on the four cores of oyster0, but not on the other three Pis. Eventually, I spotted that I had forgotten to create the file “known-hosts” on the controlling Pi. The message passing software, python3-mpi4py, would probably have told me this, if I had run it in verbose mode, but I didn’t. Still, it now works, and I have a sixteen core Pi cluster running.
I have added a program, running on Marvin, that lights up LEDs to show the state of the Pis in the cluster, and intend to add further blinkenlights to show the activity of each of the sixteen cores.
# Program to monitor status of Oyster cluster, displaying up/down indication.
#import os
import time
import subprocess as sp
machines = ["oyster0", "oyster1", "oyster2", "oyster3"]
while True:
for i in range(len(machines)):
machine = machines[i]
state = sp.call(['ping', '-c', '1', machine], stdout=sp.DEVNULL)
if state == 0:
colour = "0 200 0" # Green means UP
else:
colour = "200 0 0" # Red means DOWN
row = "12 "
if machine == "oyster0":
column = "0 "
elif machine == "oyster1":
column = "4 "
elif machine == "oyster2":
column = "8 "
elif machine == "oyster3":
column = "12 "
message = "set_pixel " + row + column + colour
fp = open("/home/chris/ftp/files/blinkenlights" + machine, 'w')
fp.write(message)
fp.close()
time.sleep(1)This code sends a text file to my Unicorn HD server program, which enables more than one program to write to its 256 LEDs, without messing up each other’s displays.
My plans for further development include improving my set of framework code for running parallel programs on the cluster, and more (or possibly less) importantly, to send messages to Marvin for the blinkenlights.
[* I haven’t included a link to AP’s e-book, because it’s easy to find online, and he is charging far too much for it, while others are giving us the same information free.]