Raspberry Pi motion detection and notification program

PiDoorCam updated

This is the amended, latest version of my Raspberry Pi program to watch the street outside, detect change, and send a picture to another computer for storage, and to my phone. I’m often in the back garden, and can’t hear the doorbell; missing deliveries is rather annoying!

It no longer tries to take pictures when it’s dark, not just because I don’t have an infra-red camera and infra-red floodlights, but mainly because the street light outside flashes on and off all night, and I don’t want hundreds of pictures of that!

You’ll need to set up a Pushover account, which is free provided you don’t send too many notifications, and a Ramdisk with a directory called /var/tmp on the Pi.

Program listing

# Program for PiDoorCam

# Detects motion, and when it spots some, takes a high resolution
# picture, and sends the picture to another computer, also
# a notification via Pushover, to my phone.

import io
import os
import picamera
import ftplib
import time
import datetime
from PIL import Image
import requests
import json
import schedule

camera = picamera.PiCamera()
picamera.PiCamera.CAPTURE_TIMEOUT = 30

# If we detect 100 pixels that changed by 30, we have seen movement.
pixels = 100
difference = 30

# Use the maximum resolution of the camera.
# This is for V1. V2 is 3280 x 2464.
# It’s also correct for the ZeroCam.
width = 2592
height = 1944

# Internet lookup of sunrise and sunset at our location
def sunrise_sunset():
    global sunrise, sunset
    
    # Location of greenhouse is lat = yyyyy lon = xxxxx
    url = ‘https://api.sunrise-sunset.org/json?lat=yyyyy&lng=xxxxx’
    response = requests.get(url)
    dict = response.json()
    res = dict.get(‘results’)

    curtim = datetime.datetime.now()
    hm = res.get(‘sunrise’).split(“:”)
    sunrise = curtim.replace(hour=int(hm[0])-1, minute=int(hm[1]))
    print(“An hour before sunrise “,sunrise)
    
    hm = res.get(‘sunset’).split(“:”)
    sunset = curtim.replace(hour=int(hm[0])+13, minute=int(hm[1]))
    print(“An hour after sunset   “,sunset)

# I copied this voodoo motion detection from somewhere. Changed the timeout
#  setting above to prevent the occasional failures to complete captures.
# Only alter this if you know what you are doing!
def compare():
   camera.resolution = (100, 75)
   stream = io.BytesIO()
   format = ‘bmp’
   camera.capture(stream, format)
   stream.seek(0)
   im = Image.open(stream)
   buffer = im.load()
   stream.close()
   return im, buffer

# Function to take a new high resolution picture, send it to another computer,
# send it to my phone, and then delete it.
def newimage(width, height):
    when = datetime.datetime.now()
    filename = “door-%04d%02d%02d-%02d%02d%02d.jpg”
               % (when.year, when.month, when.day, when.hour, when.minute, when.second)
    camera.resolution = (width, height)
    camera.capture(“/var/tmp/”+filename)

    connected = True
    ftp = ftplib.FTP()
    ftp.connect(“computer-name”)
    
    try:
        ftp.login(“user-name”,”password”)
    except ftplib.all_errors:
        connected = False
        print (“Failed to login to server.”)
        ftp.quit()
        
    if connected:
        ftp.storbinary(‘STOR ‘+filename, open(“/var/tmp/”+filename, “rb”))
        print (“Sent to server “, filename)

    ftp.quit()

# Code to send the Pushover message. Make picture smaller first.
# Note this uses a Ramdisk you must set up elsewhere.
    im = Image.open(“/var/tmp/”+filename)
    im.resize((324,243),Image.ANTIALIAS)
    im.save(“/var/tmp/”+filename)
    
    r = requests.post(“https://api.pushover.net/1/messages.json”, data = {
        “token”: “you-need-to-get-a-token-from-pushover”,
        “user”: “you-need-to-get-a-user-name-from-pushover”,
        “device”: “your-device”,
        “sound”: “intermission”,
        “message”: filename
    },
    files = {
        “attachment”: (filename, open(“/var/tmp/”+filename, “rb”), “image/jpeg”)
    })
   # Check r for problems – maybe put a delay here?
    if r.status_code != 200:
        print(“Pushover message failed.”)
    else:
        print(“Pushover accepted the message.”)
         
# Now delete the file.
    os.remove(“/var/tmp/”+filename)
    # Delay to avoid being nasty to Pushover server.
    time.sleep(5)

# Main program.

camera.rotation = 0
print(“Running door.py”)
image1, buffer1 = compare()

# Find sunrise and sunset times at two in the morning, and once
# at startup.
schedule.every().day.at(“02:00”).do(sunrise_sunset)
sunrise_sunset()
while (True):
   # See if it’s time to get sunrise and sunset.
   schedule.run_pending()

   image2, buffer2 = compare()

   changedpixels = 0
   for x in range(0, 100):
      for y in range(0, 75):
         pixdiff = abs(buffer1[x,y][1] – buffer2[x,y][1])
         if pixdiff > difference:
            changedpixels += 1

   # See if we think something moved.
   if changedpixels > pixels:
   # See if it’s light enough to take a picture.
      now = datetime.datetime.now()
      if now > sunrise and now < sunset:
          newimage(width, height)
      else:
          print(“A bit dark at “,now)

   image1 = image2
   buffer1 = buffer2

Greenhouse computer ravings continued.

Fat man in the greenhouse.
In the end, I got fed up with the temperate readings being messed up by the Sense HAT being inside the case, and no cooling fan being able to keep its sensors cool enough. I dispensed with the Sense HAT, which I will find some other use for, and put a cheap temperature and humidity sensor on wires lead out of the case.
Having arranged a shade to keep direct sunlight off the sensor, I now get sensible readings. I missed being able to look at the greenhouse and see the temperature scroll past, so I had the computer report temperature and humidity to another Pi indoors. 
Simple web site

That computer runs the Apache web server, and uses the incoming readings to make a simple web page, which I can look at from my main computer…






Code…

textA = [“<!doctype html>n”,”<html lang=”en”>n”,”<head>n”,”<meta charset=”utf-8″>n”,
         “<title>Greenhouse</title>”,”<link rel=”stylesheet” href=”mystyle.css”>”,
         “<meta http-equiv=”refresh” content=”150″ >”,
         “</head>”,”<body>”,”<h1>Greenhouse</h1>”,”<p>”]
textZ = [“</body>n”,”</html>n”]
def update_greenhouse_website():
    global previous_time, greenhouse_temp_file_modified
    greenhouse_temp_file_modified = os.stat(‘/home/pi/ftp/files/gth.txt’).st_mtime
    if greenhouse_temp_file_modified == previous_time:
        #print(‘Same time’)
        textM = “No greenhouse data received.nCheck greenhouse computer!”
    else:
        #print(‘New time’)
        fp = open(‘/home/pi/ftp/files/gth.txt’, ‘r’)
        textM = fp.read()
        fp.close()
    
    fp = open(‘/var/www/html/greenhouse.html’, ‘w’)
    for line in textA:
        fp.write(line)
    fp.write(textM)
    fp.write(“n”)
    for line in textZ:
        fp.write(line)
    fp.close()
    
    previous_time = greenhouse_temp_file_modified  
    return

Weather Station woes….

If you were making a weather station, you would try to make it waterproof. But you have to try harder, Fine Offset of China. Your WH1040 dies when the rain gets in…




Rain may only fall vertically in China, perhaps, so the marvellously cheap weather station I got from Maplin, just before they mysteriously went bust, would do there. But this is Wales, and the wind blows up the hill, carrying rain upwards with it, and it gets into the electronics and stops it working. It should look like this on the computer, although those maximum wind speeds are just plain wrong, thanks to EasyWeather corrupting its database.





I have a spare control unit, found on eBay, and wanted to swap them round, but the clever cover that keeps rain from above off somehow welded itself to the control unit. I removed it with BF&I.





Data is coming through! And it has even uploaded to Weather Underground!

And, now, having typed that, I look again, and it has stopped working.


Oh, well… Hang on, it’s back! Electronics, don’t you just love them?

Greenhouse computer, part the third.

Click for bigger!

































Here’s the greenhouse computer in situ. The air from the fan is now directed across the sensors on the Sense HAT, with the intention of measuring the temperature of the air in the green house, rather than the air in the case that the Pi 3 has warmed up. It’s just a balsawood duct, as thats an easy material to cut and glue.

The small case above the main one contains a Pi compatible camera with a fish eye lens and two infra red lights, for night photography.


The program it runs is fairly long, and specific to this hardware combination, but I can add it if anyone asks…

Greenhouse Computer Version 2

Work in progress…

























Well, the previous greenhouse computer could report the temperature, barometic pressure, and humidity, but it didn’t control anything, and the electric fan heater’s thermostat wasted a lot of electricity during the previous winter, as it wasn’t accurate, and the greenhouse got warmer than necessary.


The greenhouse needs to be kept above 5°C, to keep the citrus plants that over-winter in it alive, but not much warmer, and the fan heater doesn’t have any decent form of regulation.

Anyway… Here comes Version 2 of the greenhouse computer. The three boards are, starting at the bottom of the stack,

  • A Raspberry Pi 3B+
  • A Relay card with four mains relays
  • and a Pi Sense HAT.
The problem you get with putting these units in a case is that the Pi warms up the air in the case, so the Sense HAT gives false temperature readings. In Version 1 of the computer, there was a fan blowing air from outside the case onto the HAT’s sensors, and that’s what I’m doing in Version 2. The fan has just gone in, but I still need to put trunking to direct its air stream over those sensors.

You can see a nice long camera lead on the left, so the camera can go in a separate case. The mains wiring feeds mains to a couple of sockets on the back of the case, one for the heater, and one spare. The relays switch the sockets on and off.


More to follow…

Recent python door cam code – ugly, but works.

Updated version of the code…
It deals with the occasional hangs in the camera capture process by
rebooting when they happen. A line in /etc/rc.local runs the program again.

# Program for PiDoorCam

# Detects motion, and when it spots some, takes a high resolution
# picture, and sends the picture to PiScreen

import io
import os
import picamera
import ftplib
import time
from datetime import datetime
from PIL import Image
import requests

camera = picamera.PiCamera()
picamera.PiCamera.CAPTURE_TIMEOUT = 30

# If we detect 100 pixels that changed by 30, we have seen movement.
#difference = 30
#pixels = 100
# Desensitise!
#difference = 30
difference = 50
pixels = 150

# May as well use the maximum resolution of the camera.
# This is for V1. V2 is 3280 x 2464.
width = 2592
height = 1944

# I copied this voodoo motion detection from somewhere. Changed the timeout
#  setting above to prevent the occasional failures to complete captures.
def compare():
   camera.resolution = (100, 75)
   stream = io.BytesIO()
   format = ‘bmp’
   # Handle occasional ‘Timed out waiting for capture to end’
   try:
      camera.capture(stream, format)
   except:
      print (“Camera timed out, reboot needed!”)
      os.system(“sudo reboot now”)
       
   stream.seek(0)
   im = Image.open(stream)
   buffer = im.load()
   stream.close()
   return im, buffer

# Function to take a new high resolution picture, send it to PiScreen
# send it to my phone, and then delete it.
def newimage(width, height):
    when = datetime.now()
    filename = “door-%04d%02d%02d-%02d%02d%02d.jpg” % (when.year, when.month, when.day, when.hour, when.minute, when.second)
    camera.resolution = (width, height)
    camera.capture(filename)

    connected = True
    ftp = ftplib.FTP()
    ftp.connect(“PiScreen”)
   
    try:
        ftp.login(“pi”,”******************”)
    except ftplib.all_errors:
        connected = False
        print (“Failed to connect to server %s” % e)   
       
    if connected:
        ftp.storbinary(‘STOR ‘+filename, open(filename, “rb”))
        print (“Sent “, filename)

    ftp.quit()

# Code to send the Pushover message. Make picture smaller first.
    im = Image.open(filename)
    im.resize((324,243),Image.ANTIALIAS)
    im.save(filename)
   
    r = requests.post(“https://api.pushover.net/1/messages.json”, data = {
        “token”: “***********************************”,
        “user”: “*************************************”,
        “device”: “***********”,
        “sound”: “intermission”,
        “message”: filename
    },
    files = {
        “attachment”: (filename, open(filename, “rb”), “image/jpeg”)
    })
# Check r for problems – maybe put a delay here?
    if r.status_code != 200:
        print(“Pushover message failed.”)
    else:
        print(“Pushover accepted the message.”)
       
# Now delete the file.
    os.remove(filename)
    # Delay to avoid being nasty to Pushover server.
    time.sleep(5)

# Main program.

camera.rotation = 180
print(“Running door.py”)
image1, buffer1 = compare()

newimage(width, height)

while (True):

   image2, buffer2 = compare()

   changedpixels = 0
   for x in range(0, 100):
      for y in range(0, 75):
         pixdiff = abs(buffer1[x,y][1] – buffer2[x,y][1])
         if pixdiff > difference:
            changedpixels += 1

   if changedpixels > pixels:
      newimage(width, height)

   image1 = image2
   buffer1 = buffer2



Code below is an older version, the “error handling” doesn’t actually work.

# Program for PiDoorCam

# Detects motion, and when it spots some, takes a high resolution
# picture, and sends the picture to PiScreen

import io
import os
import picamera
import ftplib
import time
from datetime import datetime
from PIL import Image
import requests

camera = picamera.PiCamera()
picamera.PiCamera.CAPTURE_TIMEOUT = 30

# If we detect 100 pixels that changed by 30, we have seen movement.
#difference = 30
#pixels = 100
# Desensitise!
difference = 30
pixels = 150

# May as well use the maximum resolution of the camera.
# This is for V1. V2 is 3280 x 2464.
width = 2592
height = 1944

# I copied this voodoo motion detection from somewhere. Changed the timeout
#  setting above to prevent the occasional failures to complete captures.
def compare():
   camera.resolution = (100, 75)
   stream = io.BytesIO()
   format = ‘bmp’
   # Handle occasional ‘Timed out waiting for capture to end’
   try:
      camera.capture(stream, format)
   except:
      print (“Retrying camera.capture()”)
      camera.capture(stream, format)
 
   stream.seek(0)
   im = Image.open(stream)
   buffer = im.load()
   stream.close()
   return im, buffer

# Function to take a new high resolution picture, send it to PiScreen
# send it to my phone, and then delete it.
def newimage(width, height):
    when = datetime.now()
    filename = “door-%04d%02d%02d-%02d%02d%02d.jpg” % (when.year, when.month, when.day, when.hour, when.minute, when.second)
    camera.resolution = (width, height)
    camera.capture(filename)

    connected = True
    ftp = ftplib.FTP()
    ftp.connect(“PiScreen”)
 
    try:
        ftp.login(“pi”,”**********************************”)
    except ftplib.all_errors:
        connected = False
        print (“Failed to connect to server %s” % e) 
     
    if connected:
        ftp.storbinary(‘STOR ‘+filename, open(filename, “rb”))
        print (“Sent “, filename)

    ftp.quit()

# Code to send the Pushover message. Make picture smaller first.
    im = Image.open(filename)
#    im.resize((648,486),Image.ANTIALIAS)
    im.resize((324,243),Image.ANTIALIAS)
    im.save(filename)
 
    r = requests.post(“https://api.pushover.net/1/messages.json”, data = {
        “token”: “**********************************”,
        “user”: “********************************”,
        “device”: “************”,
        “sound”: “intermission”,
        “message”: filename
    },
    files = {
#        “attachment”: (“image.jpg”, open(filename, “rb”), “image/jpeg”)
        “attachment”: (filename, open(filename, “rb”), “image/jpeg”)
    })
# Check r for problems – maybe put a delay here?
    if r.status_code != 200:
        print(“Pushover message failed.”)
    else:
        print(“Pushover accepted the message.”)
       
# Now delete the file.
    os.remove(filename)
    # Delay to avoid being nasty to Pushover server.
    time.sleep(5)

# Main program.

camera.rotation = 180
print(“Running door.py”)
image1, buffer1 = compare()

newimage(width, height)

while (True):

   image2, buffer2 = compare()

   changedpixels = 0
   for x in range(0, 100):
      for y in range(0, 75):
         pixdiff = abs(buffer1[x,y][1] – buffer2[x,y][1])
         if pixdiff > difference:
            changedpixels += 1

   if changedpixels > pixels:
      newimage(width, height)

   image1 = image2
   buffer1 = buffer2

The Wisdom of the Ancients – Part 94

I tried to find this one one Google, but there was no sign of it!


mpirun mca_oob_tcp_recv_handler  invalid message type: 43


Basically, it means you have not noticed you are trying to control a cluster of Raspberry Pi 3 B+ computers that are running Raspbian Stretch with a Raspberry Pi that has somehow still got Raspbian Jessie on it.


You’re welcome!

Oyster, before I got all the bricks.



Marmalade


Marmalade.
I was never all that happy with the previous batch of marmalade I made in 2014. I thought it had been boiled too much, as it was quite a dark colour, and recently, I noticed that what I thought was the last jar was getting quite low.* There are only a couple of weeks of the year when the right oranges for marmalade making are in the shops, and last week I found some in Tesco. I grabbed the last Kilo.



In case you don’t know, proper marmalade is made with Seville oranges, named after the part of Spain where they are grown. These oranges are a bitter variety, full of pips. You wouldn’t want to eat them, or drink the juice. If you use ordinary oranges, you will end up with some sort of orange jam, that may well be very pleasant, but it won’t be marmalade.


If you are not near enough to Spain to have them in the shops, and want to make marmalade, look for something too bitter to enjoy, with unbelievable numbers of pips. The pips are more important than you might think, as they are full of the pectin that makes the marmalade set. 

As well as a Kilo of oranges, you will need two Kilos of granulated sugar, and two lemons. I don’t know why Tesco is only stocking foreign sugar, but local shops like Spar and the Co-op do have British sugar, made from beet. [It may have something to do with a certain government minister who used to be high up in that foreign sugar company.]




Ingredients.
1 Kg Seville oranges
2 lemons
2 Kg granulated sugar
500 ml water

These are Seville oranges; look at those pips!
















The method. 
You will need a pressure cooker for this recipe. There are other recipes involving boiling things for hours, and I’m sure they work pretty much as well as this one does, but we have a pressure cooker, and it saves quite a lot of time. Cut the oranges in half, squeeze them with one of those glass juice squeezing things, which I completely forgot to take a picture of. Put the juice, and the peels in the pressure cooker. Add the juice of two lemons, but not their peels. The lemon pips can go in with the orange pips.

I used a plastic strainer to stop the pips going in. There tend to be pips still hiding in the peels, but that can be sorted out at a later stage. The pips get wrapped in a nice open weave cloth, such as cheese-cloth, muslin, or whatever you have handy that seems reasonable to use in cooking. Now add 500 ml of water. The picture on the left is the contents of the pressure cooker before boiling.






And here’s a picture of the contents of the pressure cooker after ten minutes of boiling at full pressure, followed by allowing them to cool naturally to room temperature. Notice that the pith of the oranges is cooked, and very much softer than before.




Transfer the juices to your preserving pan. It’s nice if you have a big copper plated pan for this, but we use a big old Teflon saucepan. Hooked onto the side, is our old sugar thermometer.

Now you need to squeeze the pips into the pan, until they… no, just squeeze them until you don’t think you will get any more out of them. Squeezing pips until they squeak turns out to be really difficult. Only politicians can do it. 
Throw the bag of pips away once you have got as much as you can from it. The pips supply pectin, which is what makes the marmalade set nicely.
Now, chop up the peels to your preferred size chunks. Some people like very fine pieces of peel, while I quite like big chunks. I’ve cut these ones to a medium size, as my wife prefers them small. Notice that this is when you remove the pips that have been cunningly hiding in the peels. 

Put the chopped up peels in the pan with the juice, and bring them to the boil.

Tip in the two Kilograms of sugar, and stir until it is properly dissolved. Keep heating, and keep stirring. You need to raise the temperature to 105°C.
Warning! Hot, concentrated sugar solution holds much more heat than mere boiling water, and if you splash this on yourself it will burn you badly.

That old sugar thermometer is no longer doing its job properly! It was showing something a bit below 105°C, but I thought the marmalade was looking ready, so I checked it with a cheap electronic thermometer, and as you can see, the marmalade was done! If I had heated it until the old thermometer said it was done, I would have had another batch of over-boiled marmalade.


There are all sorts of ways to test whether your marmalade is going to set, including cold saucers in the fridge, with a splash of marmalade on, but I don’t think this recipe can avoid setting, if you follow it properly, and make sure you get it to the magic 105°C.

All that remains to do is put it in clean jars. Dishwashers are the best way to clean jam jars. If you don’t have one, you’ll have to boil them up in some suitable manner. Below, you see my results. It’s very much better looking, and tasting, than the previous batch.


* I found another jar of the old, dark stuff. I threw the contents away.

Online webcam project, part 2.

Trigger warning.

Stop! Don’t look at the following photographs if you are offended by ugly applications of hot glue.

I found what I thought would be the ideal case for the camera and Pi in the kitchen. It was one of those Tesco plastic food boxes with clips and a watertight seal, and it had nasty cracks in the base in just the right spot to cut a hole for the lens. Using a variety of inappropriate tools, I made the necessary hole, and fixed the lens into it with hot glue.

Ugly hot glue!

As I warned you, very ugly hot glue. Note that it covers the join between the two parts of the lens, which I hope will ensure our good Welsh rain can’t get in between the lens elements. The next steps were to screw the inner lens cap in place, and mount the Raspberry Pi Zero inside the box. I was going to put it in the same part of the box as the lens and camera, but every arrangement I tried had the USB wireless antenna rather close to the camera and its cable. So, I mounted the Pi Zero in the lid of the box with a goodly lump of White Tac.

img_20160919_144721

A notch in the edge of the box, for the power cable, was the next thing. It’s at the bottom, and only a very small percentage of the rain around here falls upwards, so it may be OK without any sealant. I have more White Tac if there is a problem…

Software

Getting the camera to take a picture every ten minutes is not a problem, thanks to cron and the bash shell.

The intention is to use lftp to upload the pictures to the web host, but finding a decent example of code that will do that is not proving easy. Please feel free to comment if you have something suitable.

I shouldn’t have too much difficulty hacking out a web page for the picture to live in, as I am not planning anything fancy, so I’ll just use HTML, like the rest of my site.

Online webcam project, part 1.

Having decided to share the view at the back of our house with the rest of the world, I’ve finally got started. After all, it’s a pretty amazing view at times. Here’s what it was like recently. Even when it’s blurred, it’s good!

img_3650

I’m using a standard Raspberry Pi camera, and a Pi Zero to do this. Here’s the camera. I took a cheap Vivitar wide angle converter, made a hole in the rear lens cap, and glued the camera’s mounting kit to it, using a cheap glue gun.

img_20160919_103757

Here’s the current state of the thing.

img_20160919_103847

I connected to the PI with x11vnc, and took a test shot. The result is nicely in focus, thanks to the cunning design of the adapter lens, and gives a good wide angle view of the study.

test

Next, I need to set up the Raspberry Pi to upload pictures to my web site every few minutes, make a page on the web site for people to gawp at, and put the Pi in a waterproof case.