Preventing useless multiple disk writes

[Steffenwolt]

Hi there,
I have this program (datalog_v26) that reads sensors and writes the readings to disk.

#!/usr/bin/env python

#======================================================================
# Program name: datalog_v26
# Versie 2.6; 20/07/2018
# Reads the sensors and stores them on HD.
# Controls bilge pumps.
# Uses function pumpcontrol and key_check
#
#
#
#
#======================================================================

      #Importeer benodigde libraries.
import sys
import time
import csv
import os
import spidev
import RPi.GPIO as GPIO
import I2C_LCD_driver_twoLCD
from key_check import status
from pumpcontrol import pumpcontrol

#    Set variabelen en GPIO pins.
mylcd1 = I2C_LCD_driver_twoLCD.lcd(0x26)
mylcd2 = I2C_LCD_driver_twoLCD.lcd(0x27)

spi = spidev.SpiDev()
spi.open(0,0)

GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)
GPIO.setup(23,GPIO.OUT)
GPIO.setup(24,GPIO.OUT)
GPIO.setup(25,GPIO.OUT)

#     Function reads mcp3008 adc. 
def read_MCP(channel):
        adc = spi.xfer2([1,(8+channel)<<4,0],20000)
        adcread = ((adc[1]&3) << 8) + adc[2]
        return adcread

#     Function to read multiple ds18b20 sensors on a 1wire bus on gpio 4
def read_ds18b20(sensorid):     
        device = "/sys/bus/w1/devices/"+ sensorid +"/w1_slave"
        try:
            with open (device) as tfile:
                text = tfile.read()
        except FileNotFoundError:
            GPIO.output(25,True)                    # Turn on GPIO25 with FileNotFound LED to indicate ID not present.
            notemp = 9999
            return notemp
        
#    Format the reading from sensor
        sensor_data =text.split(" ")[20] 
        temp_raw = float(sensor_data[2:]) 
        temp = temp_raw / 1000
        digitemp = '%.1f' % temp
#        print ("digitemp = " + digitemp)
        return digitemp
    
#    Writes data to disk
def write_to_disk(writedata):
    csvfile = "/media/pi/Seagate Portable/testdata.csv"
    with open (csvfile, 'a') as output:
        writer = csv.writer(output, delimiter = ',' ,lineterminator = '\n')
        writer.writerow(writedata)

GPIO.output(25,False)                                          # Reset FileNotFound LED

while True:
    
    datum = (time.strftime('%d/%m/%Y'))
    tijd = (time.strftime('%H:%M:%S'))
    
    while True:

        datum = (time.strftime('%d/%m/%Y'))
        tijd = (time.strftime('%H:%M:%S'))


#        time.sleep(1)

#    Checks the stus of the key switch.
        if status()== 'ON':                                      # Key switch is on
            break
        else:                                                    #  Key switch is off
            print ('Schakelaar is UIT')

#    Checks water levels when key switch is off.
        threshold = 500
        boord = 'sb'
        pumpsb = pumpcontrol(threshold,boord)    # Call pumpcontrol; return status of pump as pumpstat sb
        list = pumpsb
#        print (list[1])
        print (pumpsb)

        threshold = 500
        boord = 'bb'
        pumpbb = pumpcontrol(threshold,boord)    # Call pumpcontrol; return status of pump as pumpstat bb
        print (pumpbb)

        time.sleep(2)

#    Write pump status to file on HD.
        data = (pumpsb, pumpbb)#        write_to_disk(data)
                
#   Zet de pomp SB aan of uit door de MCP0 waarde te vergelijken met de waarde.
    threshold = 500
    boord = 'sb'
    pumpsb = pumpcontrol(threshold,boord)         # Call pumpcontrol; return status of sb pump as pumpstat sb
    print (pumpsb)

    threshold = 250
    boord = 'bb'
    pumpbb = pumpcontrol(threshold,boord)         # Call pumpcontrol; return status of bb pumps as pumpstat bb
    print (pumpbb)

#    Write pump status data to HD.
#    data = (pumpsb,pumpbb)
#    write_to_disk(data)

#    Read ds18b20 sensors with function 'read_ds18b20'.
    temp1 = float(read_ds18b20('28-000009adc801'))      # Motor 1
    temp2 = float(read_ds18b20('28-000009adc801'))      # Motor 2
    temp3 = float(read_ds18b20('28-000009adc801'))      # Future expansion

#        REMARK xxxadc801 is tijdelijk de enige digitale sensor.

#    Read the mcp channels with the function 'read_MCP'.
    lijst = []
    for offset in range(0,8):
        adcread = read_MCP(offset)
#        print (adcread)
        lijst.append(adcread)
        
    data = [temp1,temp2,temp3,round(lijst[0],1),round(lijst[1],1),
            round((((lijst[2]*3.3/1024)-0.52)/0.08),1),
            round((((lijst[3]*3.3/1024)-0.52)/0.08),1),
            round(((lijst[4]*330/1024)-50),1)]

#=========================================================
#
#    'temp1' is temparatuur motor SB.
#    'temp2' is temperatuur motor BB
#    'temp3' is not used
#    'lijst[0]' is watersensor SB.
#    'lijst[1]' is watersensor BB.
#    'lijst[2]' is accustroom SB.
#    'lijst[3]' is accustroom BB.
#    'lijst[4]' is buiten temperatuur met TMP36.
#
#=========================================================

#    blinks temp if > limit
    
#    if  int(temp1) > 25:
#    for i in range(1,5):
#        mylcd1.lcd_display_string('BB motor='+str(temp1)+chr(223)+'C',1)
#        time.sleep(1)
#        mylcd1.lcd_clear()
#        time.sleep(1)
            
    mylcd2.lcd_display_string('BB motor='+str(temp1)+chr(223)+'C',1)
    mylcd1.lcd_display_string('SB motor='+str(temp2)+chr(223)+'C',1)
    mylcd1.lcd_display_string('Outside='+str(round((lijst[4]*330/1024)-50,1))+chr(223)+'C',3)

#    print (data)

#    Write sensor data to HD

    logdata = (datum, tijd, data)
    print (logdata)
    write_to_disk(logdata)
        
    time.sleep(1)
   

It also calls "pumpcontrol" to check two water level sensors (bb, port and sb, starboard) and turn on/off two bilge pumps. Pumpcontrol also writes the status of the pumps to disk.
Like the calling program datalog_v26, its runs every few seconds resulting every time in an entry on disk that the pump is (still) on or off which is somewhat useless as i only want to see the transitions.
So in pumpcontrol i added lines 74-76 and 83-85.

#!/usr/bin/env python

#===================================================================================
# Function name = pumpcontrol
# Version 1.0, date 15/07/2018
# parameters are 'boord' bb or sb and 'threshold' 
# returns 'pumpstat' pump on or off
# Called from datalog.py
# Checks waterlevels SB and BB named reading and compares to threshold from calling
# program.
# Turn on appropriate pump and displays status message.
#
#===================================================================================

      # Import requiredf libraries.
import sys
import time
import csv
import os
import spidev
import RPi.GPIO as GPIO
import I2C_LCD_driver_twoLCD

   #  Set variables and GPIO pins.

spi = spidev.SpiDev()
spi.open(0,0)

GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)
GPIO.setup(23,GPIO.OUT)
GPIO.setup(24,GPIO.OUT)

datum = (time.strftime('%d/%m/%Y'))
tijd = (time.strftime('%H:%M:%S'))

#    Writes data to disk
def write_to_disk(writedata):
    csvfile = "/media/pi/Seagate Portable/testdata.csv"
    with open (csvfile, 'a') as output:
        writer = csv.writer(output, delimiter = ',' ,lineterminator = '\n')
        writer.writerow(writedata)

#    Function reads MCP3008 channel 'sensorid'.
def read_mcp(mcpch):
        adc = spi.xfer2([1,(8+mcpch)<<4,0],20000)
        adcread = ((adc[1]&3) << 8) + adc[2]
#        print ('adcread=',adcread)
        return adcread

def pumpcontrol(threshold,boord):
    if (boord == 'sb'):
        mcpch = 0
        gpio = 23
        displ = I2C_LCD_driver_twoLCD.lcd(0x26)
    elif (boord == 'bb'):
        mcpch = 1
        gpio = 24
        displ = I2C_LCD_driver_twoLCD.lcd(0x27)
    else:
        pumpstat = False
        return 'No valid boord'
    
# Check water sensor SB/BB, channel 0/1 from ADC.         
    reading = read_mcp(mcpch)

# Compare 'threshold' with adc reading and turn pump on/off
    if (threshold < reading):
            GPIO.output(gpio,GPIO.HIGH)         #    pump relay SB/BB ON
            displ.lcd_display_string(boord+' pomp actief',4)
            pumpstat = (boord+ ' pomp ','ON')
            print (pumpstat[1])
            logdata = (datum, tijd, pumpstat)
            if prevpumpstat == 'OFF':
                write_to_disk(logdata)
                prevpumpstat = 'ON'
            
    else:
            GPIO.output(gpio,GPIO.LOW)          #     pomp relay SB/BB OFF
            pumpstat = (boord+ ' pomp ','OFF')
            print (pumpstat[1])
            logdata = (datum, tijd, pumpstat)
            if prevpumpstat == 'ON':
                write_to_disk(logdata)
                prevpumpstat = 'OFF'

    return pumpstat
     
def _test():

   pass
 
if __name__ == '__main__':
    # This part runs only when this script is the main script, not when imported from other
    _test()

But that does not work for obvious reasons.
"prevpumpstat referenced before assignment" in line 73 or 84.

Question is: how do i prevent the useless disk writes and log only once the transition from on to off or vice versa? I wrecked my brain but my Python knowledge is insufficient.
Thansk for any pointers,
Steffen

[gontajones]

When your system starts, what is the state of the pump?
You just have to initialize the prevpumpstat with the same state.

When the pump state changes your conditional code will be sufficient to avoid useless disk writes.

[Steffenwolt]

Sorry, doesn't work.
Can't init the variable indatalog_v26 (the calling program) because the variable is local.
Can't init in pumpcontrol before the def statements because the function pumpcontrol runs starting line 51.
Can't init in the function pumpcontrol after line 51 because prevpumpstat then allways gets the same value regardless of situation.

[gontajones]

What if you check the GPIO pin state at the beginning of your pumpcontrol() function?

def pumpcontrol(threshold,boord):
    if (boord == 'sb'):
        mcpch = 0
        gpio = 23
        displ = I2C_LCD_driver_twoLCD.lcd(0x26)
    elif (boord == 'bb'):
        mcpch = 1
        gpio = 24
        displ = I2C_LCD_driver_twoLCD.lcd(0x27)
    else:
        pumpstat = False
        return 'No valid boord'

    # Like this
    GPIO.setup(gpio, GPIO.IN)
    if (GPIO.input(gpio)):
        prevpumpstat = 'ON'
    else:
        prevpumpstat = 'OFF'
    GPIO.setup(gpio, GPIO.OUT)

[Steffenwolt]

Hi Gontajones,
Tks for the answer. i'll give it a try today and let you know.

[Steffenwolt]

Tried it but it does'nt work.
I think that when you initialize the gpio as input it resets and either becomes low or floating so it does not retain it last status.
I got a bit further by defining the prevpumpstat variable in datalog.py and including it in the call to pumpcontrol.
When i get it working i will let you know.

[gontajones]

I think that when you initialize the gpio as input it resets and either becomes low or floating so it does not retain it last status.

I think you're right.
Lets try without setting it as input:

GPIO.setup(gpio, GPIO.OUT)
GPIO.output(gpio, GPIO.HIGH)
print(GPIO.input(gpio))
GPIO.output(gpio, GPIO.LOW)
print(GPIO.input(gpio))

I got a bit further by defining the prevpumpstat variable in datalog.py and including it in the call to pumpcontrol.

This is a more concise way to do it (IMHO). I thought that you couldn't modify the previous code.

[Steffenwolt]

I can't figure out what you are trying to archieve by the

print(GPIO.input(gpio))

statement.
Yes, i can modify all the code mentioned before. Its my own design so if you feel like it just modify away . Be my guest.

[Gribouillis]

Why don't you use class instances to keep track of state variables, something along the line of

class Sensor:
    def __init__(self, boord, gpio, mcpch, lcd):
        self.boord = boord
        self.gpio = gpio
        self.mcpch = mcpch
        self.lcd = lcd
        self.pumpstat = 'OFF'

sensors = {
    'sb': Sensor('sb', 23, 0, 0x26),
    'bb': Sensor('bb', 24, 1, 0x27),
}

[gontajones]

This is just a test to see if you can rely on checking the state of a GPIO without setting it as GPIO.IN.
Run it as a new script file...Or even direct with the python interpreter.

GPIO.setup(gpio, GPIO.OUT)
GPIO.output(gpio, GPIO.HIGH)
print(GPIO.input(gpio)) # Should print 1 (True)
GPIO.output(gpio, GPIO.LOW)
print(GPIO.input(gpio)) # Should print 0 (False)

And check the answer of @Gribouillis, it is a very useful idea.