Control of the number of daemon error messages to the file

[MarPan]

Hello.

I'm working on a script that uses the "pymodbus" library to read data in synchronous mode using the ModBus RTU protocol, running as a daemon on a RaspianOS system based on Debian 11. The program writes the read data to files. The daemon mode is based on the HanzDaemon script, which uses a double branch. The daemon uses both output to the standard file descriptor and error logging to the standard file descriptor.

    def _makeDaemon(self):
        """
        tworzenie demona - podwójne rozgałęzienie
        """

        try:
            pid = os.fork()
            if pid > 0:
                # Exit first parent.
                sys.exit(0)
        except OSError as err:
            print(f'Błąd rozwidlenia #1: {err}')
            sys.exit(1)

        # odłączenie od środowiska rodzica
        os.chdir('/')
        os.setsid()
        os.umask(0)

        # wykonanie drugiego odgałęzienia
        try:
            pid = os.fork()
            if pid > 0:
                # Exit from second parent.
                sys.exit(0)
        except OSError as err:
            print(f'Błąd rozwidlenia #2: {err}')
            sys.exit(1)

        print(f'Proces demona uruchomiono w tle (PID:{os.getpid()}).')

        # przekierowanie standardowych deskryptorów plików
        sys.stdout.flush()
        sys.stderr.flush()
        si = open(self.stdin, 'r')
        so = open(self.stdout, 'a+')
        se = open(self.stderr, 'a+')
        os.dup2(si.fileno(), sys.stdin.fileno())
        os.dup2(so.fileno(), sys.stdout.fileno())
        os.dup2(se.fileno(), sys.stderr.fileno())

I use the "try - except" exception handler in my program to catch "IO Exception" and "Communication Exception" errors. Writing error messages to the output file is time-delayed. The program stores the first error occurrence and, after a defined period, checks whether the error still occurs, which is then recorded with an appropriate message in the output file. The script works correctly.

The problem is the lack of any control over the number of error messages written to the error file, which, when they occur, grows to a horrendous size.

For example, a missing serial port is recorded in the output file with messages every five minutes. During this time, the error file contains 100 "[Errno 2] could not open port /dev/ttyUSB0" messages.

Another example: a lack of response from a ModBus slave device is recorded in the output file with messages every 15 minutes. During this time, the error file contains "No response received after 3 retries, continue with next request" messages, equal to the number of registers queried.

How can I control the number of error messages written to the error file?

[deanhystad]

Please include the code that runs _makeDaemon and the code that is writing messages to your files.

[MarPan]

Thank you for your interest in the topic.

The code that runs "_makeDaemon" from HanzDaemon script:

    def start(self):
        """
        uruchomienie demona
        """

        # Handle signals
        signal.signal(signal.SIGINT, self._sigterm_handler)
        signal.signal(signal.SIGTERM, self._sigterm_handler)
        signal.signal(signal.SIGHUP, self._reload_handler)

        # sprawdzenie czy demon już nie działa
        procs = self._getProces()
        
        print(f'Znalezione: {procs}')

        if procs:
            pids = ",".join([str(p.pid) for p in procs])
            print(f'Szukam poprzednich procesów demona z PIDem {pids}. Czy demon już nie działa?')
            sys.exit(1)
        else:
            print(f'Uruchomienie demona w wersji: {self.ver}')

        # demonizacja głównego procesu
        self._makeDaemon()
        # uruchomienie nieskończonej pętli która okresowo uruchamia metodę run()
        self._infiniteLoop()

    def _infiniteLoop(self):
        try:
            if self.pauseRunLoop:
                time.sleep(self.pauseRunLoop)

                while self._canDaemonRun:
                    self.run()
                    time.sleep(self.pauseRunLoop)
            else:
                while self._canDaemonRun:
                    self.run()

        except Exception as err:
            # sys.stderr.write(f'Metoda run() zakończona błędem: {err}')
            print(f'Metoda run() zakończona błędem: {err}')
            sys.exit(1)

    # metoda którą trzeba przeciążyć w dziedziczonej klasie
    def run(self):
        pass

In my program I create a class that is inherited from the HanzDaemon script.

StdErrLog = ''
StdOutLog = ''

class MBpoll(HanzDaemon):

in this class I overwrite the standard descriptors to my files:

    # odczyt nazw plików standardowych wyjść z pliku konfiguracyjnego
    LFerr = CfgList.get('LogFiles','ErrorLog') # "MBMpolld_err.log"
    LFout = CfgList.get('LogFiles','OutLog') # "MBMpolld_out.log"
    global StdErrLog
    StdErrLog = os.path.join(LogPath,LFerr)
    global StdOutLog
    StdOutLog = os.path.join(LogPath,LFout)

The "run" method from the HanzDaemon script is overloaded in my program where I polling ModBus registers and catch exceptions from the "pymodbus" library.
And at the end of the program after overloading the method "run":

if __name__ == "__main__":
    mbpolling = MBpoll(stdout=StdOutLog,stderr=StdErrLog)

I suspect the issue may be related to the version of the "pymodbus" and "pyserial" libraries. The versions that the script worked on before the porting changes were made didn't have these issues. Current versions (pymodbus 3.8.6, pyserial 3.5) have changed their error reporting methods.

[DeaD_EyE]

The logging service of the Linux kernel has a solution to prevent this.
If a message is repeated, then it prints after the next different message the amount of repeated previous messages:

last message repeated x times

You have to program it.