python multiple try except block in my code -- can we shorten code

[mg24]

Hi Team,

I am new in python, used functional programming, not perfect in oop concept yet.

In below task I have used try except block for handling error.

Can we reduce code, multiple try except block in my code.


what is the best way of doing it. thanks.



I have 10 python modules\files of a project.
Created a function in each module

imported all functions in main module.

calling all functions sequentially

I am try except block

from project_connectsql import connectsql
from project_stored_procedure import stored_procedure
from project_update_Control_Table import update_Control_Table
from project_extract_dynamic_header_name import extract_dynamic_header_name
from project_extract_sql_Records import extract_sql_Records
from project_write_Data_csv import write_Data_csv
from project_create_gzip import create_gzip
from project_backup import backup
from project_cloud_move import cloud_move
from project_Delete_files import Delete_files
from Readtime import Readtime
import os
import sys
import traceback


def main()
    try:
        validate(input1,input2)
    except Exception as e:
        print("error occured while validating input value")
    

    try:
        connectsql(server,database)
    except Exception as e:
        print("error occured")
        sys.exit()

    try:
        stored_procedure(server,database,table)
    except Exception as e:
        print("Error occured at stored procedure")
        sys.exit()


    try:
        stored_procedure(server,database,table)
    except Exception as e:
        print("Error occured at stored procedure")
        logger.info("Error occured while connecting stored procedure")
        sys.exit()


    try:
        update_Control_Table(server,database,table,fnmae,status=1)
    except Exception as e:
        print("Error occured while updating updating control table in sql")
        logger.info("Error occured while updating updating control table in sql")
        sys.exit()

    try:
        extract_dynamic_header_name(server,database,table)
    except Exception as e:
        print("Error occured while while extracting file name convention")
        logger.info("Error occured while updating updating control table in sql")
        sys.exit()

    try:
        extract_sql_Records(server,database,table)
    except Exception as e:
        print("Error occured while extracting sql recordsl")
        logger.info("Error occured while extracting sql recordsl")
        sys.exit()


    try:
        write_Data_csv(folderpath,fname)
    except Exception as e:
        print("Error occured while while writing it data into csv")
        logger.info("Error occured while extracting sql recordsl")
        sys.exit()

    try:
        generate_chksum(folderpath,fname)
    except Exception as e:
        print("Error occured while while generating checksum no")
        logger.info("Error occured while extracting sql recordsl")
        sys.exit()

    try:
        create_gzip(folderpath,fname)
    except Exception as e:
        print("Error occured while while generating gzip file")
        logger.info("Error occured while generating gzip file")
        sys.exit()

    try:
        backup(sourcepath,destinationpath)
    except Exception as e:
        print("Error occured while taking backup")
        logger.info("Error occured while taking backup")
        sys.exit()

    try:
        cloud_move(mainfolderpath)
    except Exception as e:
        print("Error occured while moving files to cloud")
        logger.info("Error occured while moving files to cloud")
        sys.exit()
    try:
        Delete_files(mainfolderpath)
    except Exception as e:
        print("Error occured while archiving")
        logger.info("Error occured while archiving")
        sys.exit()

if __name__"__main__":
    main()

[carecavoador]

I might be wrong, but I think it is not best practice to use generc Exceptions like this.

As I can see from the code you posted, this is just a simple linear routine of diffretent functions. One way you could improve ite is to write custom exceptions for each function, than you could execute one single try block and catch multiple exceptions at the end.

class StorageError(Exception):
    """There was an error on the Storage process."""

class SQLError(Exception):
    """There was an error on the SQL process."""

class UpdateError(Exception):
    """There was an error on the Update process."""

try:
    """Your routine goes here."""
    pass
except StorageError:
    """Handle Storage Exception."""
except SQLError:
    """Handle SQL Exception."""
except UpdateError:
    """Handle Update Exception."""

[rob101]

I could be wrong also, but the way I see it is that exceptions should not be the default, but rather (as the name suggests) the exception; that is to say only use a try / except block when you have no other way to catch a potential error or script crash.

Maybe I'm a lone voice in this view point.

[Gribouillis]

My best advice with exceptions in Python is: Let them propagate, unless you have a very good reason to catch them.

In this code, there is no use in catching the exceptions of the call cloud_move(mainfolderpath) just to print "Error occured while moving files to cloud". Letting the exception propagate will print the exception traceback which is much more informative and will show you that the error occurred while calling cloud_move(). It means that with catching the exception, you added no information to the output and you hid potentialy crucial debugging information, such as why it was not possible to move them to the cloud.

If you really don't want to let the exceptions propagate and crash the program, you could just catch them at the top level, meaning when the main() function is called. This could be a safety net to exit gracefully.

So I would rather write the code as follows, the call to logger.exception() will log the whole exception traceback which is what you need to debug

from project_connectsql import connectsql
from project_stored_procedure import stored_procedure
from project_update_Control_Table import update_Control_Table
from project_extract_dynamic_header_name import extract_dynamic_header_name
from project_extract_sql_Records import extract_sql_Records
from project_write_Data_csv import write_Data_csv
from project_create_gzip import create_gzip
from project_backup import backup
from project_cloud_move import cloud_move
from project_Delete_files import Delete_files
from Readtime import Readtime
import os
import sys
import traceback
 
 
def main()
    validate(input1,input2)
    connectsql(server,database)
    stored_procedure(server,database,table)
    stored_procedure(server,database,table)
    update_Control_Table(server,database,table,fnmae,status=1)
    extract_dynamic_header_name(server,database,table)
    extract_sql_Records(server,database,table)
    write_Data_csv(folderpath,fname)
    generate_chksum(folderpath,fname)
    create_gzip(folderpath,fname)
    backup(sourcepath,destinationpath)
    cloud_move(mainfolderpath)
    Delete_files(mainfolderpath)
 
if __name__"__main__":
    try:
        main()
    except Exception:
        logger.exception('Execution failed')  # log the exception traceback for debugging
        #raise  # uncomment to print the traceback to stdout

[deanhystad]

I think you mean procedural programming. Functional programming is something completely different.

I think rob101 is biased against exceptions. Probably comes from learning programming in a language where exceptions are difficult to use. Having written millions of lines of code both ways I think the code is always cleaner using exceptions in place of exhaustive testing.

That said, this is not the way to write exception handlers.

    try:
        backup(sourcepath,destinationpath)
    except Exception as e:
        print("Error occured while taking backup")
        logger.info("Error occured while taking backup")
        sys.exit()

If the only possible recovery is exiting the program, one exception handler that catches them all is adequate. The only modification's I would make to Gribouillis' code is to removing the comment on raise so I don't have to open a log file to see the error trace, and he already predicted that.

Exceptions where you can recover aren't handled all the differently. It is mostly a matter of locale. Dangerous code is wrapped by try/except. You might have multiple handlers for multiple exception types, but they should all perform some corrective action that lets you move on with execution of the program.

calculator = {
    "+": lambda a, b: a + b,
    "-": lambda a, b: a - b,
    "*": lambda a, b: a * b,
    "/": lambda a, b: a / b,
}

while True:
    try:
        equation = input("Enter equation ")
        a, op, b = equation.split()
        print(equation, "=", calculator[op](float(a), float(b)))
    except KeyError:
        print("Unrecognized operator:", equation)
    except ZeroDivisionError:
        print("Cannot divide by zero")
    except ValueError:
        print("""Equation format is "number opeator number".""")
        print("""Supported opeerators are +, -, *, /.""")

[rob101]

I think rob101 is biased against exceptions. Probably comes from learning programming in a language where exceptions are difficult to use.

Yes and no.

I simply believe that prevention is better than a cure: if you can prevent an error from accruing, you don't need to deal with the consequence.

[Gribouillis]

I simply believe that prevention is better than a cure: if you can prevent an error from accruing, you don't need to deal with the consequence.

This is called Defensive Programming. It has its domains of application, but generally speaking, I think it goes against the philosophy of the Python language.

[ndc85430]

It's not just about the language. Adding more prevention mechanisms means more complexity in the code, making it harder to understand and change. That costs and you have to decide at what point it is or isn't worth it.

For example, in one of the systems my team is responsible for, we have some validation of the input data but not for absolutely everything we could - some of the data might be a bit incorrect, or we may not be able to look up some things from another system. We allow the data into our system, because for us it's more important that the content is available on our platforms than some minor things being incorrect. Your mileage may vary, of course.

[rob101]

All I'm trying to say is that I don't believe that exceptions should be (or need to be) a go-to mechanism. Errors are errors and should be handled as such. If there's an unexpected error, then that is (or should be) an exceptional case and again, should be handled as such.

[snippsat]

I simply believe that prevention is better than a cure: if you can prevent an error from accruing, you don't need to deal with the consequence.

Python has definition for this that good to know about.
LBYL(Look before you leap) and EAFP(Easier to ask for forgiveness than permission).
Python developers generally tend to favor EAFP over LBYL.
This good read about it LBYL vs EAFP: Preventing or Handling Errors in Python

I do like EAFP(Easier to ask for forgiveness than permission) way,but use a mix that i think many do without thinking about it.
Example can use this sometime which is LBYL and convinet to do,but if look at source for code exists then will see that it's using try/except.

import os

if not os.path.exists(path):
    # Folder dos not exists,make it
    os.makedirs(filepath)

Source code for exists in Python source code in a EAFP style.

def exists(path):
    """Test whether a path exists.  Returns True for broken symbolic links"""
    try:
        st = os.lstat(path)
    except (OSError, ValueError):
        return False
    return True