Dividing certain columns in multiple CSV files

[NatesM]

I'm trying to divide all data in certain columns from multiple CSV files using python. I want to divide all columns with certain values except the first one.

This is what I wrote so far:

import os
import csv
import argparse

ap = argparse.ArgumentParser()
ap.add_argument("-i", "--input", required=True)
ap.add_argument("-o", "--output", required=True)
args = vars(ap.parse_args())


if os.path.exists(args["output"]) and os.path.isdir(args["output"]):
        print("Writing to {}".format(args["output"]))
else:
        print("Cannot write to directory {}".format(args["output"]))
        exit()

for file in os.listdir(args["input"]):
    if file.endswith(".csv"):
        print("{} ...".format(file))
        with open(os.path.join(args["input"],file), 'r', newline='') as infile, open(os.path.join(args["output"], file), 'w', newline='') as outfile:
            reader = csv.reader(infile)
            writer = csv.writer(outfile)
            rows = list(reader)
            data = rows[0:]
            def div(this_row):
                return [this_row[0], float(this_row[1])/1920, float(this_row[2])/560, float(this_row[3])/1920, float(this_row[4])/560]
            new_data = [div(row) for row in data]
            new_rows = new_data
            for row in new_rows:
                writer.writerow(row)

I get an error:

Error:
IndexError: list index out of range

Is it problem in different number of rows in CSV files?

[Larz60+]

Always show the complete unaltered error traceback as it contains valuable debugging information.
cannot determine line number with this limited error.

[ichabod801]

Each row in each file needs to have at least 5 items in it. Otherwise you will get that error. Also, next time please post the full text of the error message. There is useful information in the part you didn't post.

[NatesM]

I had empty rows in some csv files. Removed them with this code:

import os
import argparse

ap = argparse.ArgumentParser()
ap.add_argument("-i", "--input", required=True)
ap.add_argument("-o", "--output", required=True)
args = vars(ap.parse_args())

if os.path.exists(args["output"]) and os.path.isdir(args["output"]):
        print("Writing to {}".format(args["output"]))
else:
        print("Cannot write to directory {}".format(args["output"]))
        exit()

for file in os.listdir(args["input"]):
    if file.endswith(".csv"):
        print("{} ...".format(file))
        with open(os.path.join(args["input"],file), 'r') as infile, open(os.path.join(args["output"], file), 'w') as outfile:
            non_blank = (line for line in infile if line.strip())
            outfile.writelines(non_blank)

I tried it again and it worked. SOLVED

[ichabod801]

You could do that at the same time you are processing the files:

new_data = [div(row) for row in data if row.strip()]