Python: Automated Script to Read Multiple Files in Respective Matrices

[Robotguy]

Hello,

I am a beginner and trying to automate a simple problem. I am reading multiple files one-by-one and saving the data in respective matrices. I would like to make my code short by automating (e.g., by looping).

Would appreciate any suggestions, the files are delimited and named numerically (File_1, File_2, etc.). Here is what I am doing as of now:

    file1 = np.loadtxt('File_1.txt', delimiter=',')
    file2 = np.loadtxt('File_2.txt', delimiter=',')
    ...

    matrix1 = file1[:,:]
    matrix2 = file2[:,:]
    ...

[bowlofred]

Instead of storing the data in regular variables (like matrix1), store them in a list or a dictionary.

You can create the filenames in a loop by range(), but it might be better to use os.listdir() or glob.glob() to get a list of all the files that exist that match what you want.

matrix_data = {}
for file_index in range(1,10):
    filename = f"File_{file_index}.txt"
    file = np.loadtxt(filename, delimiter=',')
    matrix_data[filename] = file[:,:]

# can refer to matrix_data["File_1.txt"] for the data...

[Robotguy]

Hi Bowlofred:

Thanks, so if I am interested in accessing a particular element is there any clever way? I am doing this:

a = matrix_data["file2.txt"]
print(a[1,1])

[bowlofred]

If you have several operations on the data, your way is fine. You can temporarily (or in a loop) assign it to a shorter name (like "a" in your example) and then do your work.

But if you just need the element, you can do it in one line instead of two with

print(matrix_data["file2.txt"][1,1])

[Robotguy]

Thanks. Yes I am doing quite some operations and saving the data in separate variables is what I desire.

One last thing though. Is there a possibility to generate the variable name using loop (a1, a2, etc)? I want to do something like given below. Only change I need is to automatically let python change a1 to a2 in the second iteration and so on. Hence: file1->a1 ; file2->a2 ; ...

filenames = ['file1.txt', 'file2.txt']

for i in range(0,2):
    file = np.loadtxt(filenames[i], delimiter=',')
    a1 = file[:,:]
    print(a1)

[bowlofred]

Why do you need to change the variable? Just leave it as "a" each time through the loop as you deal with each matrix in turn.

In general you don't want to create dynamic top-level variables. If you need a dynamic name, make it the key of a dictionary.

[Robotguy]

I actually need to carry out operations on each element of a1 with each element of a2 after they are fully read. Would appreciate if you can elaborate more on how to assign keys in this particular case?

[bowlofred]

If there's only two things, just assign them outside the loop. If there's more than two things, how do you decide which is a1 and which is a2?