Create a 2-channel numpy file from two cvs file containing a 9x9 matrix

[silvialecc]

Hello everyone, I work in a CNN project for my thesis and I have problem with my Python code. I hope that somebody can help me. Thank you.

I would like to take two .cvs files with the same name (each is a 9x9 matrix) from a folder and create a unique file of the size 9x9x2 and save it in another folder.

This is my code, right now the problem is that the final file has been saved as a 9x18 instead of 9x9x2.

import os, glob
from collections import defaultdict

dirs = ['/content/drive/MyDrive/Colab Notebooks/Nuova cartella/mu1/real', '/content/drive/MyDrive/Colab Notebooks/Nuova cartella/mu1/imag']
file_pattrn = r'*.csv'
unique_files = defaultdict(list)

for d in dirs:
  for i in glob.iglob(os.path.join(d, file_pattrn)):
     unique_files[os.path.basename(i)].append(i)

destination = '/content/drive/MyDrive/Colab Notebooks/New_Dataset'
for unique_filename, copies in unique_files.items():
  with open(os.path.join(destination, unique_filename), 'w') as f:
      for copy in copies:
          with open(copy, 'r') as cp:
            for line in cp:
                f.write(line)

[Gribouillis]

Why not something like (untested)

import numpy as np
for unique_filename, copies in unique_files.items():
    a = np.array([np.loadtxt(copy) for copy in copies])
    np.savetxt(os.path.join(destination, unique_filename), a)

Well actually, the above code doesn't work because numpy.savetxt() cannot handle 3D arrays. You can save the 3D array under binary format by using numpy.save() with the allow_pickle=True option. Here is an example with 3x3 arrays

import numpy as np
import io

file1 = io.StringIO("""\
1.000000000000000000e+00 2.000000000000000000e+00 3.000000000000000000e+00
5.000000000000000000e+00 6.000000000000000000e+00 7.000000000000000000e+00
9.000000000000000000e+00 1.000000000000000000e+01 1.100000000000000000e+01
""")

file2 = io.StringIO("""\
-1.000000000000000000e+00 -2.000000000000000000e+00 -3.000000000000000000e+00
-5.000000000000000000e+00 -6.000000000000000000e+00 -7.000000000000000000e+00
-9.000000000000000000e+00 -1.000000000000000000e+01 -1.100000000000000000e+01
""")

out = io.BytesIO()
a = np.array([np.loadtxt(f) for f in (file1, file2)])
a = np.moveaxis(a, 0, -1)
np.save(out, a, allow_pickle=True)
out.seek(0)
print(np.load(out, allow_pickle=True))
print(a.shape)

Output:
[[[  1.  -1.]
  [  2.  -2.]
  [  3.  -3.]]

 [[  5.  -5.]
  [  6.  -6.]
  [  7.  -7.]]

 [[  9.  -9.]
  [ 10. -10.]
  [ 11. -11.]]]
(3, 3, 2)