Reading and writing to text file has format change

[cheerful]

Dear Sir,

I have been puzzled by the change of format between reading and writing the same content to a file. I have written my program base on the first format shown below

f= open('data.txt', 'r')

xdata=[]

for row in f:
    row = row.rstrip()
    row = row.split('\t')
    xdata.append(row)

f.close()

print(xdata[0:3])

[['DATE', 'SEC_ID', 'CTRY_ID', 'MKT_CAP'],
['9/30/2006', '67', 'US', '12.69'],
['9/30/2006', '44', 'US', '11.93']]

However when I write the above, xdata to a file and read it back. I get the following

f=open('test.txt', 'w')

for row in xdata:
    f.write('%s\n' % row)

f.close()

f= open('test.txt', 'r')

xdata1=[]

for row in f:
    row = row.rstrip('\n')
    row = row.split('\t')
    xdata1.append(row)

f.close()

print(xdata1[0:3])

[["['DATE', 'SEC_ID', 'CTRY_ID', 'MKT_CAP']"],
["['9/30/2006', '67', 'US', '12.69']"],
["['9/30/2006', '44', 'US', '11.93']"]]

Hence, my program cannot work. Do I need to have some settings to do a proper write and proper read back to remove the additional [" ... "] in each line or do I need to remove them manually? If so, how?

Thank you for your advices
L

[Larz60+]

You are adding a line feed ('\n') to the write buffer here:

for row in xdata:
f.write('%s\n' % row)

so you are not writing out what you read in.

[cheerful]

Hi

I remove \n in "f.write('%s' % row)" but it doesnt work. It keeps running and I have to close the python shell to shut it off. I check the test.txt and the data is continuous. ie ['X', 'X' , ...] [ 'X1', 'X2' , ...]. I think I should have \n so that I can see the data in each row in test.txt. Any idea to write the data in each row in test.txt and read back exactly what I write to it? tks

f= open('data.txt', 'r')

xdata=[]

for row in f:
    row = row.rstrip()
    row = row.split('\t')
    xdata.append(row)

f.close()

f=open('test.txt', 'w')

for row in xdata:
    f.write('%s' % row)

f.close()

f= open('test.txt', 'r')

xdata1=[]

for row in f:
    row = row.rstrip()
    row = row.split('\t')
    xdata1.append(row)

f.close()

print(xdata1[0:3])

[Larz60+]

I added some print statements to show what's happening in the code, this before a recommended way to write

f = open('data.txt', 'r')

xdata = []

for n, row in enumerate(f):
    print(f'row after line 5: n: {n}, {row}')
    row = row.rstrip()
    print(f'row after line 7: n: {n}, {row}')
    row = row.split('\t')
    print(f'row after line 9: n: {n}, {row}')
    xdata.append(row)
    print(f'xdata end of iteration {n}: {xdata}')
f.close()

f = open('test.txt', 'w')

for row in xdata:
    f.write('%s' % row)

f.close()

f = open('test.txt', 'r')

xdata1 = []

for row in f:
    row = row.rstrip()
    row = row.split('\t')
    xdata1.append(row)

f.close()

print(xdata1[0:3])

which produces:

Output:
row after line 5: n: 0, [['DATE', 'SEC_ID', 'CTRY_ID', 'MKT_CAP'],

row after line 7: n: 0, [['DATE', 'SEC_ID', 'CTRY_ID', 'MKT_CAP'],
row after line 9: n: 0, ["[['DATE', 'SEC_ID', 'CTRY_ID', 'MKT_CAP'],"]
xdata end of iteration 0: [["[['DATE', 'SEC_ID', 'CTRY_ID', 'MKT_CAP'],"]]
row after line 5: n: 1, ['9/30/2006', '67', 'US', '12.69'],

row after line 7: n: 1, ['9/30/2006', '67', 'US', '12.69'],
row after line 9: n: 1, ["['9/30/2006', '67', 'US', '12.69'],"]
xdata end of iteration 1: [["[['DATE', 'SEC_ID', 'CTRY_ID', 'MKT_CAP'],"], ["['9/30/2006', '67', 'US', '12.69'],"]]
row after line 5: n: 2, ['9/30/2006', '44', 'US', '11.93']]

row after line 7: n: 2, ['9/30/2006', '44', 'US', '11.93']]
row after line 9: n: 2, ["['9/30/2006', '44', 'US', '11.93']]"]
xdata end of iteration 2: [["[['DATE', 'SEC_ID', 'CTRY_ID', 'MKT_CAP'],"], ["['9/30/2006', '67', 'US', '12.69'],"], ["['9/30/2006', '44', 'US', '11.93']]"]]
[['["[[\'DATE\', \'SEC_ID\', \'CTRY_ID\', \'MKT_CAP\'],"]["[\'9/30/2006\', \'67\', \'US\', \'12.69\'],"]["[\'9/30/2006\', \'44\', \'US\', \'11.93\']]"]']]

However, I would write this way (without regard to errors at this point):

with open('data.txt') as f, open('test.txt', 'w') as fo:
    for row in f:
        row = row.strip().split('\t')
        fo.write(f'{row}\n')
        fo.write(str(row))

# Read back
with open('test.txt') as f:
    for row in f:
        print(row)

which produces:

Output:
["[['DATE', 'SEC_ID', 'CTRY_ID', 'MKT_CAP'],"]

["[['DATE', 'SEC_ID', 'CTRY_ID', 'MKT_CAP'],"]["['9/30/2006', '67', 'US', '12.69'],"]

["['9/30/2006', '67', 'US', '12.69'],"]["['9/30/2006', '44', 'US', '11.93']]"]

["['9/30/2006', '44', 'US', '11.93']]"]