Clean file with missing values

[Felipe]

Hi guys,

I need to clean a file with 13 millions lines. Some lines don't has value on the third column, so I need to exclude them.
Here's an input example:

Output:
-8.17641148055556 54.4510540761111 -178.44
-8.17642712805556 54.4511079527778 -178.29
-8.17644273722222 54.4511528463889 
-8.176458385 54.4512067233333 
-8.17648941 54.4512515941667

This is the desired output:

Output:
-8.17641148055556 54.4510540761111 -178.44
-8.17642712805556 54.4511079527778 -178.29

Here's my code, but it gives me a blank file:

#!/usr/bin/env python3

with open('cleancol.txt',"r") as fh, open('cleancol2.txt',"w") as sh:
    for row in fh:
        row_list = row.split(' ')
        if row_list[2].isnumeric():
            print(row, file=sh, end='')

Thanks for the help !!

[j.crater]

Hello,
I didn't test your code manually to see why you get a blank file. But the approach doesn't seem right. Because, what is row_list[2], when you only have 2 columns?
Instead I would rather use string split method, and check whether the returned list has 2 or 3 elements. Good luck!

[buran]

row_list[2] will have \n at the end, so isnumeric() will return False. Strange that you don't get IndexError for rows with 2 numbers...

[j.crater]

That missing IndexError is exactly what bugs me. Is it possible that file isn't opened properly, or even not the right file?

[buran]

That missing IndexError is exactly what bugs me. Is it possible that file isn't opened properly, or even not the right file?

Could it be that there are several spaces at the end, i.e. third item then I think would be '' (empty str)?

[Felipe]

That missing IndexError is exactly what bugs me. Is it possible that file isn't opened properly, or even not the right file?

Could it be that there are several spaces at the end, i.e. third item then I think would be '' (empty str)?

Exactly !!

The .txt file that I use with input was generated from a huge .csv file which has more than 13 millions lines and about 30 columns. Some fields in .csv file are empty. So it's probably that my .txt file has empty str in some lines in the third column.

Any ideas about how I can solve this question? Thanks.

[buran]

strip '\n' from row_list before split or simply check that row_list[2] is '\n'

[Larz60+]

#!/usr/bin/env python3

with open('cleancol.txt', "r") as fh, open('cleancol2.txt', "w") as sh:
    for row in fh:
        row_list = row.split(' ')
        if len(row_list) < 3:
            print(row, file=sh, end='')

results:

Output:
-8.17644273722222 54.4511528463889
-8.176458385 54.4512067233333
-8.17648941 54.4512515941667

The extra line in results created by forum markdown, not in file!

[wavic]

The thirth column can be on of these - '\t\r\x0b\x0c'. row_list.strip() should fix it