Reading and manipulating csv

[Prince_Bhatia]

hi,


I have a csv which has two columns and on the other hand i have multiple lists in my code.

I am looping through second column of each data in csv and checking if it exists in the already defined lists.

if it exists create a new csv and add them. But what has happened till now it has added data but it is separating them into different columns and i want it data of each category into one column but comma separated.

below is my code:

import csv
from itertools import groupby,chain
from operator import itemgetter

Facility1 = ["club_house", "gymnasium", "children’s_play_area", "24/7_water_supply", "power_back-up", "inter-com","lift"]
Facility2 = ["jogging_track", "swimming_pool", "aerobics_center", "badminton_court", "basketball_court", "beach_volleyball_court", "football_court", "golf_course", "lawn_tennis_court","yoga/meditation_center"]
Facility3 = ["crèche", "ATM", "car_wash_area", "community_hall", "banquet_hall", "garbage_disposal", "grocery_shop","library"]
Facility4 = ["fountains","landscape_gardens"]
Facility5 = ["gated_society"]
Facility6 = ["car_parking"]
Facility7 = ["CCTV_cameras"]
Facility8 = ["security_guards"]
Facility9 = ["pool_table", "carrom_room", "chess_room", "dart_board", "squash_court", "table_tennis_room", "skating_rink"]


def main():
    prj_structure = {}
    f = open("./facility_data.csv","r",encoding="utf-8")
    data = f.read()
    f.close()
    lst = data.split("\n")
    prj = ""
    for i in range(1, len(lst)):
        val = lst[i].split(",")
        if len(val)>0:
            prj = val[0]
        if prj!="":
            if prj not in prj_structure.keys():
                prj_structure[prj] = []
            prj_structure[prj].append(val[1])
    return prj_structure

final_data = []

def readingfiles():
    global final_data
    data = main()
    for k,v in data.items():
        for i in v:
            sublist = []
            sublist.append(k)
            if i in Facility1:
                sublist.append(i+"-facility1")
            if i in Facility2:
                sublist.append(i+"-facility2")
            if i in Facility3:
                sublist.append(i+"-facility3")
            if i in Facility4:
                sublist.append(i+"-facility4")
            if i in Facility5:
                sublist.append(i+"-facility5")
            if i in Facility6:
                sublist.append(i+"-facility6")
            if i in Facility7:
                sublist.append(i+"-facility7")
            if i in Facility8:
                sublist.append(i+"-facility8")
            if i in Facility9:
                sublist.append(i+"-facility9")
            final_data.append(sublist)
    return final_data


def writefile(filename, alldata):
    global final_data
    with open(filename,"w", encoding="utf-8")as csvfile:
        csvfile = csv.writer(csvfile, delimiter=",")
        csvfile.writerow("")
        for i in range(0, len(alldata)):
            csvfile.writerow(alldata[i])

def get_new():
    readingfiles()
    writefile("new_facilities.csv",[[k, *chain.from_iterable(r for _, *r in g)] for k, g in groupby(final_data, key=itemgetter(0))])
get_new()

attached is the sample csv that i am using to create a new csv which is by name facility_data.csv

and attached is the output csv how i want data to be which is by name "new_facilities.csv"

Please help!

[Larz60+]

The Facility_data.csv file has non standard encoding what is the codec?

[Prince_Bhatia]

Codec is UTF-8 only

[Larz60+]

UnicodeDecodeError: 'utf-8' codec can't decode byte 0x92 in position 58: invalid start byte

[Prince_Bhatia]

i made a change in the code , please use that

[Larz60+]

I already tried that, the error I show is from modified code.
I've seen this before, are you on MS windows?
I believe the character in question is a '

[perfringo]

0x92 problem will 'go away' if instead of utf-8 to use either ISO-8859-1 or windows-1252 encoding.

[Larz60+]

perfringo's codec is correct.
this is much easier done by using a dictionary:
example:

import csv
from itertools import groupby,chain
from operator import itemgetter
import os

facilities = {
    'Facility1': ["club_house", "gymnasium", "children’s_play_area", "24/7_water_supply", "power_back-up", "inter-com","lift"],
    'Facility2': ["jogging_track", "swimming_pool", "aerobics_center", "badminton_court", "basketball_court", "beach_volleyball_court", "football_court", "golf_course", "lawn_tennis_court","yoga/meditation_center"],
    'Facility3': ["crèche", "ATM", "car_wash_area", "community_hall", "banquet_hall", "garbage_disposal", "grocery_shop","library"],
    'Facility4': ["fountains","landscape_gardens"],
    'Facility5': ["gated_society"],
    'Facility6': ["car_parking"],
    'Facility7': ["CCTV_cameras"],
    'Facility8': ["security_guards"],
    'Facility9': ["pool_table", "carrom_room", "chess_room", "dart_board", "squash_court", "table_tennis_room", "skating_rink"]
}

def main():
    # Make sure in src directory to start
    os.chdir(os.path.abspath(os.path.dirname(__file__)))

    with open("facility_data.csv", encoding="ISO-8859-1") as fp:
        crdr = csv.reader(fp, delimiter=',')
        for n, row in enumerate(crdr):
            for facid, values in facilities.items():
                if row[1] in values:
                    print(f'found: from row: {n}, {row[1]} ({row[0]}) in {facid}')

if __name__ == '__main__':
    main()

partial output:

Output:
found: from row: 1, swimming_pool (R3) in Facility2
found: from row: 2, gymnasium (R3) in Facility1
found: from row: 6, swimming_pool (R5) in Facility2
found: from row: 7, gymnasium (R5) in Facility1
found: from row: 11, swimming_pool (R6) in Facility2
found: from row: 12, gymnasium (R6) in Facility1
found: from row: 16, swimming_pool (R9) in Facility2
found: from row: 17, gymnasium (R9) in Facility1
found: from row: 20, swimming_pool (R10) in Facility2
found: from row: 21, gymnasium (R10) in Facility1
found: from row: 22, swimming_pool (R703) in Facility2
found: from row: 23, gymnasium (R703) in Facility1
found: from row: 26, swimming_pool (R17) in Facility2
found: from row: 27, gymnasium (R17) in Facility1
found: from row: 30, swimming_pool (R18) in Facility2

Also (which I haven't done), write youor output incrementally as you read input

[Prince_Bhatia]

Yes i am using windows

[Larz60+]

Please respond to post 8. Do you understand the concept here?