Jul-10-2018, 08:35 AM
(This post was last modified: Jul-10-2018, 08:35 AM by Prince_Bhatia.)
i am trying to scrape few pages via reading them through csv. This is one of the example web page http://up-rera.in/AgentDetails.aspx?Id=10868
i am trying to scrape name and mobile values.
but whenever i run my code it returns BLANK. i am trying to scrape 360 REALTORS LLP and 9871898000
below is my code:
i am reading the files correctly , it is returning headings like name, Father's Name, Applicant Type but not there values
Please help
i am trying to scrape name and mobile values.
but whenever i run my code it returns BLANK. i am trying to scrape 360 REALTORS LLP and 9871898000
below is my code:
import requests
from bs4 import BeautifulSoup
import time
import os
import csv
file = {}
final_data = []
def read_file():
global file
f = open("Data.csv")
for line in f.readlines():
data = line.split(",")
file[data[0]]= data[1]
def writedata(alldata1, filename):
with open("./"+filename, "w")as csvfile:
csvfile = csv.writer(csvfile, delimiter=",")
csvfile.writerow("")
for i in range(0, len(alldata1)):
csvfile.writerow(alldata1[i])
def parse_data():
global file
global final_data
for data_obj in file.keys():
link = file[data_obj]
#print(link)
data = getbyget(link, {})
soup = BeautifulSoup(data, "html.parser")
get_details = soup.find_all("div", {"class":"col-sm-12"})[0]
#print(get_details)
mobile = get_details.find_all("span", {"id":"ctl00_ContentPlaceHolder1_lblAname"})
for i in mobile:
datas = i.text
print(datas)
return final_data
def getbyget(url, values):
res = requests.get(url, data=values)
data = res.text
return data
def main():
read_file()
datas = parse_data()
writedata(datas, "UP_DATA.csv")
main()i am trying to scrape this webpage from last 24 hours and same output i received. just found out __EVENTTARGET , __EVENTARGUMENT, __VIEWSTATE are all hidden, it there something related this with i am reading the files correctly , it is returning headings like name, Father's Name, Applicant Type but not there values
Please help
