Hello,
For some reason, while this line works fine when using a local string in a test script, I get a "KeyError" when actually fetching a web page although I can see the element with that attribute does exist in the page source:
Thank you.
--
Edit: Using this shortcut (I didn't know) triggers the same error:
Edit: No better
Edit: I can see that attribute and its value in the dump as expected :-/
Edit: This triggers the exception:
Edit: Couild it be that BS doesn't support dashes in key names?
Edit: Apparently, some kung-fu is required when the attribute contains dashes/hyphens… and to get its value instead of the whole HTML string
For some reason, while this line works fine when using a local string in a test script, I get a "KeyError" when actually fetching a web page although I can see the element with that attribute does exist in the page source:
#KeyError: 'data-share-icon' if soup.span['data-share-icon']:Any idea what it could be?
Thank you.
--
Edit: Using this shortcut (I didn't know) triggers the same error:
if soup['data-share-icon']:--
Edit: No better
if soup['data-share-icon'] is not None:--
Edit: I can see that attribute and its value in the dump as expected :-/
reqs = requests.get(url, headers=HEADERS)
soup = BeautifulSoup(reqs.text, 'lxml')
#to investigate
with open("soup.txt", "w", encoding="utf-8") as text_file:
text_file.write(str(soup))--Edit: This triggers the exception:
from bs4 import BeautifulSoup
import json
KEY = "data-share-icon"
html = '''<span data-share-icon='{"title":"Blah","locale":"en-US"}'></span>'''
soup = BeautifulSoup(html, 'lxml')
print(soup)
try:
data = soup[KEY]
except:
print("Bad")--Edit: Couild it be that BS doesn't support dashes in key names?
KEY = "data-share-icon"
html = '''<head><title>MyTitle</title></head><span data-share-icon='{"title":"Blah","locale":"en-US"}'></span>'''
KEY = "mykey"
html = '''<head><title>MyTitle</title></head><span mykey='{"title":"Blah","locale":"en-US"}'></span>'''
soup = BeautifulSoup(html, 'lxml')
if soup.span[KEY]:
print("OK")--Edit: Apparently, some kung-fu is required when the attribute contains dashes/hyphens… and to get its value instead of the whole HTML string
if soup.select_one(f"span[{KEY}]"):
#returns whole tree, not attribute value!
#data = soup.select_one(f"span[{KEY}]")
data = soup.span[KEY]
print(data)
data = json.loads(data)
print("Found title:",data['title'])
