API JSON response missing list gives keyerror

[rolfmadsen]

Python newbie - please be gentle ...
System: Windows 10 + Winpython + Spyder

I have a list of searchqueries that I use to perform search requests against an API which returns JSON.
For each response I print the values of the keys 'hitcount' and 'title' of the first object.

Some of the responses don't contain a searchresult resulting in:
"KeyError; 'searchResult' in the terminal.
NB. See full terminal output below.

I have been trying to print the values from a number of requests in a for loop and it works great until the keyerror blocks the loop.

I have tried ".get()", defaultdict and Try Except to no avail, but may just be using them incorrectly.

The missing JSON 'searchResult' structure looks like this:
NB. See full REST URL's below for full JSON response.

[...]

Output:
{
	"searchResponse": {
		"result": {
			"hitCount": {
				"$": "103554"
			},
			"searchResult": [
				{
					"collection": {
						"resultPosition": {
							"$": "1"
						},
						"numberOfObjects": {
							"$": "1"
						},
						"object": [
							{
								"record": {
									"title": [
										{
										"$": "Peter Nicolaisen",
										"@": "dc"
										},
									]
								}
							}
						]
					}
				},
				{},
				{},
				{},
				{}
			],
		},
	},
}

[...]

Query that works
https://oss-services.dbc.dk/opensearch/5...tType=json

Query that doesnt work
https://oss-services.dbc.dk/opensearch/5...tType=json

Script

#-*- coding: utf-8 -*-
"""
Created on Sat Mar 21 17:21:16 2020

@author: Rolf
"""
import json
import requests

def getsearchresult(query):
    
    querystring = f"\"{query}\""

    parameters = {
        'action':'search',
        'query':querystring,
        'agency':'100200',
        'profile':'test',
        'start':'1',
        'stepValue':'5',
        'outputType':'json'
    }
    
    endpoint = 'https://oss-services.dbc.dk/opensearch/5.2/?'
    
    response = requests.get(
        url=endpoint, 
        params=parameters,
        )
    print(response.url)
    response_json = response.json()
    return response_json

#query = 'peter' #works
query = 'verdens 100 mærkeligste dyr' #doesnt work

response_dict = getsearchresult(query)

hitcount = response_dict['searchResponse']['result']['hitCount']['$']
title = response_dict['searchResponse']['result']['searchResult'][0]['collection']['object'][0]['record']['title'][0]['$']

print(f"A search for the query '{query}' has {hitcount} hits, and the first title is {title}!")

My suspicion is that I simply dont understand how nested dictionaries combined with lists work, but after days of scouring the web for solutions I am throwing in the towel.

Question: How do I handle the keyerror so it doesnt block a for loop posting values from a number of responses.

I hope you can help! :-)

Terminal output

Error:
runfile('C:/Users/**** ******/Dropbox/Development/Python/opensearch/testscript_python-forum_io.py', wdir='C:/Users/**** ******/Dropbox/Development/Python/opensearch')
https://oss-services.dbc.dk/opensearch/5.2/?action=search&query=%22verdens+100+m%C3%A6rkeligste+dyr%22&agency=100200&profile=test&start=1&stepValue=5&outputType=json
Traceback (most recent call last):

  File "<ipython-input-19-badb5bb4e229>", line 1, in <module>
    runfile('C:/Users/**** ******/Dropbox/Development/Python/opensearch/testscript_python-forum_io.py', wdir='C:/Users/Rolf Madsen/Dropbox/Development/Python/opensearch')

  File "C:\Users\**** ******\Dropbox\Development\WPy64-3810\python-3.8.1.amd64\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 827, in runfile
    execfile(filename, namespace)

  File "C:\Users\**** ******\Dropbox\Development\WPy64-3810\python-3.8.1.amd64\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 110, in execfile
    exec(compile(f.read(), filename, 'exec'), namespace)

  File "C:/Users/**** ******/Dropbox/Development/Python/opensearch/testscript_python-forum_io.py", line 40, in <module>
    title = response_dict['searchResponse']['result']['searchResult'][0]['collection']['object'][0]['record']['title'][0]['$']

KeyError: 'searchResult'

[buran]

one way is to check the hitcount value:

hitcount = response_dict['searchResponse']['result']['hitCount']['$']
if hitcount == '0':
    print(f"A search for the query '{query}' has {hitcount} hits")
else:
    title = response_dict['searchResponse']['result']['searchResult'][0]['collection']['object'][0]['record']['title'][0]['$']
    print(f"A search for the query '{query}' has {hitcount} hits, and the first title is {title}!")

another ways is use .get() as you tried

hitcount = response_dict['searchResponse']['result']['hitCount']['$']
search_result = response_dict['searchResponse']['result'].get('searchResult')
if search_result:
    title = search_result[0]['collection']['object'][0]['record']['title'][0]['$']
    print(f"A search for the query '{query}' has {hitcount} hits, and the first title is {title}!")
else:
    print('No search result found')

another option is to error-handle the KeyError:

hitcount = response_dict['searchResponse']['result']['hitCount']['$']
try:
    title = response_dict['searchResponse']['result']['searchResult'][0]['collection']['object'][0]['record']['title'][0]['$']
    print(f"A search for the query '{query}' has {hitcount} hits, and the first title is {title}!")
except KeyError:
    print(f"A search for the query '{query}' has {hitcount} hits")

[rolfmadsen]

@buran thank you so much!

The .get() example was missing the hitcount variable and the underscore in "search response", but after fixing that all three examples worked as a charm.

Thank you again for helping me learn, and aside from this keyerror problem Python has been a joy to work with after a number of tries with JavaScript/Node.

[buran]

The .get() example was missing the hitcount variable

It was presumably kept from original code, maybe I was unclear, sorry. The other was just typo :-)
I will edit my post