Read nested data from JSON - Getting an error

[marlonbown]

i

mport json

#Later this will be extracted from a file
RAW_DATA = """ {
    “991998795651": {
    "cloudformation": {
    "CloudFormation Detect: {
    "analysis_results": [
    {
    "account_id": “991998795651",
    "analysis_scope": "Service",
    "analyzer_name": "CloudFormation Drift",
    "compliance": "Pass",
    "evidence": "Cloudformation stack.",
    "is_system_created": false,
    "mitigation": {
    "link": "https://www.myexample.com”,
    "summary": “Test.”
    },
    "region": "us-west-2",
    "resource": {
    "_arn_is_constructed": false,
    "account_id": “991998795651",
    "data_source": "AwsSdk",
    "region": "us-west-2",
    "resource_arn":
    "arn:aws:cloudformation:us-west-2:991998795651:stack/test”,
    "resource_id": “Test”,
    "scan_time": "2021-12-20T20:42:30.118583Z"
    }
"""

def scanvuln(rawdata, account):

    data = json.loads(rawdata) #<== Getting error  return _default_decoder.decode(s)

    data = json.loads(rawdata)

myaccount = "131998795651"
scanvuln(RAW_DATA, myaccount)

[bowlofred]

I think you're not showing the entire error. It should be telling you there's a JSON error at a particular point in the json. You're missing some quote marks, some of your quote marks aren't the correct kind, and it appears to be incomplete (several items aren't closed).

The error should tell you exactly what line the first problem is on.

[mrapple2020]

Thank you for your professionalism. My bad.

I ran the code below and it executes. Can you do me a favor and point out from here how can I retrieve the value of the "compliance" key?

import json

def new_func():
    RAW_DATA =  """
    {"131998795651": {
            "cloudformation": {
                "CloudFormation Drift Detection Analysis": {
                    "analysis_results": [{
                        "account_id": "131998795651",
                        "analysis_scope": "Service",
                        "analyzer_name": "CloudFormation Drift Detection Analysis",
                        "compliance": "Pass",
                        "evidence": "Cloudformation stack is in sync for drift detection."
    }]}}}}"""

    return RAW_DATA

RAW_DATA = new_func()

def scanvuln(rawdata, account):

    data = json.loads(rawdata)
    for i in data.items():
        print(i)
    

myaccount = "131998795651"
scanvuln(RAW_DATA, myaccount)

[deanhystad]

It is just a list of dictionaries inside a dictionary inside a dictionary inside a .... Wow, that is a bad looking data structure.

data[account]["cloudformation"]["CloudFormation Drift Detection Analysis"]["analysis_results"][0]["compliance"]

[DeaD_EyE]

To understand nested JSON better, you could use JSON Crack: https://jsoncrack.com/editor

[snippsat]

It is just a list of dictionaries inside a dictionary inside a dictionary inside a .... Wow, that is a bad looking data structure.

That's how data structure are many times coming from largere API resonse in Json format,i would not say it's bad structure.
The data comes for AWS CloudFormation

.
I ran the code below and it executes. Can you do me a favor and point out from here how can I retrieve the value of the "compliance" key?

The RAW_DATA may already been encoded for use in Python,how are you getting the data?
Also for output to work on site DeaD_EyE postet have to this like this to get double quotation marks.
Or as mention just the RAW_DATA that's in string could be used if as it's alradt a dictionary put in a string.

import json
from nested_lookup import nested_lookup

def new_func():
    RAW_DATA =  """
    {"131998795651": {
            "cloudformation": {
                "CloudFormation Drift Detection Analysis": {
                    "analysis_results": [{
                        "account_id": "131998795651",
                        "analysis_scope": "Service",
                        "analyzer_name": "CloudFormation Drift Detection Analysis",
                        "compliance": "Pass",
                        "evidence": "Cloudformation stack is in sync for drift detection."
    }]}}}}"""

    return RAW_DATA

RAW_DATA = new_func()
data = json.loads(RAW_DATA)
js_data = json.dumps(data, indent=2)
print(js_data)

Now can copy data into Json Crack

Output:
{
  "131998795651": {
    "cloudformation": {
      "CloudFormation Drift Detection Analysis": {
        "analysis_results": [
          {
            "account_id": "131998795651",
            "analysis_scope": "Service",
            "analyzer_name": "CloudFormation Drift Detection Analysis",
            "compliance": "Pass",
            "evidence": "Cloudformation stack is in sync for drift detection."
          }
        ]
      }
    }
  }
}

See also that i have imported nested_lookup,this make it easy to search direclity in a nested stucture like this.

>>> nested_lookup("compliance", data)
['Pass']

So the same result as deanhystad break down into the dictionary.

>>> data["131998795651"]["cloudformation"]["CloudFormation Drift Detection Analysis"]["analysis_results"][0]["compliance"]
'Pass'