changing key names in nested dict

[wardancer84]

hi,

nested dict:

 'identity': {'AIX_REG_CLIENT_VER': '0.1-alpha',
              'HOSTNAME': 'sag01147',
              'SYSTEM_NAME': 'SAG01147_JBOSS4_HA'},
 'nodeinfo': {'CLUSTER_NAME': 'JBOSS4',
              'CLUSTER_NODENAME': 'sag01147',
              'CLUSTER_NODES': 'kug01147,sag01147',
              'CLUSTER_RGS': 'JBOSS33,JBOSS34,JBOSS37,JBOSS38',
              'CL_RGS': 'JBOSS38',
              'CPU_ENTC': '0.60',
              'CPU_VALUE': '4',
              'HAS_COBOL': '0',
              'HAS_JBOSS': '1',
              'HAS_ORACLE': '0',
              'HAS_SAP': '0',
              'HAS_TUXEDO': '0',
              'HAS_UC4': '0',
              'HA_LEVEL': '7.2.4 SP2',
              'IP': {'en0': '172.17.10.147',
                     'en1': '172.16.88.57',
                     'en2': '172.16.12.147'},
              'IP_LONG': {'en0': '172.17.10.147/255.255.248.0/172.17.15.255/',
                          'en1': '172.16.88.57/255.255.192.0/172.16.127.255/sag01147-ipl.some.domain',
                          'en2': '172.16.12.147/255.255.254.0/172.16.13.255/'},
              'IS_CLUSTER': '1',
              'IS_LPM': '0',
              'IS_ZPI': '0',
              'IS_ZPV': '0',
              'Java6': '6.0.0.655',
              'Java6_64': '6.0.0.655',
              'Java7': '7.0.0.665',
              'Java7_64': '7.0.0.665',
              'Java8_64': '8.0.0.610',
              'LOCATION': 'Geiselberg',
              'MACHINE_SERIAL': 'XXXXXX',
              'MAN_SYS_NAME': 'GBG02',
              'MEM_VALUE': '61440',
              'OS_LEVEL': '7200-04-03-2038',
              'STORAGE_VALUE': '258',
              'UPTIME': '54',
              'YUM_REPO': 'IBM'},
 'timestamp': {'DATETIME': '2021-09-09T17:04:32'}}

the usuel flattening code:

for key, value in sample['nodeinfo'].items():
            if isinstance(value, dict):
                for key, value in value.items():
                    entries.append([key, value])
            else:
                entries.append([key, value])
        print(entries)

which produces this

[['HAS_COBOL', '0'], ['HAS_TUXEDO', '0'], ['HAS_SAP', '0'], ['HAS_ORACLE', '0'], ['IS_LPM', '0'], ['HAS_JBOSS', '1'], ['IS_ZPI', '0'], ['IS_ZPV', '0'], ['HAS_UC4', '0'], ['MEM_VALUE', '61440'], ['CPU_VALUE', '4'], ['STORAGE_VALUE', '258'], ['MAN_SYS_NAME', 'GBG02'], ['LOCATION', 'Geiselberg'], ['OS_LEVEL', '7200-04-03-2038'], ['UPTIME', '54'], ['MACHINE_SERIAL', 'XXXXX'], ['CPU_ENTC', '0.60'], ['YUM_REPO', 'IBM'], ['Java6', '6.0.0.655'], ['Java6_64', '6.0.0.655'], ['Java7', '7.0.0.665'], ['Java7_64', '7.0.0.665'], ['Java8_64', '8.0.0.610'], ['IS_CLUSTER', '1'], ['CLUSTER_NAME', 'JBOSS4'], ['CLUSTER_NODES', 'kug01147,sag01147'], ['CLUSTER_NODENAME', 'sag01147'], ['CLUSTER_RGS', 'JBOSS33,JBOSS34,JBOSS37,JBOSS38'], ['HA_LEVEL', '7.2.4 SP2'], ['CL_RGS', 'JBOSS38'], ['en0', '172.17.10.147'], ['en1', '172.16.88.57'], ['en2', '172.16.12.147'], ['en0', '172.17.10.147/255.255.248.0/172.17.15.255/'], ['en1', '172.16.88.57/255.255.192.0/172.16.127.255/sag01147-ipl.some.domain'], ['en2', '172.16.12.147/255.255.254.0/172.16.13.255/']]

as the "en" stuff is only to keep dict keys unique, i need to replace them with the name of the containing dict, "IP" for example.
any clever ideas how to do this in a reuseable way? i tried list str.replace and dict key.pop methods which all failed.

[wardancer84]

tried to create a list and then append that to the dict...also no way, whats wrong here? everybody seems to doing like this....

def nested_defaultdict(default_factory, depth=1):
    result = partial(defaultdict, default_factory)
    for _ in repeat(None, depth - 1):
        result = partial(defaultdict, result)
    return result()

 my_dict = nested_defaultdict(list, 5)

ifconfig_output = subprocess.check_output(['/usr/sbin/ifconfig', '-a']).decode('utf-8')
    result = re.findall(r'^(\S+):.*?[\s]+inet ([\d\.]+) netmask ([\w.]+) broadcast ([\d\.]+)', ifconfig_output, re.S | re.M)

    for interface in result:
        if (interface[1] in skip_ip_list):
            continue
        if_name = interface[0]
        ip_addr = interface[1]
        net_mask = dec_netmask(interface[2])
        broad_cast = interface[3]
        r_dns = do_rDNS(interface[1])

        ip_lst = []
        ip_lst.append(("IP", ip_addr))
        print(ip_lst)

    my_dict['nodeinfo']['ip'].append(ip_lst)

[('IP', '172.17.10.147')]
[('IP', '172.16.88.57')]
[('IP', '172.16.12.147')]
Traceback (most recent call last):
  File "/tmp/aix_reg_client_json.py", line 335, in <module>
    reg_client_runner()
  File "/tmp/aix_reg_client_json.py", line 284, in reg_client_runner
    my_dict['nodeinfo']['ip'].append(ip_lst)
AttributeError: 'collections.defaultdict' object has no attribute 'append'

[naughtyCat]

you mean like this?

entries = []
for key, value in sample['nodeinfo'].items():
    if isinstance(value, dict):
        entries.append([key, list(value.values())])
    else:
        entries.append([key, value])

for i in entries:
    print(*i)

Output:
CLUSTER_NAME JBOSS4
CLUSTER_NODENAME sag01147
CLUSTER_NODES kug01147,sag01147
CLUSTER_RGS JBOSS33,JBOSS34,JBOSS37,JBOSS38
CL_RGS JBOSS38
CPU_ENTC 0.60
CPU_VALUE 4
HAS_COBOL 0
HAS_JBOSS 1
HAS_ORACLE 0
HAS_SAP 0
HAS_TUXEDO 0
HAS_UC4 0
HA_LEVEL 7.2.4 SP2
IP ['172.17.10.147', '172.16.88.57', '172.16.12.147']
IP_LONG ['172.17.10.147/255.255.248.0/172.17.15.255/', '172.16.88.57/255.255.192.0/172.16.127.255/sag01147-ipl.some.domain', '172.16.12.147/255.255.254.0/172.16.13.255/']
IS_CLUSTER 1
IS_LPM 0
IS_ZPI 0
IS_ZPV 0
Java6 6.0.0.655
Java6_64 6.0.0.655
Java7 7.0.0.665
Java7_64 7.0.0.665
Java8_64 8.0.0.610
LOCATION Geiselberg
MACHINE_SERIAL XXXXXX
MAN_SYS_NAME GBG02
MEM_VALUE 61440
OS_LEVEL 7200-04-03-2038
STORAGE_VALUE 258
UPTIME 54
YUM_REPO IBM

[wardancer84]

more like this if possible...

[('IP', '172.17.10.147'), ('IP', '172.16.88.57'), ('IP', '172.16.12.147')]

[bowlofred]

Why the "5" in the defaultdict factory? That means the 3rd level down is a dict, not a list. So you can't append to it.

[wardancer84]

good point, maybe i should redo the whole json construction part...

#

Why the "5" in the defaultdict factory? That means the 3rd level down is a dict, not a list. So you can't append to it.

[wardancer84]

ok got it...thank you for pushing me in the right direction i do this for self educational purposes to make me
a little bit mor python fluent, so sorry for some potential stupid questions.

 #
    # IP's ..
    #
    my_dict = nested_defaultdict(list, 2) 
    ifconfig_output = subprocess.check_output(['/usr/sbin/ifconfig', '-a']).decode('utf-8')
    result = re.findall(r'^(\S+):.*?[\s]+inet ([\d\.]+) netmask ([\w.]+) broadcast ([\d\.]+)', ifconfig_output, re.S | re.M)
    ip_lst = []

    for interface in result:
        if (interface[1] in skip_ip_list):
            continue
        if_name = interface[0]
        ip_addr = interface[1]
        net_mask = dec_netmask(interface[2])
        broad_cast = interface[3]
        r_dns = do_rDNS(interface[1])

        ip_lst.append(("IP", ip_addr))


    my_dict['nodeinfo']['ip'].append(ip_lst)
    print(ip_lst)
    print(my_dict['nodeinfo']['ip'])
        #my_dict['nodeinfo']['IP_LONG'].update({if_name : ip_addr + "/" + net_mask + "/" + broad_cast + "/" + r_dns})

'nodeinfo': {'CLUSTER_NAME': 'JBOSS4',
              ...
              'YUM_REPO': 'IBM',
              'ip': [[['IP', '172.17.10.147'],
                      ['IP', '172.16.88.57'],
                      ['IP', '172.16.12.147']]]},