Re-structuring A List Of Python Dicts Using Setdefault

I am trying to re-structure a list of Python dictionaries by 'grouping' (that's probably not the correct expression, but using it as a proxy based on SQL) the dictionaries based on

Solution 1:

Here are some hints:

The first thing is to sort by selection_id:

by_selection_id = operator.itemgetter('selection_id')
market.sort(key=by_selection_id)

Then you can group by selection_id:

for selection_id, group in itertools.groupby(market, key=by_selection_id):
    print(selection_id, list(group))

You get:

(1097, [{'value_name': 'b', 'value': '39', 'selection_id': 1097},
        {'value_name': 'f', 'value': '52', 'selection_id': 1097}])
(1098, [{'value_name': 'd', 'value': '98', 'selection_id': 1098},
        {'value_name': 'r', 'value': '4', 'selection_id': 1098}])
(1099, [{'value_name': 'a', 'value': '11', 'selection_id': 1099},
        {'value_name': 'p', 'value': '78', 'selection_id': 1099},
        {'value_name': 'y', 'value': '13', 'selection_id': 1099}])

Then it is easy to build the final list.

Here is a solution using comprehension list/dict:

new_structure = [{'selection_id': selection_id,
                  'value_dict': [{'value': item['value'],
                                  'value_name': item['value_name']} for item in group]}
                 for selection_id, group in itertools.groupby(market, key=by_selection_id)]

Or using a classic list with append:

new_structure = []
for selection_id, group in itertools.groupby(market, key=by_selection_id):
    value_dict = [{'value': item['value'], 'value_name': item['value_name']} for item in group]
    new_structure.append({'selection_id': selection_id,
                          'value_dict': value_dict})

Solution 2:

I end up with what looks like a tuple rather than a dict)

This is because you are appending a tuple, not a dict:

.append((z['value'], z['value_name']))

Solution 3:

So, just to update, based on the answer from @Code-Apprentice, I have re-written my code as follows:

market = [
    {'selection_id': 1099, 'value': '11', 'value_name': 'a'},
    {'selection_id': 1099, 'value': '78', 'value_name': 'p'},
    {'selection_id': 1097, 'value': '39', 'value_name': 'b'},
    {'selection_id': 1097, 'value': '52', 'value_name': 'f'},
    {'selection_id': 1098, 'value': '98', 'value_name': 'd'},
    {'selection_id': 1099, 'value': '13', 'value_name': 'y'},
    {'selection_id': 1098, 'value': '4', 'value_name': 'r'},
]

new_structure = {}
new_structure2 = []

for z in market:
    new_structure.setdefault(z['selection_id'], []).append({'value': z['value'],
                                                            'value_name': z['value_name']})
    t = [{'selection_id': m, 'value_dict': n} for m, n in new_structure.items()]
    new_structure2.append(t)

print(new_structure2)

Which then gets me very close to my desired output. The only remaining issue I have, is a superfluous list (i.e. [[{....}]] instead of [{...}]), I believe this is probably because of the way I have written the line t=....

Here is my output, I will post an update, once I've got rid of that extra list in case the answer is of use to anyone else (but please feel free to shout if you spot it before me):

[[{'selection_id': 1099,
   'value_dict': [{'value': '11', 'value_name': 'a'},
                  {'value': '78', 'value_name': 'p'},
                  {'value': '13', 'value_name': 'y'}]},
  {'selection_id': 1097,
   'value_dict': [{'value': '39', 'value_name': 'b'},
                  {'value': '52', 'value_name': 'f'}]},
  {'selection_id': 1098,
   'value_dict': [{'value': '98', 'value_name': 'd'},
                  {'value': '4', 'value_name': 'r'}]}]]

So ridiculously basic! I was over-complicating it, just simply removed the extra list reference:

market = [
    {'selection_id': 1099, 'value': '11', 'value_name': 'a'},
    {'selection_id': 1099, 'value': '78', 'value_name': 'p'},
    {'selection_id': 1097, 'value': '39', 'value_name': 'b'},
    {'selection_id': 1097, 'value': '52', 'value_name': 'f'},
    {'selection_id': 1098, 'value': '98', 'value_name': 'd'},
    {'selection_id': 1099, 'value': '13', 'value_name': 'y'},
    {'selection_id': 1098, 'value': '4', 'value_name': 'r'},
]

new_structure = {}

for z in market:
    new_structure.setdefault(z['selection_id'], []).append({'value': z['value'],
                                                            'value_name': z['value_name']})

new_structure2 = [{'selection_id': m, 'value_dict': n} for m, n in new_structure.items()]