Free memory during loop

Don't store many instance of class in array, instead do it inline. Example.

% cat test.intest1:OK,test3:OKtest1:OK,test3:OKtest1:OK,test3:OK,test4:test_again% cat test.py import jsonwith open("test.in", "rb") as src:    with open("out.out", "wb") as dst:        for line in src:            pairs, obj = [x.split(":",1) for x in line.rstrip().split(",")], {}            for k,v in pairs:                if k not in obj: obj[k] = []                obj[k].append(v)            dst.write(json.dumps(obj)+"\n")% cat out.out{"test1": ["OK"], "test3": ["OK"]}{"test1": ["OK"], "test3": ["OK"]}{"test1": ["OK"], "test3": ["OK"], "test4": ["test_again"]}

If it is slow, don't write to file line by line, but store dumped json string in array and do dst.write("\n".join(array))

In order for obj to be free-able, all references to it have to be eliminated. Your loop didn't do that because the reference in list_obj still existed. The following will fix that:

def write_to_file(list_obj):    with open("out.out", "w") as fout:        for ix in range(list_obj):            obj = list_obj[ix]            list_obj[ix] = None            json_out = obj.to_json()            fout.write(json_out + "\n")            obj.free_all()

Alternatively, you could destructively pop the element from the front of list_obj, although that could potentially cause performance problems if it has to reallocate list_obj too many times. I haven't experimented with this so I'm not really sure. That version looks like this:

def write_to_file(list_obj):    with open("out.out", "w") as fout:        while len(list_obj) > 0:            obj = list_obj.pop(0)            json_out = obj.to_json()            fout.write(json_out + "\n")            obj.free_all()