Why does repeated JSON parsing consume more and more memory? Why does repeated JSON parsing consume more and more memory? json json

Why does repeated JSON parsing consume more and more memory?


When Ruby parses a JSON file, it creates many intermediate objects to achieve the goal. These objects stays on memory until GC start working.

If the JSON file has a complicated structure, many arrays and inner objects, the number will grow fast too.

Did you try to call "GC.start" to suggest Ruby clean up unused memory? If the amount of memory decrease significantly, its suggest that is just intermediate objects used to parse the data, otherwise, your data structure is complex or there is something your data that the lib can't deallocate.

For large JSON processing I use yajl-ruby (https://github.com/brianmario/yajl-ruby). It is C implemented and has a low footprint.