How do I make my JSON less verbose? How do I make my JSON less verbose? ajax ajax

How do I make my JSON less verbose?


HTTP compression (i.e. gzip or deflate) already does exactly that. Repeated patterns, like your JSON keys, are replaced with tokens so that the verbose pattern only has to occur once per transmission.


Not an answer, but to give a rough estimate of "savings" based on 10k entries and some bogus data :-) This is in response to a comment I posted. Will the added complexity make the schema'ized approach worth it?

"It depends."

This C# is LINQPad and is ready-to-go for testing/modifying:

string LongTemplate (int n1, int n2, int n3, string name) {    return string.Format(@"            {{                ""codePractice"": {0},                ""codeScheduleObject"": {1},                ""codeScheduleObjectType"": """",                ""defaultCodeScheduleObject"": {2},                ""name"": ""Dr. {3}""            }}," + "\n", n1, n2, n3, name);}string ShortTemplate (int n1, int n2, int n3, string name) {    return string.Format("[{0}, {1}, \"\", {2}, \"Dr. {3}\"],\n",        n1, n2, n3, name);}string MinTemplate (int n1, int n2, int n3, string name) {    return string.Format("[{0},{1},\"\",{2},\"Dr. {3}\"],",        n1, n2, n3, name);}long GZippedSize (string s) {    var ms = new MemoryStream();    using (var gzip = new System.IO.Compression.GZipStream(ms, System.IO.Compression.CompressionMode.Compress, true))    using (var sw = new StreamWriter(gzip)) {        sw.Write(s);    }    return ms.Position;}void Main(){    var r = new Random();    var l = new StringBuilder();    var s = new StringBuilder();    var m = new StringBuilder();    for (int i = 0; i < 10000; i++) {        var n1 = r.Next(10000);        var n2 = r.Next(10000);        var n3 = r.Next(10000);        var name = "bogus" + r.Next(50);        l.Append(LongTemplate(n1, n2, n3, name));        s.Append(ShortTemplate(n1, n2, n3, name));        m.Append(MinTemplate(n1, n2, n3, name));    }    var lc = GZippedSize(l.ToString());    var sc = GZippedSize(s.ToString());    var mc = GZippedSize(s.ToString());    Console.WriteLine(string.Format("Long:\tNormal={0}\tGZip={1}\tCompressed={2:P}", l.Length, lc, (float)lc / l.Length));    Console.WriteLine(string.Format("Short:\tNormal={0}\tGZip={1}\tCompressed={2:P}", s.Length, sc, (float)sc / s.Length));    Console.WriteLine(string.Format("Min:\tNormal={0}\tGZip={1}\tCompressed={2:P}", m.Length, mc, (float)mc / m.Length));    Console.WriteLine(string.Format("Short/Long\tRegular={0:P}\tGZip={1:P}",        (float)s.Length / l.Length, (float)sc / lc));    Console.WriteLine(string.Format("Min/Long\tRegular={0:P}\tGZip={1:P}",        (float)m.Length / l.Length, (float)mc / lc));}

My results:

Long:  Normal=1754614  GZip=197053  Compressed=11.23 %Short:  Normal=384614  GZip=128252  Compressed=33.35 %Min:  Normal=334614  GZip=128252  Compressed=38.33 %Short/Long  Regular=21.92 %  GZip=65.09 %Min/Long  Regular=19.07 %  GZip=65.09 %

Conclusion:

  • The single biggest savings is to use GZIP (better than just using schema'ize).
  • GZIP + schema'ized will be the smallest overall.
  • With GZIP there is no point to use a normal JavaScript minimizer (in this scenario).
  • Use GZIP (e.g. DEFLATE); it performs very well on repetitive structured text (900% compression on normal!).

Happy coding.


Here's an article that does pretty much what you're looking to do:

http://stevehanov.ca/blog/index.php?id=104

At first glance, it looks like your example would be compressed down to the following after the first step of the algorithm, which will actually do more work on it in subsequent steps):

{    "templates": [         ["codePractice", "codeScheduleObject", "codeScheduleObjectType", "defaultCodeScheduleObject", "name"]     ],    "values": [         { "type": 1, "values": [ 35, 576, "", 12, "Dr. 1" ] },        { "type": 1, "values": [ 35, 169, "", 43, "Dr. 2" ] },        { "type": 1, "values": [ 35, 959, "", 76, "Dr. 3" ] }    ]}

You can start to see the benefit of the algorithm already. Here's the final output after running it through the compressor:

{    "f" : "cjson",    "t" : [              [0,"schedules"],              [0,"codePractice","codeScheduleObject","codeScheduleObjectType","defaultCodeScheduleObject","name"]          ],    "v" : {        "" : [ 1, [                { "" : [2, 35, 576, "", 12, "Dr. 1"] },                { "" : [2, 35, 169, "", 43, "Dr. 2"] },                { "" : [2, 35, 959, "", 76, "Dr. 3"] }            ]        ]    }}

One can obviously see the improvement if you have several thousands of records. The output is still readable, but I think the other guys are right too: a good compression algorithm is going to remove the blocks of text that are repeated anyway...