Timelion series from cumulative sum of array length

I suggest to add another field called userCount to your documents so you don't need to mess with scripting (+ it'll be more performant).

So your documents should look like this:

{    dateTime: /* My time field */,    message: {        users: ['1', '2']    },    userCount: 2,                  <--- add this field    messageType: 'test'}

Solution 1:

You'd need to change your code a tiny bit to this:

elasticClient.Index(    new    {        DateTime = DateTime.Now,        Message = evt.EventArgs.Message,        UserCount = evt.EventArgs.Message.Users.Length    },    idx => idx.Index($"logstash-{evt.MessageCode}"));

Solution 2:

If you're using ES 5, you can leverage the Ingest API in order to create a pipeline that will automatically add that userCount field for you. You don't have to change anything in your code.

PUT _ingest/pipeline/user-count-pipeline{  "description" : "Creates a new userCount field",  "processors" : [    {      "script": {        "lang": "painless",        "inline": "ctx.userCount = ctx.message?.users?.size() ?: 0"      }    }  ]}

Then, in Timelion, it'll be very easy to chart what you need using metric='sum:userCount' to sum the userCount values and the cusum() function to get the cumulative sum of the userCountover time. The whole expression would look like this:

.es(index=logstash-*,timefield='dateTime',q='messageType:UserList',metric='sum:userCount').label('Users Online').cusum()

Using a few sample documents, the time series looks like this, which seems to be what you're looking for.