Mongo aggregation with paginated data and totals
If you have a lot of events, {$ push: "$$ ROOT"}, will make Mongo return an error, I have solved it with $facet (Only works with version 3.4+)
aggregate([ { $match: options }, { $facet: { edges: [ { $sort: sort }, { $skip: skip }, { $limit: limit }, ], pageInfo: [ { $group: { _id: null, count: { $sum: 1 } } }, ], }, }, ])
Did this in two steps instead of one:
// Get the totalsdb.mongoAuditEvent.aggregate([{$group: {_id: "$corrId"}}, {$group: {_id: 1, total: {$sum: 1}}}]);// Get the datadb.mongoAuditEvent.aggregate([ {$group: { _id : "$corrId", currentEvent: {"$last": "$event.status"}, "events": { $push: "$$ROOT"} }}, {$sort: {"events.timestamp": -1} }, // Latest first {$skip: 0 }, {$limit: 10}], {allowDiskUse: true}).pretty();
I would be very happy if anybody got a better solution to this though.
A performance optimization tip:
When you use $facet
stage for pagination, Try to add it as soon as it's possible.
For example: if you want to add $project
or $lookup
stage, add them after $facet
, not before it.
it will have impressive effect in aggregation speed. because $project
stage require MongoDB to explore all documents and get involve with all fields(which is not necessary).