group in mongo excluding null values

You need an extra $match pipeline step that will filter the incoming documents based on the embedded field "$productAttribute.colour" existing and not null:

    db.productMetadata.aggregate([    {         "$match": {            "productAttribute.colour": {                 "$exists": true,                 "$ne": null             }        }        },    {         "$group": {            "_id": {                "color": "$productAttribute.colour",                "gender": "$productAttribute.gender"            },            "count": {                 "$sum": 1             }        }       }        ]);

Perhaps you should use $match: {'color': {$exists: true}} before $group operation. With sparse index it will work pretty fast.And do not store "null" fields in collections at all, that will reduce db size and will increase search speed for sparse indexes (less documents in index -> more speed)

this example includes two different Collections. For this we use aggregate function. I am also using Mongoose

1. I am joining the cusmtomfield with customfiellabels with $lookup
2. Flat the array with $unwind
3. $match to exclude the name that have INACTIVE in the text (I'm using REGEX)

4. $project to rename the Fields to show properly on the client

   . async getAllMasterDataCustomFields(req) {

       let response = {};    try {      response = await customfieldsModel.aggregate([        {          $lookup: {            from: 'customfieldlabels',            localField: 'cfId',            foreignField: 'cfId',            as: 'info'          }        },        { '$unwind': { 'path': '$info', 'preserveNullAndEmptyArrays': true } },        { '$match': { 'childs.name': { $not: /INACTIVE/ }}},        {          $project: {            'cfId': 1,            'label': '$info.label',            'type': '$info.type',            'childs': 1          }        }]).exec();    } catch (e) {      logger.log('error', `Error while getting response ${e.meesage}`);    }    return response;  }

   .