MongoError when uploading a file using mongoose, gridfs-stream and multer

The problem lies in the fact that you aren't piping just a file request to gridfs-stream. Using Multer as a middleware with catch any multipart/form-data post request. As the stream comes in, Multer (built on Busboy) watches for the on('field') and on('file') events and parses accordingly. What you are piping to Multer is not just a file.

This bit of code works fine because Multer has indeed parsed out for you req.files and req.body by this point:

   // Create a gridfs-stream   var gfs = new Gridfs(db, mongoDriver);   var file = req.files.myFile;   var fileId = new ObjectId();   console.log("Creating WriteStream");   var writeStream = gfs.createWriteStream({        _id: fileId,        filename: file.originalname,        mode: 'w',        content_type: file.mimetype,        metadata: {            id: '123',            number: '2',            name: "Kenny Erasmuson"        }     });     console.log("Created WriteStream");

But where your code runs into problems is here for reasons mentioned above:

 req.pipe(writeStream);

I have yet to find a way to stream a multipart/form-data post request with more than just file uploads straight to GridFS. If your post request doesn't contain anything other than the file (meaning no other html input fields in the form), then you might want to consider removing Multer from the middleware at least for this route.

For my use case, I require the ability to receive an html form post with text inputs along with the file upload (I store metadata about the file on upload). Here is how I accomplished what I needed with Multer:

var uploadImg = function(req,res) {      var writestream = gfs.createWriteStream({        filename: req.files.file.name,        mode:'w',        content_type:req.files.file.mimetype,        metadata:req.body,      });    fs.createReadStream(req.files.file.path).pipe(writestream);    writestream.on('close', function (file) {        res.send("Success!");        fs.unlink(req.files.file.path, function (err) {          if (err) console.error("Error: " + err);          console.log('successfully deleted : '+ req.files.file.path );        });    });};

By default, Multer will store your files on disk. One quick solution is to just create a readstream and stream it right back into GridFS. Once the writing is completed, remove the tmp file.

As a side note, there seems to be some thought going around that it's best to use Multer as middleware only on the routes that need it. You can read more on that thought on the last section of this.

Update

I think I am close to finding a way to stream directly to GridFS with metadata by use of Skipper. I'm in the process of seeing if I can't get some updates going to skipper-gridfs that will take the text inputs on the html form and set those as metadata, etc. The tweak to the skipper-gridfs seems pretty minor. I'll update when that gets flushed out. If you look at skipper be sure you recognize the order of your html form inputs (if you have any) does matter.

In my code:

var fileId = new ObjectId();console.log("Creating WriteStream");var writeStream = gfs.createWriteStream({    _id: fileId,    filename: file.originalname,    mode: 'w',    content_type: file.mimetype,    metadata: {        id: '123',        number: '2',        name: "Kenny Erasmuson"    } });

I am assigning an ObjectId rather than the string representation of an ObjectId to the _id property of the object passed into gfs.createWriteStream. It turns out this is causing the MongoError in my code.

The fix, discovered here, is to change the line of code causing the issue to be:

_id: fileId.str,

Having done that I then came to the issue of the file not being put into the fs.chunks mongodb collection although the metadata does end up in the fs.files mongodb collection. AddieD's answer here starts to address this :)