Piping stream from busboy to request post
The problem here is that you need to provide 'Content-Length' for the multipart upload manually, because request
(and underlying form-data
) can't figure it out by themselves.So request sends invalid Content-Length: 199 (the same for any incoming file size), which breaks the java multipart parser.
There are multiple workarounds:
1) Use incoming request 'Content-Length'
request.post({ url: server.baseURL + 'api/data', formData: { file: { value: fileStream, options: { knownLength: req.headers['content-length'] } } }}, function (err, r, body) { // Do rendering stuff, handle callback})
This will produce a bit incorrect request though, because incoming length includes other upload fields and boundaries, but busboy was able to parse it w/o any complaints
2) Wait until file is completely buffered by the node app then send it to java
var concat = require('concat-stream')req.busboy.on('file', function (fieldName, fileStream, fileName, encoding, mimeType) { fileStream.pipe(concat(function (fileBuffer) { request.post({ url: server.baseURL + 'api/data', formData: { file: fileBuffer } }, function (err, r, body) { // Do rendering stuff, handle callback }) }))})
This will increase app memory consumption, so you needed to be careful and consider using busboy limits
3) Buffer file to disk before uploading (just for the reference)
- express + multer - I recommend using express for webservers, it makes things more manageable, and multer is based on the busboy
- formidable
If possible, send a custom header with exactly size file (bytes). The header always can be read before handle payload stream. Use this instead of content-length header of previous answer, because that sometimes doesn't works (with small files, i guess, but i cannot ensure that works with large files).
For the answer of Afanasii Kurakin
request.post({ url: server.baseURL + 'api/data', formData: { file: { value: fileStream, options: { knownLength: req.headers['content-length'] } } }}, function (err, r, body) { // Do rendering stuff, handle callback})
You should change from req.headers['content-length']
to the real file size, normally the content-length from header is bigger than the file size. I got pain because of the content-length and after using the file size, everything worked perfectly.