HTML5 video of type video/mp4 playing audio only
Turns out the video was not properly encoded to be supported by HTML5 video tag.
HTML5 video requires MP4 videos with H264 video codec and AAC audio codec.
The problem video was not of codec H264. Running it through a converter fixed it.