How to get the latest frame from capture device (camera) in opencv

I think the solution mentioned in the question, namely having a separate thread that clears the buffer, is the easiest non-brittle solution for this. Here reasonably nice (I think) code for this:

import cv2, queue, threading, time# bufferless VideoCaptureclass VideoCapture:  def __init__(self, name):    self.cap = cv2.VideoCapture(name)    self.q = queue.Queue()    t = threading.Thread(target=self._reader)    t.daemon = True    t.start()  # read frames as soon as they are available, keeping only most recent one  def _reader(self):    while True:      ret, frame = self.cap.read()      if not ret:        break      if not self.q.empty():        try:          self.q.get_nowait()   # discard previous (unprocessed) frame        except queue.Empty:          pass      self.q.put(frame)  def read(self):    return self.q.get()cap = VideoCapture(0)while True:  time.sleep(.5)   # simulate time between events  frame = cap.read()  cv2.imshow("frame", frame)  if chr(cv2.waitKey(1)&255) == 'q':    break

The frame reader thread is encapsulated inside the custom VideoCapture class, and communication with the main thread is via a queue.

I posted very similar code for a node.js question, where a JavaScript solution would have been better. My comments on another answer to that question give details why a non-brittle solution without separate thread seems difficult.

An alternative solution that is easier but supported only for some OpenCV backends is using CAP_PROP_BUFFERSIZE. The 2.4 docs state it is "only supported by DC1394 [Firewire] v 2.x backend currently." For Linux backend V4L, according to a comment in the 3.4.5 code, support was added on 9 Mar 2018, but I got VIDEOIO ERROR: V4L: Property <unknown property string>(38) not supported by device for exactly this backend. It may be worth a try first; the code is as easy as this:

cap.set(cv2.CAP_PROP_BUFFERSIZE, 0)

On my Raspberry Pi 4,

cap = cv2.VideoCapture(0)cap.set(cv2.CAP_PROP_BUFFERSIZE, 1)

does work and was all that I needed for my pi camera to give me the latest frame, with a consistent 3+ second delay between the scene in front of the camera and displaying that scene in the preview image. My code takes 1.3 seconds to process an image, so I'm not sure why the other 2 seconds of delay are present, but it's consistent and works.

Side note: since my code takes over a second to process an image, I also added

cap.set( cv2.CAP_PROP_FPS, 2 )

in case it reduces any unneeded activity, since I can't quite get a frame a second. When I put cv2.CAP_PROP_FPS to 1, though, I got a strange output of all my frames being almost entirely dark, so setting FPS too low can cause an issue

Here's a slightly simplified version of Ulrich's solution.OpenCV's read() function combines grab() and retrieve() in one call, where grab() just grabs the next frame, and retrieve does the actual decoding of the frame (demosaicing & motion jpeg decompression). We're only interested in decoding the frame we're actually reading, so this solution will save some CPU.

import cv2import threading# bufferless VideoCaptureclass VideoCapture:    def __init__(self, name):        self.cap = cv2.VideoCapture(name)        self.t = threading.Thread(target=self._reader)        self.t.daemon = True        self.t.start()    # grab frames as soon as they are available    def _reader(self):        while True:            ret = self.cap.grab()            if not ret:                break    # retrieve latest frame    def read(self):        ret, frame = self.cap.retrieve()        return frame