Capture video data from screen in Python

There is an other solution with mss which provide much better frame rate. (Tested on a Macbook Pro with MacOS Sierra)

import numpy as npimport cv2from mss import mssfrom PIL import Imagemon = {'left': 160, 'top': 160, 'width': 200, 'height': 200}with mss() as sct:    while True:        screenShot = sct.grab(mon)        img = Image.frombytes(            'RGB',             (screenShot.width, screenShot.height),             screenShot.rgb,         )        cv2.imshow('test', np.array(img))        if cv2.waitKey(33) & 0xFF in (            ord('q'),             27,         ):            break

With all of the above solutions, I was unable to get a usable frame rate until I modified my code in the following way:

import numpy as npimport cv2from mss import mssfrom PIL import Imagebounding_box = {'top': 100, 'left': 0, 'width': 400, 'height': 300}sct = mss()while True:    sct_img = sct.grab(bounding_box)    cv2.imshow('screen', np.array(sct_img))    if (cv2.waitKey(1) & 0xFF) == ord('q'):        cv2.destroyAllWindows()        break

With this solution, I easily get 20+ frames/second.

For reference, check this link: OpenCV/Numpy example with mss

You will need to use ImageGrab from Pillow (PIL) Library and convert the capture to numpy array. When you have the array you can do what you please with it using opencv. I converted capture to gray color and used imshow() as a demonstration.

Here is a quick code to get you started:

from PIL import ImageGrabimport numpy as npimport cv2img = ImageGrab.grab(bbox=(100,10,400,780)) #bbox specifies specific region (bbox= x,y,width,height *starts top-left)img_np = np.array(img) #this is the array obtained from conversionframe = cv2.cvtColor(img_np, cv2.COLOR_BGR2GRAY)cv2.imshow("test", frame)cv2.waitKey(0)cv2.destroyAllWindows()

you can plug an array there with the frequency you please to keep capturing frames. After that you just decode the frames. don't forget to add before the loop:

fourcc = cv2.VideoWriter_fourcc(*'XVID')vid = cv2.VideoWriter('output.avi', fourcc, 6, (640,480))

and inside the loop you can add:

vid.write(frame) #the edited frame or the original img_np as you please

UPDATE
the end result look something like this (If you want to achieve a stream of frames that is. Storing as video just a demonstration of using opencv on the screen captured):

from PIL import ImageGrabimport numpy as npimport cv2while(True):    img = ImageGrab.grab(bbox=(100,10,400,780)) #bbox specifies specific region (bbox= x,y,width,height)    img_np = np.array(img)    frame = cv2.cvtColor(img_np, cv2.COLOR_BGR2GRAY)    cv2.imshow("test", frame)    cv2.waitKey(0)cv2.destroyAllWindows()

Hope that helps