Split text lines in scanned document Split text lines in scanned document python python

Split text lines in scanned document


From your input image, you need to make text as white, and background as black

enter image description here

You need then to compute the rotation angle of your bill. A simple approach is to find the minAreaRect of all white points (findNonZero), and you get:

enter image description here

Then you can rotate your bill, so that text is horizontal:

enter image description here

Now you can compute horizontal projection (reduce). You can take the average value in each line. Apply a threshold th on the histogram to account for some noise in the image (here I used 0, i.e. no noise). Lines with only background will have a value >0, text lines will have value 0 in the histogram. Then take the average bin coordinate of each continuous sequence of white bins in the histogram. That will be the y coordinate of your lines:

enter image description here

Here the code. It's in C++, but since most of the work is with OpenCV functions, it should be easy convertible to Python. At least, you can use this as a reference:

#include <opencv2/opencv.hpp>using namespace cv;using namespace std;int main(){    // Read image    Mat3b img = imread("path_to_image");    // Binarize image. Text is white, background is black    Mat1b bin;    cvtColor(img, bin, COLOR_BGR2GRAY);    bin = bin < 200;    // Find all white pixels    vector<Point> pts;    findNonZero(bin, pts);    // Get rotated rect of white pixels    RotatedRect box = minAreaRect(pts);    if (box.size.width > box.size.height)    {        swap(box.size.width, box.size.height);        box.angle += 90.f;    }    Point2f vertices[4];    box.points(vertices);    for (int i = 0; i < 4; ++i)    {        line(img, vertices[i], vertices[(i + 1) % 4], Scalar(0, 255, 0));    }    // Rotate the image according to the found angle    Mat1b rotated;    Mat M = getRotationMatrix2D(box.center, box.angle, 1.0);    warpAffine(bin, rotated, M, bin.size());    // Compute horizontal projections    Mat1f horProj;    reduce(rotated, horProj, 1, CV_REDUCE_AVG);    // Remove noise in histogram. White bins identify space lines, black bins identify text lines    float th = 0;    Mat1b hist = horProj <= th;    // Get mean coordinate of white white pixels groups    vector<int> ycoords;    int y = 0;    int count = 0;    bool isSpace = false;    for (int i = 0; i < rotated.rows; ++i)    {        if (!isSpace)        {            if (hist(i))            {                isSpace = true;                count = 1;                y = i;            }        }        else        {            if (!hist(i))            {                isSpace = false;                ycoords.push_back(y / count);            }            else            {                y += i;                count++;            }        }    }    // Draw line as final result    Mat3b result;    cvtColor(rotated, result, COLOR_GRAY2BGR);    for (int i = 0; i < ycoords.size(); ++i)    {        line(result, Point(0, ycoords[i]), Point(result.cols, ycoords[i]), Scalar(0, 255, 0));    }    return 0;}


Basic steps as @Miki,

  1. read the source
  2. threshed
  3. find minAreaRect
  4. warp by the rotated matrix
  5. find and draw upper and lower bounds

enter image description here


While code in Python:

#!/usr/bin/python3# 2018.01.16 01:11:49 CST# 2018.01.16 01:55:01 CSTimport cv2import numpy as np## (1) readimg = cv2.imread("img02.jpg")gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)## (2) thresholdth, threshed = cv2.threshold(gray, 127, 255, cv2.THRESH_BINARY_INV|cv2.THRESH_OTSU)## (3) minAreaRect on the nozerospts = cv2.findNonZero(threshed)ret = cv2.minAreaRect(pts)(cx,cy), (w,h), ang = retif w>h:    w,h = h,w    ang += 90## (4) Find rotated matrix, do rotationM = cv2.getRotationMatrix2D((cx,cy), ang, 1.0)rotated = cv2.warpAffine(threshed, M, (img.shape[1], img.shape[0]))## (5) find and draw the upper and lower boundary of each lineshist = cv2.reduce(rotated,1, cv2.REDUCE_AVG).reshape(-1)th = 2H,W = img.shape[:2]uppers = [y for y in range(H-1) if hist[y]<=th and hist[y+1]>th]lowers = [y for y in range(H-1) if hist[y]>th and hist[y+1]<=th]rotated = cv2.cvtColor(rotated, cv2.COLOR_GRAY2BGR)for y in uppers:    cv2.line(rotated, (0,y), (W, y), (255,0,0), 1)for y in lowers:    cv2.line(rotated, (0,y), (W, y), (0,255,0), 1)cv2.imwrite("result.png", rotated)

Finally result:

enter image description here