How can I improve my paw detection? How can I improve my paw detection? python python

How can I improve my paw detection?


If you're just wanting (semi) contiguous regions, there's already an easy implementation in Python: SciPy's ndimage.morphology module. This is a fairly common image morphology operation.


Basically, you have 5 steps:

def find_paws(data, smooth_radius=5, threshold=0.0001):    data = sp.ndimage.uniform_filter(data, smooth_radius)    thresh = data > threshold    filled = sp.ndimage.morphology.binary_fill_holes(thresh)    coded_paws, num_paws = sp.ndimage.label(filled)    data_slices = sp.ndimage.find_objects(coded_paws)    return object_slices
  1. Blur the input data a bit to make sure the paws have a continuous footprint. (It would be more efficient to just use a larger kernel (the structure kwarg to the various scipy.ndimage.morphology functions) but this isn't quite working properly for some reason...)

  2. Threshold the array so that you have a boolean array of places where the pressure is over some threshold value (i.e. thresh = data > value)

  3. Fill any internal holes, so that you have cleaner regions (filled = sp.ndimage.morphology.binary_fill_holes(thresh))

  4. Find the separate contiguous regions (coded_paws, num_paws = sp.ndimage.label(filled)). This returns an array with the regions coded by number (each region is a contiguous area of a unique integer (1 up to the number of paws) with zeros everywhere else)).

  5. Isolate the contiguous regions using data_slices = sp.ndimage.find_objects(coded_paws). This returns a list of tuples of slice objects, so you could get the region of the data for each paw with [data[x] for x in data_slices]. Instead, we'll draw a rectangle based on these slices, which takes slightly more work.


The two animations below show your "Overlapping Paws" and "Grouped Paws" example data. This method seems to be working perfectly. (And for whatever it's worth, this runs much more smoothly than the GIF images below on my machine, so the paw detection algorithm is fairly fast...)

Overlapping PawsGrouped Paws


Here's a full example (now with much more detailed explanations). The vast majority of this is reading the input and making an animation. The actual paw detection is only 5 lines of code.

import numpy as npimport scipy as spimport scipy.ndimageimport matplotlib.pyplot as pltfrom matplotlib.patches import Rectangledef animate(input_filename):    """Detects paws and animates the position and raw data of each frame    in the input file"""    # With matplotlib, it's much, much faster to just update the properties    # of a display object than it is to create a new one, so we'll just update    # the data and position of the same objects throughout this animation...    infile = paw_file(input_filename)    # Since we're making an animation with matplotlib, we need     # ion() instead of show()...    plt.ion()    fig = plt.figure()    ax = fig.add_subplot(111)    fig.suptitle(input_filename)    # Make an image based on the first frame that we'll update later    # (The first frame is never actually displayed)    im = ax.imshow(infile.next()[1])    # Make 4 rectangles that we can later move to the position of each paw    rects = [Rectangle((0,0), 1,1, fc='none', ec='red') for i in range(4)]    [ax.add_patch(rect) for rect in rects]    title = ax.set_title('Time 0.0 ms')    # Process and display each frame    for time, frame in infile:        paw_slices = find_paws(frame)        # Hide any rectangles that might be visible        [rect.set_visible(False) for rect in rects]        # Set the position and size of a rectangle for each paw and display it        for slice, rect in zip(paw_slices, rects):            dy, dx = slice            rect.set_xy((dx.start, dy.start))            rect.set_width(dx.stop - dx.start + 1)            rect.set_height(dy.stop - dy.start + 1)            rect.set_visible(True)        # Update the image data and title of the plot        title.set_text('Time %0.2f ms' % time)        im.set_data(frame)        im.set_clim([frame.min(), frame.max()])        fig.canvas.draw()def find_paws(data, smooth_radius=5, threshold=0.0001):    """Detects and isolates contiguous regions in the input array"""    # Blur the input data a bit so the paws have a continous footprint     data = sp.ndimage.uniform_filter(data, smooth_radius)    # Threshold the blurred data (this needs to be a bit > 0 due to the blur)    thresh = data > threshold    # Fill any interior holes in the paws to get cleaner regions...    filled = sp.ndimage.morphology.binary_fill_holes(thresh)    # Label each contiguous paw    coded_paws, num_paws = sp.ndimage.label(filled)    # Isolate the extent of each paw    data_slices = sp.ndimage.find_objects(coded_paws)    return data_slicesdef paw_file(filename):    """Returns a iterator that yields the time and data in each frame    The infile is an ascii file of timesteps formatted similar to this:    Frame 0 (0.00 ms)    0.0 0.0 0.0    0.0 0.0 0.0    Frame 1 (0.53 ms)    0.0 0.0 0.0    0.0 0.0 0.0    ...    """    with open(filename) as infile:        while True:            try:                time, data = read_frame(infile)                yield time, data            except StopIteration:                breakdef read_frame(infile):    """Reads a frame from the infile."""    frame_header = infile.next().strip().split()    time = float(frame_header[-2][1:])    data = []    while True:        line = infile.next().strip().split()        if line == []:            break        data.append(line)    return time, np.array(data, dtype=np.float)if __name__ == '__main__':    animate('Overlapping paws.bin')    animate('Grouped up paws.bin')    animate('Normal measurement.bin')

Update: As far as identifying which paw is in contact with the sensor at what times, the simplest solution is to just do the same analysis, but use all of the data at once. (i.e. stack the input into a 3D array, and work with it, instead of the individual time frames.) Because SciPy's ndimage functions are meant to work with n-dimensional arrays, we don't have to modify the original paw-finding function at all.

# This uses functions (and imports) in the previous code example!!def paw_regions(infile):    # Read in and stack all data together into a 3D array    data, time = [], []    for t, frame in paw_file(infile):        time.append(t)        data.append(frame)    data = np.dstack(data)    time = np.asarray(time)    # Find and label the paw impacts    data_slices, coded_paws = find_paws(data, smooth_radius=4)    # Sort by time of initial paw impact... This way we can determine which    # paws are which relative to the first paw with a simple modulo 4.    # (Assuming a 4-legged dog, where all 4 paws contacted the sensor)    data_slices.sort(key=lambda dat_slice: dat_slice[2].start)    # Plot up a simple analysis    fig = plt.figure()    ax1 = fig.add_subplot(2,1,1)    annotate_paw_prints(time, data, data_slices, ax=ax1)    ax2 = fig.add_subplot(2,1,2)    plot_paw_impacts(time, data_slices, ax=ax2)    fig.suptitle(infile)def plot_paw_impacts(time, data_slices, ax=None):    if ax is None:        ax = plt.gca()    # Group impacts by paw...    for i, dat_slice in enumerate(data_slices):        dx, dy, dt = dat_slice        paw = i%4 + 1        # Draw a bar over the time interval where each paw is in contact        ax.barh(bottom=paw, width=time[dt].ptp(), height=0.2,                 left=time[dt].min(), align='center', color='red')    ax.set_yticks(range(1, 5))    ax.set_yticklabels(['Paw 1', 'Paw 2', 'Paw 3', 'Paw 4'])    ax.set_xlabel('Time (ms) Since Beginning of Experiment')    ax.yaxis.grid(True)    ax.set_title('Periods of Paw Contact')def annotate_paw_prints(time, data, data_slices, ax=None):    if ax is None:        ax = plt.gca()    # Display all paw impacts (sum over time)    ax.imshow(data.sum(axis=2).T)    # Annotate each impact with which paw it is    # (Relative to the first paw to hit the sensor)    x, y = [], []    for i, region in enumerate(data_slices):        dx, dy, dz = region        # Get x,y center of slice...        x0 = 0.5 * (dx.start + dx.stop)        y0 = 0.5 * (dy.start + dy.stop)        x.append(x0); y.append(y0)        # Annotate the paw impacts                 ax.annotate('Paw %i' % (i%4 +1), (x0, y0),              color='red', ha='center', va='bottom')    # Plot line connecting paw impacts    ax.plot(x,y, '-wo')    ax.axis('image')    ax.set_title('Order of Steps')

alt text


alt text


alt text


I'm no expert in image detection, and I don't know Python, but I'll give it a whack...

To detect individual paws, you should first only select everything with a pressure greater than some small threshold, very close to no pressure at all. Every pixel/point that is above this should be "marked." Then, every pixel adjacent to all "marked" pixels becomes marked, and this process is repeated a few times. Masses that are totally connected would be formed, so you have distinct objects. Then, each "object" has a minimum and maximum x and y value, so bounding boxes can be packed neatly around them.

Pseudocode:

(MARK) ALL PIXELS ABOVE (0.5)

(MARK) ALL PIXELS (ADJACENT) TO (MARK) PIXELS

REPEAT (STEP 2) (5) TIMES

SEPARATE EACH TOTALLY CONNECTED MASS INTO A SINGLE OBJECT

MARK THE EDGES OF EACH OBJECT, AND CUT APART TO FORM SLICES.

That should about do it.


Note: I say pixel, but this could be regions using an average of the pixels. Optimization is another issue...

Sounds like you need to analyze a function (pressure over time) for each pixel and determine where the function turns (when it changes > X in the other direction it is considered a turn to counter errors).

If you know at what frames it turns, you will know the frame where the pressure was the most hard and you will know where it was the least hard between the two paws. In theory, you then would know the two frames where the paws pressed the most hard and can calculate an average of those intervals.

after which I'll get to the problem of deciding which paw it is!

This is the same tour as before, knowing when each paw applies the most pressure helps you decide.