Hello,
I am working on a Python script to do some image processing and Optical Character Recognition of shipping containers. I'm trying to get a 4-sided shape (rectangle or nearly-rectangular trapezoid, depending on perspective) that represents the back of the container. I've written a color-filter/mask routine that is pretty good at isolating the container but it doesn't always yield that nice 4-sided shape I'm after. More often, it yields a bunch of these little (light blue) boxes:
(ignore green, black, and dark blue lines)
if I could stitch together:
You can see my best result above, the yellow box. It is generated by taking the left-most, top-most, right-most, and bottom-most coordinates of all the little blue boxes, but it makes a perfect square, which is not what I need. I need that trapezoid so that I can turn it into a square (de-skew/anti-keystone) and remove the perspective error from the image.
When I try to make the code look at top-most/left-most, top-most/right-most, bottom-most/right-most, bottom-most/left-most coordinates instead of just left-most, top-most, right-most, and bottom-most, I get this (see the yellow "box," fallen so far from near-glory; if you can't see it, it's a long, skinny yellow triangle along the bottom of the container):
I spent a lot of time going over my code but ultimately decided it was all correct, but the principle behind it is flawed.
Below is the code which generated that messed up triangle if you're interested, but I don't think you need to understand Python or sift through my bad programming to give feedback here.
I'm looking for help on the mathematical aspect of the problem and the code is something I can figure out once I understand the math part. But, if it's easier to illustrate the answer with counter-code, I'll take whatever help I can get.
Thanks!
I am working on a Python script to do some image processing and Optical Character Recognition of shipping containers. I'm trying to get a 4-sided shape (rectangle or nearly-rectangular trapezoid, depending on perspective) that represents the back of the container. I've written a color-filter/mask routine that is pretty good at isolating the container but it doesn't always yield that nice 4-sided shape I'm after. More often, it yields a bunch of these little (light blue) boxes:
(ignore green, black, and dark blue lines)
if I could stitch together:
- the top left corner of the top-most/left-most box
- the top right corner of the top-most/right-most box
- the bottom right corner of the bottom-most/right-most box
- the bottom left corner of the bottom-most/left-most box
You can see my best result above, the yellow box. It is generated by taking the left-most, top-most, right-most, and bottom-most coordinates of all the little blue boxes, but it makes a perfect square, which is not what I need. I need that trapezoid so that I can turn it into a square (de-skew/anti-keystone) and remove the perspective error from the image.
When I try to make the code look at top-most/left-most, top-most/right-most, bottom-most/right-most, bottom-most/left-most coordinates instead of just left-most, top-most, right-most, and bottom-most, I get this (see the yellow "box," fallen so far from near-glory; if you can't see it, it's a long, skinny yellow triangle along the bottom of the container):
I spent a lot of time going over my code but ultimately decided it was all correct, but the principle behind it is flawed.
Below is the code which generated that messed up triangle if you're interested, but I don't think you need to understand Python or sift through my bad programming to give feedback here.
I'm looking for help on the mathematical aspect of the problem and the code is something I can figure out once I understand the math part. But, if it's easier to illustrate the answer with counter-code, I'll take whatever help I can get.
Python:
# Coordinates are pixels [x,y], as measured left to right and top to bottom.
# [0,0] = top left, [3024,4032] = bottom right of image
# Rectangles are specified starting at top-left, to top-right, to bottom-right, to bottom left
# Top left point starts out at bottom right
bigBox_x1 = image.shape[1] # (width of image (3024))
bigBox_y1 = image.shape[0] # (height of image (4032))
# Top right point starts out at bottom left
bigBox_x2 = 0
bigBox_y2 = image.shape[0] # (height of image (4032))
# bottom right point starts out at top left
bigBox_x3 = 0
bigBox_y3 = 0
# bottom left point starts out at top right
bigBox_x4 = image.shape[1] # (width of image (3024))
bigBox_y4 = 0
bigBox = np.int0([[bigBox_x1,bigBox_y1],[bigBox_x2,bigBox_y2],[bigBox_x3,bigBox_y3],[bigBox_x4,bigBox_y4]])
for smallBox in boxList:
print(smallBox)
# big box is stretching up and to the left
# evaluate upper left corner of small box.
# is it Left-er than than current top-left?
# Is it higher than current top-left?
if smallBox[0][0] < bigBox[0][0]: # bigBox_x1 starts at image width (3024) and gets incrementally decreased
# Ok, it's left-er than current top left. But is it higher than current top left?
if smallBox[0][1] < bigBox[0][1]:# bigBox_y1 starts at image height (4032) and gets incrementally decreased
# if so, set its top-left as our new top-left:
bigBox[0][0] = smallBox[0][0]
bigBox[0][1] = smallBox[0][1]
print("new top left:",[bigBox[0][0],bigBox[0][1]])
# big box is stretching down and to the left.
# evaluate lower left corner of small box.
# is it Left-er than than current bottom-left?
# Is it lower than current bottom-left?
if smallBox[3][0] < bigBox[3][0]:# bigBox_x4 starts at image width (3024) and gets incrementally decreased
# Ok, it's left-er than current bottom left. But is it lower than current bottom left?
if smallBox[3][1] > bigBox[3][1]:# bigBox_y4 starts at 0 and gets incrementally increased
# if so, set its bottom-left as our new bottom-left:
bigBox[3][1] = smallBox[3][1]
bigBox[3][0] = smallBox[3][0]
print("new bottom left:",[bigBox[3][0],bigBox[3][1]])
# big box is stretching up and to the right.
# evaluate upper right corner of small box.
# is it right-er than than current top-right?
# Is it higher than current top-right?
if smallBox[1][0] > bigBox[1][0]:# bigBox_x2 starts at 0 and gets incrementally increased
# Ok, it's right-er than current top right. But is it higher than current top right?
if smallBox[1][1] < bigBox[1][1]:# bigBox_y2 starts at image height (4032) and gets incrementally decreased
# if so, set its top-right as our new top-right:
bigBox[1][1] = smallBox[1][1]
bigBox[1][0] = smallBox[1][0]
print("new top right:",[bigBox[1][0],bigBox[1][1]])
# big box is stretching down and to the right.
# evaluate lower right corner of small box.
# is it right-er than than current bottom-right?
# Is it lower than current bottom-right?
if smallBox[2][0] > bigBox[2][0] :# bigBox_x3 starts at 0 and gets incrementally increased
# Ok, it's right-er than current bottom right. But is it lower than current bottom right?
# if so, set its bottom-right as our new bottom-right:
if smallBox[2][1] > bigBox[2][1]:# bigBox_y3 starts at 0 and gets incrementally increased
bigBox[2][1] = smallBox[2][1]
bigBox[2][0] = smallBox[2][0]
print("new bottom right:",[bigBox[2][0],bigBox[2][1]])
print("\n")
Thanks!