In general, you need to have a parsing convention for your polygon. Once you adopt this for all your polygons, the answer is immediate. For instance, the following picture illustrates “when parsing the list of points, the inside of the polygon is always on the right”:
Now what if you don't have this information yet? Here is one simple technique:
- find the leftmost point of the polygon, call it
A = P[n]; if there are several such points, pick the uppermost
- pick the previous point and the next point in the list,
B = P[n-1] and
C = P[n+1]
- compute the z coordinate of cross product
AB × AC, aka.
z = (B.x - A.x) * (C.y - A.y) - (C.x - A.x) * (B.y - A.y)
z is positive, the inside of the polygon is on the right when parsing the list of points.
z is negative, the inside of the polygon is on the left when parsing the list of points.
Note that the above needs only to be done once per polygon.
Now when you travel from
B, a simple vector pointing to the left of the path is
(A.y - B.y, B.x - A.x).