Edit

FWIW: This answer serves its educational purpose by explaining what CGPathAddArcToPoint(...) does for you. I would highly recommend that you read through it as it will help you understand and appreciate the CGPath API. Then you should go ahead and use that, as seen in an0's answer, instead of this code when you round edges in your app. This code should only be used as a reference if you want to play around with and learn about geometry calculations like this.

Original answer

Because I find questions like this so fun, I had to answer it :)

This is a long answer. There is no short version :D

Note: For my own simplicity, my solution is making some assumptions about the points that are being used to form the triangle such as:

• The area of the triangle is large enough to fit the rounded corner (e.g. the height of the triangle is greater than the diameter of the circles in the corners. I'm not checking for or trying to prevent any kind of strange results that may happen otherwise.
• The corners are listed in counter clock-wise order. You could make it work for any order but it felt like a fair enough constraint to add for simplicity.

If you wanted, you could use the same technique to round any polygon as long as it's strictly convex (i.e. not a pointy star). I won't explain how to do it though but it follows the same principle.

It all starts of with a triangle, that you want to round the corners of with some radius, r:

The rounded triangle should be contained in the pointy triangle so the first step is to find the locations, as close to the corners as possible, where you can fit a circle with the radius, r.

A simple way of doing this is to create 3 new lines parallel to the 3 sides in the triangle and shift each of the the distance r inwards, orthogonal to the side of the original side.

To do this you calculate the slope/angle of each line and the offset to apply to the two new points:

CGFloat angle = atan2f(end.y - start.y,                       end.x - start.x);CGVector offset = CGVectorMake(-sinf(angle)*radius,                                cosf(angle)*radius);

Note: for clarity I'm using the CGVector type (available in iOS 7), but you can just as well use a point or a size to work with previous OS versions.

then you add the offset to both start and end points for each line:

CGPoint offsetStart = CGPointMake(start.x + offset.dx,                                  start.y + offset.dy);CGPoint offsetEnd   = CGPointMake(end.x + offset.dx,                                  end.y + offset.dy);

When you do tho you will see that the three lines intersect each other in three places:

Each intersection point is exactly the distance r from two of the sides (assuming that the triangle is large enough, as stated above).

You can calculate the intersection of two lines as:

//       (x1⋅y2-y1⋅x2)(x3-x4) - (x1-x2)(x3⋅y4-y3⋅x4)// px =  –––––––––––––––––––––––––––––––––––––––––––//            (x1-x2)(y3-y4) - (y1-y2)(x3-x4)//       (x1⋅y2-y1⋅x2)(y3-y4) - (y1-y2)(x3⋅y4-y3⋅x4)// py =  –––––––––––––––––––––––––––––––––––––––––––//            (x1-x2)(y3-y4) - (y1-y2)(x3-x4)CGFloat intersectionX = ((x1*y2-y1*x2)*(x3-x4) - (x1-x2)*(x3*y4-y3*x4)) / ((x1-x2)*(y3-y4) - (y1-y2)*(x3-x4));CGFloat intersectionY = ((x1*y2-y1*x2)*(y3-y4) - (y1-y2)*(x3*y4-y3*x4)) / ((x1-x2)*(y3-y4) - (y1-y2)*(x3-x4));CGPoint intersection = CGPointMake(intersectionX, intersectionY);

where (x1, y1) to (x2, y2) is the first line and (x3, y3) to (x4, y4) is the second line.

If you then put a circle, with the radius r, on each intersection point you can see that it will indeed for the rounded triangle (ignoring the different line widths of the triangle and the circles):

Now to create the rounded triangle you want to create a path that changes from a line to an arc to a line (etc.) on the points where the original triangle is orthogonal to the intersection points. This is also the point where the circles tangent the original triangle.

Knowing the slopes of all 3 sides in the triangle, the corner radius and the center of the circles (the intersection points), the start and stop angle for each rounded corner is the slope of that side - 90 degrees. To group these things together, I created a struct in my code, but you don't have to if you don't want to:

typedef struct {    CGPoint centerPoint;    CGFloat startAngle;    CGFloat endAngle;} CornerPoint;

To reduce code duplication I created a method for myself that calculates the intersection and the angles for one point given a line from one point, via another, to a final point (it's not closed so it's not a triangle):

The code is as follows (it's really just the code that I've shown above, put together):

- (CornerPoint)roundedCornerWithLinesFrom:(CGPoint)from                                      via:(CGPoint)via                                       to:(CGPoint)to                               withRadius:(CGFloat)radius{    CGFloat fromAngle = atan2f(via.y - from.y,                               via.x - from.x);    CGFloat toAngle   = atan2f(to.y  - via.y,                               to.x  - via.x);    CGVector fromOffset = CGVectorMake(-sinf(fromAngle)*radius,                                        cosf(fromAngle)*radius);    CGVector toOffset   = CGVectorMake(-sinf(toAngle)*radius,                                        cosf(toAngle)*radius);    CGFloat x1 = from.x +fromOffset.dx;    CGFloat y1 = from.y +fromOffset.dy;    CGFloat x2 = via.x  +fromOffset.dx;    CGFloat y2 = via.y  +fromOffset.dy;    CGFloat x3 = via.x  +toOffset.dx;    CGFloat y3 = via.y  +toOffset.dy;    CGFloat x4 = to.x   +toOffset.dx;    CGFloat y4 = to.y   +toOffset.dy;    CGFloat intersectionX = ((x1*y2-y1*x2)*(x3-x4) - (x1-x2)*(x3*y4-y3*x4)) / ((x1-x2)*(y3-y4) - (y1-y2)*(x3-x4));    CGFloat intersectionY = ((x1*y2-y1*x2)*(y3-y4) - (y1-y2)*(x3*y4-y3*x4)) / ((x1-x2)*(y3-y4) - (y1-y2)*(x3-x4));    CGPoint intersection = CGPointMake(intersectionX, intersectionY);    CornerPoint corner;    corner.centerPoint = intersection;    corner.startAngle  = fromAngle - M_PI_2;    corner.endAngle    = toAngle   - M_PI_2;    return corner;}

I then used that code 3 times to calculate the 3 corners:

CornerPoint leftCorner  = [self roundedCornerWithLinesFrom:right                                                       via:left                                                        to:top                                                withRadius:radius];CornerPoint topCorner   = [self roundedCornerWithLinesFrom:left                                                       via:top                                                        to:right                                                withRadius:radius];CornerPoint rightCorner = [self roundedCornerWithLinesFrom:top                                                       via:right                                                        to:left                                                withRadius:radius];

Now, having all the necessary data, starts the part where we create the actual path. I'm going to rely on the fact that CGPathAddArc will add a straight line from the current point to the start point to not have to draw those lines myself (this is documented behaviour).

The only point I manually have to calculate is the start point of the path. I choose the start of the lower right corner (no specific reason). From there you just add an arc with the center in the intersection points from the start and end angles:

CGMutablePathRef roundedTrianglePath = CGPathCreateMutable();// manually calculated start pointCGPathMoveToPoint(roundedTrianglePath, NULL,                  leftCorner.centerPoint.x + radius*cosf(leftCorner.startAngle),                  leftCorner.centerPoint.y + radius*sinf(leftCorner.startAngle));// add 3 arcs in the 3 corners CGPathAddArc(roundedTrianglePath, NULL,             leftCorner.centerPoint.x, leftCorner.centerPoint.y,             radius,             leftCorner.startAngle, leftCorner.endAngle,             NO);CGPathAddArc(roundedTrianglePath, NULL,             topCorner.centerPoint.x, topCorner.centerPoint.y,             radius,             topCorner.startAngle, topCorner.endAngle,             NO);CGPathAddArc(roundedTrianglePath, NULL,             rightCorner.centerPoint.x, rightCorner.centerPoint.y,             radius,             rightCorner.startAngle, rightCorner.endAngle,             NO);// close the pathCGPathCloseSubpath(roundedTrianglePath); 

Looking something like this:

The final result without all the support lines, look like this:

@David's geometry is cool and educational. But you really don't need to go through the whole geometry this way. I'll offer a much simpler code:

CGFloat radius = 20;CGMutablePathRef path = CGPathCreateMutable();CGPathMoveToPoint(path, NULL, (center.x + bottomLeft.x) / 2, (center.y + bottomLeft.y) / 2);CGPathAddArcToPoint(path, NULL, bottomLeft.x, bottomLeft.y, bottomRight.x, bottomRight.y, radius);CGPathAddArcToPoint(path, NULL, bottomRight.x, bottomRight.y, center.x, center.y, radius);CGPathAddArcToPoint(path, NULL, center.x, center.y, bottomLeft.x, bottomLeft.y, radius);CGPathCloseSubpath(path);UIBezierPath *bezierPath = [UIBezierPath bezierPathWithCGPath:path];CGPathRelease(path);

bezierPath is what you need. The key point is that CGPathAddArcToPoint does the geometry for you.

Rounded Triangle in Swift 4

A playground based on @an0's answer above.

import UIKitimport PlaygroundSupportimport CoreGraphicsvar view = UIView(frame: CGRect(x: 0, y: 0, width: 300, height: 300))view.backgroundColor = .blackPlaygroundPage.current.liveView = viewlet triangle = CAShapeLayer()triangle.fillColor = UIColor.systemGreen.cgColortriangle.path = createRoundedTriangle(width: 220, height: 200, radius: 15)triangle.position = CGPoint(x: 140, y: 130)view.layer.addSublayer(triangle)func createRoundedTriangle(width: CGFloat, height: CGFloat, radius: CGFloat) -> CGPath {    // Draw the triangle path with its origin at the center.    let point1 = CGPoint(x: -width / 2, y: height / 2)    let point2 = CGPoint(x: 0, y: -height / 2)    let point3 = CGPoint(x: width / 2, y: height / 2)    let path = CGMutablePath()    path.move(to: CGPoint(x: 0, y: height / 2))    path.addArc(tangent1End: point1, tangent2End: point2, radius: radius)    path.addArc(tangent1End: point2, tangent2End: point3, radius: radius)    path.addArc(tangent1End: point3, tangent2End: point1, radius: radius)    path.closeSubpath()    return path}