Let f be analytic on a simply connected region containing p, and let z be any point in this region. Now the contour integral from p to z does not depend on the path. Two different paths give an integral of 0 all the way around, so the two paths from p to z must give the same integral.
Let g(z) be the contour integral of f from p to z, which is well defined. Note that g(p) = 0.
Let's see if g is analytic. Evaluate g(z+h)-g(z) over h. The numerator is the contour integral from z to z+h, which does not depend on the path, so choose a straight line. Remember that f is continuous, so f is close to f(z) when h is small. Let u be the unit direction vector of h, and let t run from 0 to the length of h, which we will call l. Now the integrand is approximately f(z)ut throughout the interval, and we're dividing by ul, hence the difference quotient approaches f(z)ul over ul, or f(z). The derivative of g exists, and it is f.
We can add any constant c to g(z) and find another function whose derivative is f. Conversely, if g and h have the same derivative f, then g-h has a derivative of 0. The Cauchy Riemann condition shows g-h is constant.
Had we chosen another base point q, rather than p, then g would shift by a constant, namely the contour integral from q to p.