Let f(r) (continuous) describe the force field at a distance r. If f is positive then the field is pointed away from the origin; if f is negative the field is attractive. This theorem holds for all dimensions, but we'll prove it in 3 dimensions for clarity. Thus r = sqrt(x2+y2+z2). Place a test charge or test mass at the point x,y,z and measure the force in all three directions. The amount of force in the x direction is x/r times f(r), and similarly for y and z. Hence the force field has been written in rectangular components.
Let g be minus the integral of f, and differentiate -g(r) with respect to x using the chain rule. The result is x/r times f(r). Similar results hold for the other two partials, hence our force field f is minus the gradiant of g(r), and is conservative.
We can add a constant to g without changing its gradient f. A common convention sets the potential energy to 0 at infinity. If the force is attractive the potential energy decreases from 0 as you move towards the origin. The potential energy becomes negative, as some of it is traded in for kinetic energy. This is just a mathematical convention of course. An engineer might use sea level as the sphere of 0 potential. Thus potential energy becomes positive as the rocket rises into the sky.
The electric and gravitational forces are radial inverse square, -k/r2 for some constant k. Hence the potential energy is k/r, give or take a constant of course. It is not practical to set the 0 potential at the origin, since k/r is infinite there. That's why we place the 0 potential at the earth's surface, or at infinity.
Interestingly, when an electron and a positron (point charges as far as we can tell) fall together they annihilate each other, producing gamma rays of energy. Yet the resulting energy is not infinite. The abstract formula k/r is not valid when r enters the realm of quantum mechanics.