Let k be the exponent. Thus n = 2k-1, and we're wondering if n is prime. Remember that k must be prime for n to be prime.
Let e1 e2 e3 etc be the sequence where e1 = 4, and ei+1 = ei2 - 2 mod n. The number n is prime iff ek-1 = 0. This is the mersenne prime test.
Suppose k is even (and greater than 2). Since k is not prime, n is not prime; and each ei beyond e1 = 2 mod 3, so ek-1 is not 0. Hereinafter, we can assume k is odd.
Let z = 2(k+1)/2, and note that z is a square root of 2 mod n. Let R be the ring of polynomials mod n over the quadratic x2-zx-1. Let a and b be the roots of the quadratic. These lie in Zn iff the quadratic splits. In either case, a+b = z, and a*b = -1.
Evaluate 2 = z2 = (a+b)2. The 2ab becomes -2, so a2+b2 = 4 = e1.
Prove by induction that ei = a2i+b2i.
The condition ek-1 = 0 becomes a2k-1 = - b2k-1. Multiply through by an appropriate power of a and get a2k = -1. The order of a is 2k+1.
Get ready to apply the nst prime test. The ring R has n2-1 nonzero elements, which factors into n-1 times n+1. Let s = 2×(n+1), which is certainly larger than the square root of n. It is also easily factored, as it is a power of 2. Set u = a, and us = 1, while us/2-1 = -2, which is a unit in Zn. Looks like we're on the right track.
Remember that a2k = -1, and divide by a, giving an = b. The polynomial with roots a and an is the polynomial with roots a and b. This is the aforementioned quadratic, which has integer coefficients. The conditions of the theorem are satisfied, and n is prime. Technically we need to run one trial division, but it's n into n, so we can skip that step.
Conversely, let n be prime. the quadratic has discriminant 6, which is not a residue mod n. First, 2 is a square mod n. To deal with 3, we need k to be odd. Thus n is 1 mod 3, and by reciprocity, 3 is not a square mod n. Since 6 is not a square the quadratic is irreducible, giving a finite field of order n2.
Every finite field extension is galois, so there is but one nontrivial field automorphism, which can be expressed as conjugation, i.e. swapping the roots a and b, or as the frobenius automorphism, i.e. raising to the nth power. Thus, an has to be a conjugate of a, namely b. Similarly, bn = a.
With this in mind, consider the square of ek-1. This is 0 iff ek-1 is 0. Remember that ab = -1.
ek-12 =
a2k + b2k + 2 =
an×a + bn×b + 2 =
ba + ab + 2 =
0
With this algorithm in hand, mersenne primes are the largest (verified) primes. As of this writing, the largest exponent is 25,964,951. That's almost 8 million decimal digits. If you have some spare cycles on Windows, Unix, or Linux, you can help find the next mersenne prime.