Recall our first theorem, which forces the existence of the empty set. This is obviously finite. Suppose an infinite set is implied by a chain of reasoning, even though all shorter chains lead to finite sets. The last step acts on finite sets and creates an infinite set.
The last step cannot invoke the pairing axiom, since the resulting set has two elements, and is finite. The last step cannot be comprehension; that only makes the set smaller. The finite union of finite sets is finite, the power set of a finite set is finite, and replacement produces a set that is no larger than the domain, hence finite. By induction, all sets implied by these axioms are finite.
Actually we just blew past a lot of details. Let me try my hand at comprehension. A set S is mapped to a finite ordinal, and S is restricted by a formula f. Let the new set be T, a subset of S. Mapt T through S, and into the ordinal, and let z be the least element in T. Build the ordered pair that maps z to 0. remove z from T and find the next smallest element. Map this to 1. Map the next element to 2, and so on. After finitely many steps, a collection of ordered pairs becomes a function that maps T onto a smaller ordinal. How do we know this can be done in finitely many steps? That is yet another inductive proof, on the size of S. As I say, there are many details. In any case, the subset of a finite set is finite.
Next show that the union of two finite sets is finite. This is an inductive proof on the size of x plus the size of y. From here we can jump to the finite union of finite sets. This is another imvokation of induction, on the number of sets in the union.
If the power set of a "smallest" set is infinite, remove one element and the power set becomes finite. Bring in the last element and the new power set is essentially the union of two copies of the previous power set, one with the new element and one without. This is induction on the size of S.
After replacement, map the range back to the domain, and onto the finite ordinal, whence the range is finite.
That's the last axiom; all implied sets are finite. We can prohibit infinite sets, or assert the existence of an infinite set, without contradicting set theory.