Production-wise, it's because back in the 60s-70s BBC wanted to continue the show but had to replace the main actor, so they wrote up a justification that allowed them to continue the show while giving the character a refresh, which is why it's still around to this day. They can replace the actor without replacing the character, and keep chugging along.
In-universe, it's because the Doctor is a member of an alien race that, when they suffer fatal injuries or reach the end of their normal lifespan, can regenerate into a new incarnation of themselves as an adult. (Whether that means Gallifreyan or not is an open question, depending on if you believe the Master was telling the truth that time.) Theoretically there is a limit of 12 regenerations, but because of reasons that would require a discussion of their own, the Doctor has gone beyond that.
They're all the same Doctor, but with a new face and personality quirks each time.
Asking who's the best Doctor is like asking who's the best Trek Captain, you're going to get a lot of different answers. I liked 4 the best of the classic Doctors, followed by 7 and 6. Of the post-revival Doctors, 12 is my favorite so far, as he gave me a very "classic Doctor" vibe. 13 did kind of call back to 5 though I think, and while Tennant is technically 14(I say technically as he was already the 10th Doctor, and there may be some hijinks going on with this regeneration), the "real" 14(or maybe everyone will call him 15 going forward - assuming we keep numbering them) is supposed to debut during or after the upcoming specials.
It's been running for 60 years(with a long break between the late 80s and early 2000s) and can easily keep running if the will is there, thanks to regenerations.