Every society has something that unites it: a common ancestor and a patriarchal lineage system. Or a common language, common borders, or a governing body. Generally, it is a combination of several factors that molds a mass of people into a unified whole.
The Jewish people are unique in that they have only one core group, and it is not any of those mentioned above.
All that unites us is the Torah. Nothing else has proven capable of keeping us together for more than a generation or two. Nothing else, except the Torah itself that forged us into a nation.