Abraham's God doesn't play by the world's rules. The world, then and now, makes a big deal out of status: not what you are, but who you are. In a world that gave leadership and power to the eldest son, the blessings and promises went to the second son of Abraham, the second son of Isaac, and the fourth and eleventh sons of Jacob. (We find this later in the Old Testament as well: Moses, David and Solomon were also younger sons.)
Even before their birth, Rebekah received a prophetic message that her twin sons would be rivals, and the younger would supplant the elder. Esau played into the first act of this drama when his impatience, hunger, and disrespect for his birthright set up the offer from Jacob (who probably had heard his mother's story) to trade that birthright for a pot of stew. The next act was when Rebekah, like her mother-in-law, saw a chance to "help" God fulfill His promise--by setting up Jacob to steal the first-born's traditional blessing, and very nearly cost Jacob his life.
Genesis goes on to tell us the rest of Jacob's story--but what about Esau? We see him three more times: once, that he married two local Hittite or Canaanite girls (pagans) and then a daughter of Ishmael (not his father's favorite person). Then, when Jacob, who had run for his life, returned home, he met Esau, and they were reconciled. One commentator noted that just before Jacob met Esau again, he had wrestled with an angel and had his hip socket damaged and so was walking with a limp--which could have brought pity to Esau's heart. Then we read that he became the ancestor of a tribal confederation known as Edom, which settled to the south and west of the Dead Sea around Mount Seir above what later became the city of Petra, south of their cousins Moab and Ammon.
Four hundred years later, we see the Edomites refusing passage to the migrating Israelites on their journey from Egypt into their promised homeland. Later they had quarrels and alliances with the kings of Judah, culminating in prophecies of doom and destruction from Isaiah, Jeremiah, and Ezekiel, and the most poignant one--Obediah, describing how the Edomites joined the Babylonian sack of Jerusalem, looting the cities and cutting off the escape of refugees.
And then during the restored southern kingdom under the Maccabees, the second and first centuries BC, we find them resettling in southern Judah and being forcibly integrated into that kingdom and required to be circumcised. The resulting tribe became known as Idumaeans, the most famous of which was the family of the Herods. They considered themselves "Jews" and kept kosher, but were not recognized as such by religious Jews, even though Jews of Jesus' day worshipped in the restored Second Temple, completely remodeled and enlarged by the Herods. (After Herod the Great murdered his favorite wife, Mariamne, the last Maccabean princess, and their sons, it was said that it was safer to be Herod's pig than his son.) After the Roman conquests of Judea in 65-70AD and then 132-135AD, they seem to have disappeared into the general chaos of the region, ruled by outsiders until 1948.
The Edomites are sometimes used as a symbol of those who, though sharing ancestry, are enemies of God's people. One day, Jesus was debating those who claimed to be Abraham's descendants. “ They answered and said to Him, ‘Abraham is our father.’ Jesus said to them, ‘If you are Abraham’s children, do the deeds of Abraham’" (John 8:39). Again, God cares what kind of person you are, not who you claim to be.