Bel-Phegor was a Moabite/Assyrian deity, whose rites were celebrated on Mount Phegor, and were noted for their obscenity. He is referred to as Baal-Peor {The god of Baal or Peor} as well as Baal-Phegor and Bel-Phegor.  The modern day modern Belphegor is a corruption or adaptation of the Biblical story in the Book of Numbers in which 24,000 Hebrews were executed for fornicating with moabite women and participating in the Moabite rites on Mt. Phegor.

Numbers 25:3 So Israel yoked themselves to the Baal of Peor. And the LORD’s anger burned against them.

Numbers 25:5 So Moses said to Israel’s judges, “Each of you must put to death those of your people who have yoked themselves to the Baal of Peor.”

 {The following passage refer to the occurrences of Numbers 25 above in which Moses had put to Death 24,000 Hebrews}

Deuteronomy 4:3 You saw with your own eyes what the LORD did at Baal Peor. The LORD your God destroyed from among you everyone who followed the Baal of Peor.

Psalm 106:28 They yoked themselves to the Baal of Peor and ate sacrifices offered to lifeless gods;

Hosea 9:10 “When I found Israel, it was like finding grapes in the desert; when I saw your ancestors, it was like seeing the early fruit on the fig tree. But when they came to Baal Peor, they consecrated themselves to that shameful idol and became as vile as the thing they loved.

Belphegor presided over rites on Mt. Phegor which the Hebrews considered abominations . He has been called "the Lord of the Opening" which has lead to some depictions of him as always having an open mouth, this is probably based on a misinterpretation.  "The openings of which he is the master includes caves, crevices, wombs,  and vaginas." Encyclopedia of Spirits. By Judika Illes
 

As per An Encyclopaedia of Occultism by Lewis Spence he is the "demon of discoveries and ingenious inventions. He appears always in the shape of a young woman. ..he it is who bestows riches"

According to the Kabala, Belphegor was once an "Angel of the order of the principalities but later became a god of licentiousness in ancient Moab" Satanism Today: An Encyclopedia Of Religion, Folklore and Popular Culture .   Rabbinical opinions of Belphegor are summed up very handsomely by the Dictionairre infernal where it states "Some rabbis claim that he must be worshipped on a toilet, with offerings being the residue of ones' digestion."

Eros and Evil by R.Masters equates Belphegor with the Hindu God Ruttrem who is usually depicted as an erect Phallus or penis. 

According to legend, Belphegor was sent from Hell by Lucifer {or Pluto in some instances} to find out if there really was such a thing as marital harmony. Rumors had reached the demons but they believed people were not designed to live harmonically. Belphegor's experiences in the world soon convinced him that the rumor was false. This theme is repeated in several early works of modern literature.

• Niccolo Machiavelli (1469-1527) wrote The marriage of Belphegor , a satire on marriage in which a demon comes to earth to prove that women damn men to hell.
• Dictionairre Infernal - Belphegor was Hells ambassador to France.- certain dignitaries of the infernal empire served as special envoys or ambassadors to the nations of the earth, and Belphegor was accredited to France. This is repeated in Victor Hugo's "Toilers in the Sea"
• Paradise Lost by John Milton draws upon Kabbalistic writings when he describes Belphegor as "of principalities of the Prime"