By Jove

Marble statue of Jupiter from c. 100 AD

There are some milder, euphemistic substitutes for using the name of God, such as “gosh darn it,” “for Pete’s sake,” “by George,”and “good golly." "By Jove" is another one but is also different. "By Jove" did not start as a euphemism, and when it first showed up in English. 

Both “by Jove” and “by Jupiter” were originally Latin oaths as pro Iovem and pro Iuppiter. Roman would use this literally in the way we might say “my God!” or “good God!” Jove or Jupiter was the top deity for the Romans. This sender of thunderbolts was Zeus to the Greeks and equated to Germanic Thor.

Today, the phrase is used to express surprise or to emphasize a statement. "By Jove, that was some explosion."

In classical times, the name was written as Iovis or Iuppiter
(Iuppiter was a compound of the archaic Latin Iovis and pater).
There was no “j” in classical Latin.
The letter “i” was both a consonant and a vowel;
as a consonant, it sounded like the English letter “y.”