By Jove

Marble statue of Jupiter from c. 100 AD

There are some milder, euphemistic substitutes for using the name of God, such as “gosh darn it,” “for Pete’s sake,” “by George,”and “good golly." "By Jove" is another one but is also different. "By Jove" did not start as a euphemism, and when it first showed up in English. 

Both “by Jove” and “by Jupiter” were originally Latin oaths as pro Iovem and pro Iuppiter. Roman would use this literally in the way we might say “my God!” or “good God!” Jove or Jupiter was the top deity for the Romans. This sender of thunderbolts was Zeus to the Greeks and equated to Germanic Thor.

Today, the phrase is used to express surprise or to emphasize a statement. "By Jove, that was some explosion."

In classical times, the name was written as Iovis or Iuppiter
(Iuppiter was a compound of the archaic Latin Iovis and pater).
There was no “j” in classical Latin.
The letter “i” was both a consonant and a vowel;
as a consonant, it sounded like the English letter “y.”

Ad-lib

Ad-lib is one of the most common Latin phrases used in English. Ad-lib is the shortened version of "ad libitum" which is Latin for "at one's pleasure" or "as you desire." Sometimes it is translated as meaning "at liberty" simply because of that "lib" syllable, but that is not an accurate translation.

The most common use in drama when used to describe times in performance when a performer uses words not found in the text. (When the entire performance is spontaneous and unscripted it is called improvisation.) This occurs on the live stage, in films, on television and frequently in situations such as the conversations on talk-shows, news, podcasts etc.

Larry David's HBO series, Curb Your Enthusiasm, is often pointed to as an example of ad-lib drama. The show can also be said to use "retroscripting." The series also has created its own collection of words and phrases, as noted in the video shown here.

Less well known is its use as a direction in sheet music where "ad libitum" indicates that the performer or conductor has one of a variety of types of discretion with respect to a given passage.

Ad libitum is also used in psychology and biology to refer to the "free-feeding" weight of an animal when the animal eats as it wishes rather than its weight on a restricted diet.