Herman & Gloria Cain's Problems Go Way Back: Report

As Herman Cain headed home on Friday to Georgia to face his wife for the first time since Ginger White alleged an affair with him, a new report surfaced stories of the couple having deep marital unrest long before Cain's presidential campaign was rocked by allegations of sexual harassment.

"It never felt like a real marriage when I was around them," someone close to the family told the Daily Beast. "Mostly he was always gone and his wife seemed to be OK with it. Not being together seemed the norm for their marriage, and Gloria didn't seem to mind. His kids didn’t seem to mind either."

The Cains have been married for 43 years. They have two children and three grandchildren. Sources told the Daily Beast that Cain's attraction to other women has always been a strain, and the couple has even lived apart during periods of their marriage.

Cain previously told HuffPost that Gloria supported him throughout the sexual harassment allegations that surfaced last month, but he has acknowledged that his wife never knew about his friendship with Ginger White or that he was giving her money and gifts over the years. He insists their relationship was not sexual. White has apologized to Gloria for having sex with Cain.

Gloria told FOX News last month that the sexual harassment charges other women have leveled against her husband don't ring true because he "totally respects women."

But a campaign worker told the Daily Beast that she was pressured to do the interview and won't defend her husband again against Ginger White's story.

"She doesn't want to be forced to do that again because she knows he's had girlfriends for many years," the source said. "She just looked the other way. But if he won't get out of the race, she may have to."

In light of the Ginger White allegations, Cain is reassessing his candidacy and his conversation with his wife Friday may be pivotal in his decision whether to remain in the race.

"It is having a toll and anybody would do the same thing," he told FOX News.

Meanwhile, "Women For Herman Cain" launched Friday and Gloria is listed as the group's National Chairperson. The site uses an image of several ethnically diverse women giving the thumbs up; these are not Cain supporters but models for a stock photo. Actual female supporters have posted words of encouragement for the couple and harsh attacks against Cain's accusers. Here are some words from Barbara Dayan of Nipomo, Calif.:

Dear Mrs. Cain Don't pay attention to these pathetic husbandless women who are jealous of women like you in happy long-term marriages. These vindictive women can't find a husband or keep one. They are like stalkers who try to latch on to any man who shows a bit of kindness or attention to them. When these unstable women come out of the woodwork to make accusations about Herman just say, "Honey, get a life, I believe my husband." We want you to be our First Lady Mrs. Cain!

The initiative's press release follows:

"Women For Cain" is an online national fellowship of women dedicated to helping elect Herman Cain as the next President of the United States.

Mr. Cain has been a strong advocate for women throughout his lifetime, defending and promoting the issues of quality health care, family, education, equality in the workplace and many other concerns so important to American women.

Gloria Cain is the National Chairperson for "Women for Cain" and is the very special woman who Mr. Cain devoted his life to many years ago. Mr. Cain and Gloria celebrated their 43rd wedding anniversary earlier this year. The couple has two children and three grandchildren and a legacy of family, friends, and community and church involvement.

"Women For Cain" was formed to inspire a national women's alliance in support of Herman Cain 2012 and Friends of Herman Cain. Please join our conversation to learn of volunteer opportunities and to be informed of issues and events surrounding the Herman Cain campaign.

Thank you for your support. We welcome your thoughts and encouragement for Mr. Cain. We also welcome your involvement in our effort to renew America with the common-sense solutions and principled leadership that Mr. Cain provides. Join us as we work to bring true opportunity and prosperity to all Americans.

Here are the women who have been linked to Cain:

Herman Cain's Women

CORRECTION: An earlier version of this story misstated Cain's home state as South Carolina, not Georgia, and also incorrectly claimed the Cains have two sons. The Cains have one daughter and one son. We regret the error.