Glenn Beck is catching heat from the Tea Party movement that he helped champion — and helped champion him — after suggesting Tea Partiers who support Newt Gingrich over President Obama would do so only because Gingrich is white.

The former Fox News host made the comments in an interview over the weekend on Fox Business Network. Beck, after calling Gingrich a “progressive” and the “only candidate” in the Republican presidential race he could not vote for, issued what he described as a “challenge” to the Tea Party base — which in large part has flocked to the former speaker’s candidacy.

“This man is a progressive,” Beck said. “So if you’ve got a big government progressive … in Obama — one in Newt Gingrich, one in Obama — ask yourself this, Tea Party. Is it about Obama’s race?”

Judge Andrew Napolitano, host of the FBN show, questioned Beck on the argument.

“It must be about race,” Beck responded. “I mean, what else is it? … It’s the policies that matter.”

But conservatives — both those who support Gingrich and those who don’t — say it is a stretch to tie support of Gingrich to bigotry.

“It’s very wrong to pull the race card on this side of the aisle,” said Gina Loudon, an Alabama-based Tea Party activist and radio host who also helped found the St. Louis Tea Party.

Loudoun rejected that notion that racism is a factor, claiming many of Gingrich’s newest supporters had been backing Herman Cain, who is black, before he suspended his campaign.

“So saying it’s a race issue is positively disproven,” Loudon said.

Despite Gingrich’s surge in the polls, conservatives have a series of beefs with the ex-speaker — among them, his support of an individual health insurance mandate in the 1990s, his decision to cut a climate change ad alongside House Democratic Leader Nancy Pelosi, and his criticism earlier this year of Republicans’ Medicare overhaul proposal.

Still, the most recent Fox News poll showed that Gingrich was pulling in an impressive 44 percent of Tea Party support — far more than any other candidate.

Loudon, who said she’s no fan of Gingrich and that the ex-speaker has indeed championed “progressive” causes, explained that many Tea Partiers who do support him are doing so out of “desperation” because they don’t want former Massachusetts Gov. Mitt Romney to be the nominee.

Beck also drew complaints from conservative media mogul Andrew Breitbart and Tea Party Nation founder Judson Phillips.

“Those comments can only be described as nutty,” Phillips wrote on the Tea Party Nation site. “Some Tea Party members, such as myself, like Newt Gingrich. … Calling us racists because we prefer Gingrich, who despite his flaws, loves America and wants the best for America, is absolutely nuts.”

Levi Russell, spokesman for the Tea Party-tied Americans for Prosperity, also chided Beck.

He said Gingrich’s record certainly raises concern for some Tea Party activists. But the idea that race can explain Gingrich’s level of Tea Party support is “completely false and a little offensive,” Russell said.

“He’s not a perfect candidate but I think it’s completely unfair to say there’s no difference between him and President Obama other than race or skin color,” he said.

A representative for Beck could not be reached for comment; neither could a representative with the Gingrich campaign.

Beck has been particularly critical of Gingrich in recent weeks. Last week, he conducted a contentious interview with the former speaker on his GBTV show. He later said in the interview with Fox Business Network that Romney looks like a “small government guy” compared with Gingrich.

Beck said he would vote for Romney “against my will” if he had to, claiming Romney at least “is not Barack Obama.” He would not say the same for Gingrich.

Beck said last month that he will not endorse in the presidential race. But he cited Minnesota Rep. Michele Bachmann as the candidate he trusts most in the race.

“I think I could fight shoulder to shoulder with Michele Bachmann and not worry about my back,” Beck said.

Bachmann has emerged as one of Gingrich’s toughest critics, on everything from illegal immigration to the health care mandate. She took him on at the most recent presidential debate, repeatedly referring to him and Romney as “Newt-Romney” in an attempt to label them collectively as moderate forces in the race.