Bill O'Reilly is joining in on the gay marriage discussion as the Supreme Court hears cases regarding California's Proposition 8 and the Defense of Marriage Act (DOMA).

O'Reilly made statements supporting the legalization of gay marriage on a state level that seem to contradict his earlier views.

"The compelling argument is on the side of homosexuals," he said during Tuesday's airing of "The O'Reilly Factor". "That's where the compelling argument is. We're Americans. We just want to be treated like everyone else.

"That is a compelling argument. And to deny that, you've got to have a very strong argument on the other side. The argument on the other side hasn't been able to do anything other than thump the bible.

"I support civil unions. I always have. All right, the gay marriage thing, I don't feel that strongly about it one-way or the other," he said.

O'Reilly's only concession on the subject regarded the level of government involvement.

"I think the states should do it," he said.

The comments are surprising since O'Reilly is a well-known conservative who has previously spoken against gay marriage, calling it a cause of chaos. 

Another staunch conservative, Megyn Kelly, who appeared on O'Reilly's show on Tuesday echoed his same thoughts. 

"The country's views on this issue are changing," she said. "Same sex marriage advocates have done a credible job of getting out there and making their case."

They also discussed an argument often made by Christian conservatives, that homosexual marriages undermine heterosexual ones. 

"I didn't hear anything articulated that was particularly persuasive," she said. 

As a resident of New York, where gay marriage is legal, O'Reilly said, "I live in New York. New York is fine with it."

These are comments are a complete change from the talk show host's previous views. 

In August 2012, he said, "It is not wrong or bigoted to believe American society is stronger and that the family unit is better served when marriage is between a man and a woman."

In April 2004 he said, "We told you this would happen. If gay marriage is legalized, then much chaos would follow."

In October 2004, he said, "I've been saying that all along, that if you open the door for gay marriage, then you have to have the polygamists and the triads and the commune people and everybody else, right?"