Speaking at a press conference in Pyeongchang, South Korea Tuesday, Rippon said he stood by his recent criticism of Pence's position on LGBT rights.
"I have no problem about what I've said because I stand by it, but I think right now the Olympics are about Olympic competition," Rippon said.
Rippon turned down a meeting with Pence earlier this week, after criticizing him and his history with the LGBT community in an interview.
Speaking Tuesday, Rippon regretted his feud with Pence overshadowing the Olympics itself: "It's brought a lot of attention to my other teammates and I don't want it to distract from them."
"I don't want my Olympic experience being about Mike Pence," he said.
Despite that, the US figure skating star, who was part of the American roster that won the bronze medal in Monday's team competition, said he will keep using his platform.
"I can't tone it down. I'm being me and being myself," Rippon said. "I've got so many messages from kids all over the country -- I'm getting so emotional thinking about it -- I think that's why it's so important."
"I think as an athlete I use this platform to my advantage. I think it's giving my skating a greater purpose," he added.