Despite winning the first set tie-break, Djokovic was second best to Thiem, who found his form to comeback 6-7 (5-7) 6-3 7-6 (7-5) in a thrilling encounter.
The 26-year-old Thiem is competing at his fourth consecutive ATP Finals but had never beaten Djokovic on the hard court before.
"This was really one of these special matches, what I've practiced all my life for, all my childhood for," said Thiem.
"Really epic one in an amazing atmosphere, beating a legend of our game."
Defeat for 16-time major winner Djokovic means his next round-robin match against 20-time grand slam champion Roger Federer will now be a straight shootout for the last semifinal place.
The prospect of the two champions battling it out will trigger memories of this year's stunning Wimbledon final in which Djokovic saved two match points to triumph after five breathtaking sets.
"We have played a lot of matches since, and I think we both look back at a great match," Federer said, saying the defeat at Wimbledon in July took a couple of weeks to shake off.
"I think we both can take away some confidence from the match. Him, obviously a lot. Me, maybe a tad bit less."
By winning the final in London, Djokovic would also tie with Federer on six career ATP Finals titles.
Speaking about his latest defeat, the Serbian was full of praise for Thiem.
"The entire match he [Thiem] played same way he played the last point. I have to put my hat down and congratulate him, because he just played a great match," he said, not keen to talk up his match against his Swiss rival.
Djokovic and Federer will go head to head Thursday in their final round-robin match.
The ATP Finals invites eight of the world's top players to compete for an end of season prize. The field is split into two groups with the top two players from each qualifying for the semifinals.
Stefanos Tsitsipas, Daniil Medvedev, Nadal and Alexander Zverev are all competing in the other side of the draw.