Djokovic was attempting to become just the second tennis player, and first man, to win all four Grand Slams and the Olympic gold medal in the same calendar year.
Steffi Graf achieved the feat in 1998.
Wins in the Australian and French Opens was followed by triumphing at Wimbledon just two weeks ago, but Djokovic fell to Zverev in Tokyo on Friday.
The German came from behind to win 1-6, 3-6, 6-1 and will face Karen Khachanov of the Russian Olympic Committee in Sunday’s gold medal match.
Djokovic will play Pablo Carreno Busta for bronze.
Khachanov beat no6 seed Carreno Busta 6-3, 6-3 earlier in the day.
Zverev had only beaten Djokovic twice before in their eight meetings and will now be confident of claiming a gold medal for Germany.