Kane, who will turn 28 in July, has been linked with a move away from Spurs this summer and speaking to talkSPORT Defoe hinted the England captain should go.
Defoe said: “He can hopefully win a few trophies [at Tottenham], but if he’s really craving medals then he really has to go.”
Defoe, who currently plays for Scottish champions Rangers after two spells at Spurs, has now looked to clarify his comments after a backlash from sections of the Spurs support.
The former England striker has denied saying that Kane has to leave Tottenham and added the decision is in the striker’s hands.
“So I just want to quickly put the record straight,” Defoe said during a video posted on Twitter on Monday.
“Obviously I woke up this morning and I saw a headline that said that I said Harry Kane has to leave Tottenham.
“That’s not what I meant, because being a Tottenham fan I love the club. Everyone knows my relationship with the club and the fans.
“What I meant was, and what I actually said was, Harry is ambitious. He might want to finish up as the highest top goalscorer in the club’s history, ahead of Jimmy Greaves – which would be an unbelievable achievement – and along the way win some trophies.
“Or he might want to leave and go and win loads of trophies elsewhere.
“At the end of the day, ultimately, it’s up to Harry. But I just want to put the record straight, because I don’t want it to come across like I said Harry Kane has to leave Tottenham.
“Being a Tottenham fan that doesn’t make sense, so I just wanted to put the record straight.”