Abu-Abdillah Shu`la, in his book صفوة الراسخ في علم المنسوخ والناسخ, pages 124-126, approves this claim. First, he reports that scholars have differed in interpreting 4:24. Mujaahid and Al-Hasan understood it to refer to proper marriage and both emphatically stated that God has never allowed temporary marriage in His Book. How true.
But then, he reports that most scholars have understood the verse to refer to temporary marriage, and
based on that interpretation, they differed still on whether that assumption of allowing temporary marriage has been abrogated. Shu`la says that a group of people, none of whom he named, said that temporary marriage was abrogated by the Quran. He quotes `Aa'isha (RA) saying, "Between them (those who allow temporary marriage) and us is the Book of God: we don't find in it but wives and bond maids." And she would then recite,
Shu`la reports the hadeeth of the Prophet (PBUH) in which he forbade temporary marriage on the day of Khaybar. Shu`la rejects the attribution to Ali that he allowed it, saying how could he when Ali was one of those who narrated that hadeeth? Furthermore, Shu`la says that whatever has been reported that Ibn Abbaas allowed temporary marriage happened before Ali drew the attention of Ibn Abbaas to the prohibition by the Prophet (PBUH).
Shu`la finally adds that Ibn Umar said that temporary marriage is fornication and his father Umar said he would "put under stones anyone who engages in a temporary marriage." Dr. M. Ibrahim Faaris, who verified the book could not find that quote by Umar anywhere though.
It is clear to me therefore that conflation of two issues took place in this claim: (a) What kind of marriage verse 4:24 talks about, and (b) The reports about the prophet (PBUH) allowing temporary marriage then forbidding it.
For the purpose of this claim, `Aa'isha (RA) is definitely right. The verse has nothing to do with temporary marriage. Therefore, any discussion of its abrogation is moot.
For the purpose of a ruling on temporary marriage, it is forever forbidden by the Prophet (PBUH). The fact that the Prophet (PBUH) allowed it once, on his own, then rescinded his previous ruling, has nothing to do with 4:24.