On shabbat we don't wear tefillin because shabbat is an אות. However yom tov is not referred to as an אות. So why don't we wear tefillin on yom tov?
The Rambam (Hil. Tefilin 4:10) disagrees with you. He says:
The Rambam says clearly that Yom Tov is an אות and therefor does not warrant putting on Tefillin.
The Rambam's source is a Gemara (Menachot 36):
The question now becomes: How is Yom Tov an אות?
The Aruch Hashulchan in Hil. Tefilin 31:3 answers this in 2 ways:
Additionally, Pesach Mitzrayim is refered to in the Torah as an ot, and as all the yamim tovim are connected to the Exodus, there's a sort of transitive operation by which the others can be considered otot as well.