Interesting quote from HalachaBlog here:
20) The view of Rav Elazar Ben Tzadok (Pesachim 114a) is that charoses constitutes a mitzvah. The Gemara presents two explanations of Rav Elazar ben Tzadok’s opinion. One explanation is that the thick texture and cloudy color of the Charoses serve to recall the mortar that the Jewish slaves used for making bricks in Egypt. A second explanation is that the Charoses serves to remind us of the “Tapuchim” in Egypt. Rashi and Rashbam explain that the Jewish women in Egypt would painlessly and quietly give birth beneath the apple trees so that the Egyptians would not discover that a Jewish male was born. We follow the view of Rav Elazar Ben Tzadok.
21) The Rishonim (Tosafos, Tur etc.) write that Charoses is made from fruits mentioned in Tanach symbolizing the Jewish people (apples, figs, dates, walnuts, almonds and pomegranates, with some adding grapes and pears). The Ari z”l would eat Charoses comprised of grapes, figs, dates, nuts, apples, pomegranates and pears. The Ari z”l did mention that the common custom amongst Ashkenazim was to make Charoses comprised of nuts, apples and pears.(See Kol Bo and Kaf Hachaim 473:99)
22) As noted above, we explained that one of the reasons that we eat charoses is that the Charoses serves to remind us of the “Tapuchim” in Egypt. Rashi and Rashbam explain that the Jewish women in Egypt would painlessly and quietly give birth beneath the apple trees so that the Egyptians would not discover that a Jewish male was born.
23) It should be noted that Tosafos (Taanis 29b) explains that the Biblical word “Tapuach” refers to a citrus fruit, like an esrog or an orange. This view of Tosafos was also cited by Harav Yosef Dov Soloveichick zt”l. (Nefesh Harav 209) It is for this reason that Rav Hershel Schachter shlit”a (in a shiur) maintains that if one wants to be accurate, he should use oranges (or other citrus fruits) for the charoses. Harav Avraham Blumenkrantz zt”l adds that in many Sefardic homes apples are not used at all for the Charoses. And that those who do use apples for the Charoses should also include some citrus fruit or juice. However, this does not seem to be the common custom.
By the way, ignoramuses treat Charoses as a late custom (while elevating baseless habits as halacha), but it is brought in the Gemara.