Shulchan Aruch O.C. 423:4:
נוהגים לחלוץ תפילין כשרוצים להתפלל מוסף
Mishna Berurah idem 423:10:
נוהגים וכו’, והטעם דכמו דאין מניחים תפילין ביו”ט משום שיו”ט בעצמו אות כמו כן יש לנהוג בראש חדש עכ”פ בשעת מוסף שמזכירים מוספי היום דאותה זכירה ג”כ הוא כעין אות וכתב הפמ”ג שיש לחלוץ התפילין בובא לציון קודם יהי רצון שנשמור חוקיך ויש נוהגים רק לחלוץ הרצועות מהאצבע קודם יהי רצון מיהו כל זה בר”ח אבל בחוה”מ שהוא יו”ט גמור יש לחלצם קודם הלל (מג”א).
Question: Why do we take off our tefillin before musaf on Rosh Chodesh?
Simple answer: Because everyone else is doing it.
Refined Question: Why is there a widespread practice to do so?
Answer: Because, as the Beth Yosef (Tur Orah Hayim 25) writes, in some places the congregation recites the text of q’dusha that begins with kether yittnu l’cha (“The angels shall give Thee a crown”), and it is inappropriate for us to be wearing our crowns while reciting it. Many of our greatest authorities have found this whole line of reasoning hard to understand, as 1. many other congregations do not recite that version, and 2. even if they did, why would that necessitate removing our tefillin? (Recall that it is also a weekly occurrence in many congregations that recitation of said q’dusha elicts others to put on their “crowns.”) As the Taz (Orah Hayim 25:16) points out, this practice is not universal, and at one point it never existed, and perhaps it is now on its way out. That is, many Rishonim never heard of this idea before, and as we pointed out earlier, the Vilna Gaon held that there can never be a minhag to not perform a commandment. And, see this from Rabbi Melamed, where he cites the same L’vush the Mishna Berura cites: somehow, the musaf of Rosh Hodesh is like an oth, a Sign of the Covenant, like observing the Sabbath, and we generally only have two signs per day: Circumcision everyday, combined with either a Sabbath or tefillin. On Hol Hamoed, those who do not wear tefillin or who remove them early do so because according to some opinions, Hol Hamoed is a day similar to the sabbbath in that m’lacha is forbidden thereon, if not by biblical law then by rabbinic law, but no such rule has ever existed for Rosh Hodesh. Notice how Rabbi Melamed mentions that he did not initially give the older, classical reason that was already dismissed, and instead he resorts to the later reason, that of the L’vush, because even though it is wrong, it is not as wrong. After all, we are saying Musaf. But he does offer that it seems that the L’vush was also bothered by this strange practice, and tried to suggest a post-facto explanation for something everyone was already doing. This goes to show you how strong precedent was and how afraid certain authorities were to change a practice they knew to be problematic. And it was that unfortunate attitude that kept Jews in Europe before the Holocaust, and it was going against that attitude that allowed us to once again become a sovereign people in our own land, both in the past century, and 2,000 years ago during the time of the Maccabees, whose story is directly analogous to ours.
Question: So why did the minhag become so common among ashkenazim if there was never a good reason?
Answer: Because, unfortunately, every weekday men are antsy to remove their tefillinand leave the services, even if the prayers are short, like on no-tahanun days. You know this to be true, and it is less so for stronger groups, like in the yeshivas, where the congregants have more time for personal prayers and less pressing responsibilities after prayers. Now, consider what happens when you suddenly add hallel and musaf to the service. The prayers go twenty minutes longer, and it just increases the pressure to leave quickly on a work day. If you’re wearing tefillin, it’s a work day. Note that. Rosh Hodesh was always a workday, and no one ever considered that there should be an issur m’lacha on it, and unlike Hol Hamoed, when at the very least the Rabbis instituted such a a prohibition. The fact that the congregants get a chance to at least remove their tefillin on time in preparation for leaving the services is very enticing. You have also seen how the post-musaf aleinu on Rosh Hodesh is often attended by about half of those who were there for prayers. Years ago, I did not have a quorum left to say the psalm of the day afterwards.
Question: So what do you do on Rosh Hodesh?
Answer: If alone, I keep my tefillin on until I finish my prayers. After all, it is a shame that I only usually get to perform the commandment during morning prayers, so why should I cut it short, especially on a day when I need to pray for longer? Because I never intentionally pray alone on Rosh Hodesh, I would prefer one of those minyanim where they keep their tefillin on for musaf, much like prominent Bnei Brak rabbis were known not to pray in traditional Ashkenazic synagogues when visiting the United States because those congregations would typically omit the daily priestly blessing. Tefillin-at-Musaf minyanim are out there, and it is my hope that they will become more common. Anticipating your next question, I believe that “renew our days as of old” is not an empty prayer, and it is both with in our power and incumbent upon us to restore observance of the commandments to the way it once was, the way it was intended.
When in the synagogue, I try to gauge the congregation. If I have to lead the service, which is often, and no one notices, I only remove my shel rosh, and if I can not get away with just that, I take off my shel yad also. This is because that I, as the emissary of the congregation, have to do what they want. However, I make a point of putting my tefillin completely back on, with a blessing, immediately after musaf so that “the children will ask,” i.e., to raise awareness of the issues, and I continue wearing them for a few more minutes. As the Mishna Berura points out, all agree that the entire day of Rosh Hodesh is the prescribed time for wearing tefillin. When I do not have to lead the prayers, if I can hide the fact that I do not remove my tefillin, then I do that, which is very rare, and if I can not, then I only remove my shel rosh and cover up my shel yad and its straps. Afterwards, I make a point to put the shel rosh back on on and recite a blessing, and this is also in order to raise awareness.