Rashi to Deuteronomy 12:4:
You shall not do so [to the Lord your God]: to burn sacrifices to God in any place you choose, but rather at the place that He will choose. Another explanation is: “And you shall tear down their altars… and destroy their name… [but] do not do so [to the Lord your God]”; this is an admonition [addressed] to one who would erase the Name [of God from any writing] or remove a stone from the altar or from the courtyard (Mak. 22a).
The first of Rashi’s explanations is the one preferred by his grandson, Rashbam, and better fits the context of the subject matter. The gentiles create places of worship wherever they wish and as many they wish; the Jews shall only have one Temple where God resides. The second explanation is a secondary level of interpretation and is used to derive biblical prohibitions. It is interesting that if this commandment is taken as the plain meaning of the verse, then we should note the following verse,
But only to the place which the Lord your God shall choose from all your tribes, to set His Name there; you shall inquire after His dwelling and come there. There you shall bring your burnt offerings, and your sacrifices, and your tithes, and the separation by your hand, and your vows and your donations, and the firstborn of your cattle and of your sheep.
Then we see that the way to avoid destroying God’s Temple and altar is by inquiring after it (i.e., finding where it is and caring about it) and going there to perform the commandments and that if we ignore the place of His abode, and choose not to seek it out and care for it, and to serve Him there, then we could, God forbid, lead to it being harmed…