Our response must consider not only the question of whether an avel should be chazan on Rosh Chodesh, but also the issue of whose call it is – the shul’s or the avel’s.
The Maharil[1] is the primary source for limiting which days an avel may serve as chazan. He holds that an avel should not be chazan for the tefillot of Shabbat and Yom Tov or for Hallel, because of the simcha associated with those tefillot. The Rama[2] cites this idea only concerning Shabbat and Yom Tov, writing that the practice is for aveilim to refrain from being chazanim on those days, although it is not actually forbidden. Ashkenazi Acharonim accept this limitation regarding Hallel as well.[3] There are a wide range of opinions and practices regarding the rest of Shacharit other than Hallel, as well as regarding Mincha and Maariv on days other than Yom Tov on which Hallel is recited in Shacharit (Rosh Chodesh, Chanuka) and on Purim.[4] Therefore, both practices you refer to are legitimate; deciding between them is not our present focus.
The minhag to prevent an avel from serving as chazan on a happy day is related to the incongruity between the festivity of the day, which pervades its tefilla (or tefillot), and the avel’s sadness (or an aura of strict judgment[5]). What is not clear is whose interests we are “protecting” when we prevent the avel from being chazan, and this question impacts on the question of who gets to make the decision. If we are protecting the congregation from a chazan who is incapable of sufficiently elevating the congregation to the proper festive mood, then the proper practice is clearly the shul’s call, and ostensibly the avel in your case (unintentionally) acted improperly by pushing himself onto a congregation that did not want him. If, however, the issue is that it is wrong for the avel to involve himself in the joyousness by leading a happy tefilla, because that is incongruous with his aveilut, then the matter logically depends on his rabbi’s ruling.
Sources that connect the matter to the suitability of the avel to act as a shaliach (agent) of the tzibbur on a joyous day include the Taz[6] and the Zera Emet.[7] My reading of the Maharil[8] and the Maharam Shick[9] indicates that they reason that the issue is one of inappropriateness for the avel vis-à-vis his mourning obligations. It is possible to distinguish between different days or between parts of tefilla. Perhaps on Shabbat, Yom Tov, and during the leading of Hallel itself, the community requires festiveness that the avel is assumed to be unable to provide. On the other hand, Mincha, Maariv, and the rest of Shacharit other than the joyous Hallel on days of less overarching joyousness have no special requirements for the chazan, but such prominent involvement in the tefillot of a happy day is improper for him. In any case, we leave the fundamental question of whose issue is it without any clear conclusion.
Some of the authorities opposed to an avel serving as chazan for any part of Shacharit, including the Gra[10] and the Chatam Sofer,[11] were reportedly in the practice of not allowing an avel who wanted to be chazan to do so. Why would we stop someone who wants to be chazan if the source for this is post-Talmudic and respected opinions permit it? One explanation is that the Gra and the Chatam Sofer indeed held that the avel’s status affected the community, whom these distinguished rabbis represented, regardless of what the avel or his rabbi might have held. Another possibility is that even regarding matters that affect the individual, it is generally improper to contradict a local minhag.[12] This point is especially poignant regarding someone as visible as the chazan.[13]A third possibility, albeit one that I find difficult, is that these poskim were so convinced that the other practice was wrong that they felt a need to save the avel from compromising his aveilut responsibility.
Our practical advice is as follows. An avel who received a ruling that he may serve as chazan on Rosh Chodesh should ask the rabbi or gabbai in the shul in which he finds himself if he agrees with this ruling. If he does not, he should ask whether the community minds if he is chazan anyway. If they do not mind, it should be fine for the avel to lead the part of davening of Rosh Chodesh that he wants to. However, if the shul’s minhag of not having the avel lead that tefilla is one that they regularly adhere to, the avel should not serve as chazan even if no one actually tries to stop him. This can either be because it is the community’s prerogative to decide or due to the idea of publicly conforming to local minhag.