The general principle to which you refer does exist, but we must refine some details.
The mishna[1] lists parts of the tefilla that require a minyan, “devarim shebekedusha” – including Kaddish/Barchu and chazarat hashatz.[2] The Yerushalmi[3] comments that if a minyan was present at the beginning of a section, the remaining people can continue to recite it even after some left and there is no longer a quorum of ten. Nevertheless, a pasuk[4] is cited to harshly criticize those who leave the group short of a minyan, even when the remaining people may continue.
The Rashba,[5] in applying this ruling, says that if a group started chazarat hashatz with a minyan and someone then left, those remaining should recite even Kedusha (which is in chazarat hashatz), despite the fact they had not started Kedusha with a minyan. The Terumat HaDeshen[6] also extends the confines of what is included in these sections of tefilla. If a group was in the midst of chazarat hashatz when they lost their quorum, they can go so far as to say the Kaddish (Titkabel) that follows U’va L’Tzion, which marks the completion of the section that began with Shemoneh Esrei/chazarat hashatz. This follows from that Kaddish’s key additional sentence that requests that Hashem accept the completed tefilla of Shemoneh Esrei (“Titkabel tzlot’hon…”). This demonstrates that all of the tefilla until this point was a continuation of Shemoneh Esrei/chazarat hashatz.
The Shulchan Aruch[7] codifies the above principle and its applications. Thus, what you remember learning is basically correct; if the tefilla began with a minyan, its members may continue to recite a davar shebekedusha that they had already begun, even if there is no longer a minyan.
There is a further leniency, greater than you mentioned, regarding the number of people who can be missing. The Ran,[8] reasoning that a significant part of the minyan must remain for the group to continue as a minyan, sets the minimum at a simple majority of six (including the chazan). This view is also accepted by the Shulchan Aruch.[9]
However, we must stress that the group may not “continue normally.” Rather, they may only recite those prayer components that are directly connected to the davar shebekedusha that began with a minyan. The Yerushalmi[10] posits that each of the sections mentioned separately in the mishna constitutes a separate section. Therefore, the fact that there was a minyan for Kaddish/Barchu, for example, does not entitle the group to recite chazarat hashatz without ten men. Similarly, if a group began chazarat hashatz with a minyan but people left during chazarat hashatz, they would have to skip nesi’at kapayim.[11] They would also have to skip kri’at haTorah, even though the Shemoneh Esrei unit is not completed until after the Torah is returned to the aron.[12]
While we cannot discuss every application of the principle in this forum, we will mention an interesting distinction. At Ma’ariv, if there was a minyan for the opening Barchu, the group may recite the Kaddish before Shemoneh Esrei, because Barchu marks the beginning of the berachot of Kri’at Shema, which only conclude with Kaddish. However, the group would need to start Shemoneh Esrei with a minyan in order to recite Kaddish Titkabel with (at least) six men, because the Kaddish Titkabel at the end of Ma’ariv relates to Shemoneh Esrei. In contrast, at Shacharit and Mincha, the group would have to start chazarat hashatz with a minyan, not just the silent Shemoneh Esrei, in order to continue and complete Kaddish Tikabel without a minyan. The reason is that Kaddish Titkabel was composed primarily for chazarat hashatz. However, at Ma’ariv, at which there is no chazarat hashatz, it relates to the silent Shemoneh Esrei.[13]
The Shulchan Aruch[14] writes about the critical need to have (a minimum of) nine men listening to all of chazarat hashatz. Based on the principle at hand, if there are nine men listening in the beginning and three stop paying attention, the chazan may continue, as the lack of mental focus among some individuals is no worse than their complete absence after having left. However, such a situation is to be seriously criticized, just as we saw above regarding those who leave physically.[15]