Your impression has a basis but is not precise. Let us explain.
We fulfill the main mitzva of Kiddush (verbally sanctifying Shabbat) on the evening of Shabbat by making a special beracha (“mekadesh haShabbat”) both in tefilla and before eating. When it comes to the daytime Kiddush (called Kiddusha Rabba[1]), the main purpose is simply to add prominence to the meal by beginning it with wine[2] [or possibly a substitute – beyond our present scope]. If, for whatever reason, one did not make Kiddush at the morning meal (i.e., after davening), then he begins se’uda shlishit with Kiddush.[3] The time element of Kiddush (i.e. having to make Kiddush by chatzot) is not an inherent issue. Rather, whenever the first Shabbat day meal is, Kiddush should precede it. The question, then, is whether the meal must be started by chatzot.
There are classical sources that refer to the three meals of Shabbat as taking place at night, in the morning and in the afternoon, respectively.[4] However, it is not clear whether the stated times are halachic requirements, assumptions or suggestions.[5] There is extensive discussion among poskim concerning the questions of whether the first meal must be at night and whether the third must be in the afternoon. In contrast, there is little discussion about whether the second meal must begin in the morning, and the common practice is to not be careful that it is. See an interesting discussion of possible reasons for this phenomenon in Yisrael V’Haz’manim.[6]
The more serious problem is that of fasting on Shabbat. All agree that, under normal circumstances, it is forbidden to fast on Shabbat, the day the Torah[7] and Nevi’im[8] refer to as a day of eating and indulging. The Shulchan Aruch[9] says that one may not fast on Shabbat beyond the end of six hours (a standard term for chatzot) even if he does not intend to fast.[10] The Rama[11] rejects the minority opinion[12] that holds that one who is preoccupied with tefilla or Torah study need not be concerned about chatzot. Thus since one must eat by the end of six hours and he may not eat or drink before Kiddush,[13] he ostensibly has no choice but to make Kiddush by chatzot. Of course, one can satisfy the requirement of not fasting by having Kiddush and cake before chatzot. He does not need to eat challa by that time.
However, there are grounds and means for leniency (if you call not eating leniency). Most commentators[14] say that the significance of six hours into the day is that after that time the stomach is so empty that it is not receptive to food, a situation one really should avoid all week long. The Magen Avraham[15] and Mishna Berura[16] say that, in this regard, the period of six hours[17] is calculated from the time a person awakens. This solves the problem for those who start eating late because shul starts late.
However, on Shabbat, there may be an independent problem of fasting until midday, even if one has not been up for six hours.[18] One can remedy this by drinking a cup of water before the beginning of tefilla (before there is a need for Kiddush). After that, not eating is not considered fasting. It is unclear whether this also remedies the issue of an empty stomach.[19]
In summary, Kiddush, per se, need not be made by chatzot, but one may not fast until that time. Since most people eat within six hours of awakening, it is possible that there is no problem, and, if there is, it can be remedied by drinking water before tefilla.