We will start with the background and some basic rules. The laws of people being invalid to testify about their relatives are derived from the following pasuk: “Fathers shall not be killed because of sons, and sons shall not be killed because of fathers.”[2] The gemara determines that “because of” refers to testimony by relatives and that these laws apply to all forms of testimony.[3] It also derives the degree of relationship that is disqualified by analyzing linguistic nuances. The Shulchan Aruch presents the details in the section regarding monetary law.[4] Let us note that a ketuba is primarily a monetary document and does not effectuate the marriage. The requirements for witnesses for the marriage are basically the same.[5]
The basic halachic terminology follows. A first-level relationship (parent/child; siblings) is called rishon b’rishon. The next generation relationship (first cousins) is called sheni b’sheni. Members of these first groups may not testify for one another (or together). Second cousins are shlishi b’shlishi and are valid witnesses. Halacha also deals with mixed generations. Using your case as an example, an uncle is a rishon b’sheni, which is a closer relationship than cousins are, and is certainly an invalid witness.
What about the fact that the uncle in question is only an uncle through marriage? The Torah, in forbidding relations with an uncle’s wife, calls her an aunt.[6] The gemara[7] derives from this that relationships through marriage are much the same as those of blood relatives. There are two areas of practical difference, however. Consider a borderline relation such as a great-uncle (shlishi b’rishon). If the relationship is through marriage, then the relatives can testify for or with each other.[8] Additionally, if cousins (sheni b’sheni) are double in-laws, then they can testify. In other words, husbands of two female first cousins may testify together,[9] although this too should be avoided.[10] Regarding double in-laws of uncles and nephews (rishon b’sheni), who are closer, there are two opinions in the Shulchan Aruch,[11] and the Rama is lenient b’d’ieved (after the fact). Although the uncle in question has this type of relationship with the groom, everyone would agree that he is invalid because he is a single in-law of the bride. After all, a witness may not be related to either the bride or the groom, as, at the wedding, he is testifying about both of them. Thus, the question of a double in-law of the couple will never arise at a wedding.[12]
Valid witnesses are required to effectuate a marriage or, in this case, ensure a valid ketuba [which is (rabbinically) required for the couple to live together]. Witnesses for a ketuba should preferably have a working knowledge of its language and basic laws. In addition to the laws of relatives, witnesses should be beyond suspicion of sins that could render them invalid. Decisions regarding witnesses are the domain of the mesader kiddushin (officiating rabbi). As he is responsible to ensure that everything is done properly, he should be allowed the opportunity to make calm, informed decisions and not be pressured to accept someone as a witness just because the family feels close or obligated to him. The rabbi can make a decision concerning how “distant” a relative is only after full disclosure of the relationship, which people sometimes forget to make. In your case, halachically, this uncle (by marriage) is not distant at all, and it is good that you had the foresight to determine this with time to spare.