The Torah forbids cutting down fruit trees,[2] which is the strictest application of the concept not to be destructive.[3] It is thus not surprising that the gemara and poskim identify “non-destructive” cases in which it is permitted to cut down fruit trees.
The gemara[4] grants permission in the following cases: 1) The tree no longer produces a kav (a relatively small amount) of fruit. 2) It is worth more for wood than for fruit.[5] 3) It is significantly damaging a more valuable tree. 4) It is damaging someone else’s property.[6]
The Rosh[7] deduces from the above that one may cut down a fruit tree if he needs to use its location, which the Taz[8] applies to building a home. Most poskim rule that this includes expanding a home, at least when the addition is objectively more valuable than the tree.[9] However, the gemara tells of an Amora’s son who died because he cut down a fruit tree prematurely, and Rav Yehuda HaChasid also warned about it. Therefore, even when it is apparently permitted, some prefer that the work be done by a non-Jew[10] and/or that the tree be transplanted rather than disposed of.[11]
To what extent can we rely on the Rosh’s thesis that making room available for something else is a legitimate excuse for cutting down a fruit tree? The Beit Yaakov[12] claims that Tosafot and others disagree with the Rosh. The Meishiv Davar[13] adds that it is difficult to be certain that after cutting down the tree, the building project will actualize. However, many Acharonim[14] strongly reject the Beit Yaakov and adopt the leniency of the Rosh/Taz.
How important must the need for the spot be? Although the gemara’s cases do not refer to huge benefits (e.g., the fruit is worth less than the wood), they relate to situations in which the tree’s existence itself is relatively directly wasteful. In contrast, in the Rosh’s case (and yours), the tree is fully viable; it simply precludes another future use of the area. It is therefore not surprising that some authorities who accept the Rosh’s view say that the need must be substantial. For example, the She’eilat Ya’avetz[15] writes about a shul that was too small and needed to be extended to an area occupied by fruit trees. The Chavot Yair,[16] while allowing cutting down a fruit tree that darkens one’s house, forbids doing so just to create a place to walk around or to simply increase space and light there. Several Acharonim, including important poskim such as the Aruch HaShulchan[17] and Yabia Omer,[18] adopt this middle-of-the-road approach.
Appraising the cases you raised is tricky. On the one hand, building a swimming pool or a basketball court is expensive. Consequently, if one wants to build such a structure, it shows how important it is to him,[19] in which case the existence of a fruit tree should not preclude the project. On the other hand, some poskim[20] indicate that the value of the change should be an objective one that applies to the average person. Swimming pools and basketball courts are not likely to qualify in that regard (even if we focus on the positive and permitted uses of those facilities). It is difficult to ignore the warning that one who uses honest but faulty judgment in this case could be punished severely (as mentioned above[21]). Another factor is that it might be possible, even if less convenient, to build what is desired without cutting down the fruit tree.
Therefore, we suggest that if you are willing to have a professional, preferably a non-Jew, transplant the tree, you may do so.[22] Otherwise, we would have difficulty permitting removing the fruit tree unless we were convinced that the benefit/need and the lack of an alternative were absolutely clear.