Good question. I came up with two separate answers.
a) It’s a theorem of set theory that every infinite set contains a countably infinite subset so it’s no loss of generality to simply assume your set of oranges is countably infinite. If the set is uncountable we can adapt the same idea. So we label the oranges 0, 1, 2, 3, 4, … We can do that since they’re countable, which means there’s a bijection between the naturals and the oranges. So we can number each orange by the natural number that maps to it in the bijection.
Now the entire set of oranges is in bijection with the set of even-numbered oranges, by the usual mapping n => 2n. Since the set of oranges can be bijected with a proper subset of itself, it’s an infinite set of oranges.
b) Set theory as currently understood is purely about mathematical sets, the sets of ZFC or some similar axiom system. In ZFC, everything is a set. We start with the empty set, and the set containing the empty set, and the set containing those two, and so forth, and the the powersets and unions of all those sets, and so forth.
So in math, there is no set of oranges. If you have two oranges, I do NOT CLAIM that there is a set containing the two oranges. I do not personally believe in set theory outside of the pure sets of mathematics! That’s essentially a formalist position. A formalist is a philosopher who maintains that math is simply about the formal manipulation of meaningless symbols according to arbitrary rules. It means NOTHING.
So there is no set of oranges. There are no sets of anything, other than the empty set and all the other sets that can be built from it via the axioms.
Either of those float your boat?