The unpeeled orange floats because the rind is very porous and filled with tiny pockets of air. Even though you're removing mass when you peel the orange, the peeled orange is more dense and sinks in the water.
What happens when you put an orange in water?
The orange with the peel floats because the peel is porous and filled with tiny air pockets. These pockets of air help increase the buoyancy of the orange. This increase in buoyancy helps the orange become less dense than the water, so the orange will float in the water.