Cannot upgrade pytorch to version `>=2.3`

We are blocked from upgrading pytorch to versions 2.3.x and superior due to new pin introduced by the pytorch channel on llvm-openmp<16.

The full issue is logged at issue 271 of the scikit-learn feedstock at conda-forge.

Once that is sorted, we can try an upgrade on our end.