[SampleSet] Do not load delayed attributes by not copying them over

4 jobs for lighter-samplesets in 14 minutes and 1 second (queued for 1 second)
Status Name Job ID Coverage
  Build
passed build_linux_37 #237887
docker

00:11:53

passed build_linux_38 #237888
docker

00:11:37

passed build_macos_intel_37 #237885
macos intel

00:14:01

passed build_macos_intel_38 #237886
intel macos

00:13:48