Just collect and store. They can be purified and have commercial use. Otherwise, since they came from the earth in the first place, storing it back in the earth doesn't change the amount of toxins we have in the ground.

So why is "collect and store" OK for this stuff, which lasts forever, but not OK for nuclear waste, which "only" lasts some thousands of years?

Because this stuff already existed in the ground (just not as concentrated), is not as toxic as radioactive waste, and can be recovered and reused. Nuclear waste generates new highly toxic substances that are extremely dangerous. And even after the waste has "decayed", the products are still toxic (just not radioactive) and last forever.