I'd bet that there are corners of non-UB vlaid programs that would be sensitive to the fact that pointers are no longer simple numbers, but maybe that's wrong or at least could be worked around.
I would add that to make this memory safe for multi-threaded programs, you also need to implcitly synchronize on fat pointer accesses, to prevent corrupted pointers from forming.
I'd bet that there are corners of non-UB vlaid programs that would be sensitive to the fact that pointers are no longer simple numbers, but maybe that's wrong or at least could be worked around.
I would add that to make this memory safe for multi-threaded programs, you also need to implcitly synchronize on fat pointer accesses, to prevent corrupted pointers from forming.