The advice isn't to mathematically prove anything, which as you mentioned is hard and may even be unsolvable.
The advice is to look at the issue as you would a mathematical problem, critically and analytically. Don't use shotgun debugging or quick fixes, but instead try to understand the code thoroughly, what its goal is and how it accomplishes it. Instead of testing with random data, mentally walk through all the possibilities and branches.
This is akin to making a mental model of what you are trying to do, and then verifying that the model is correct, and that your code matches the model.
The advice is to look at the issue as you would a mathematical problem, critically and analytically. Don't use shotgun debugging or quick fixes, but instead try to understand the code thoroughly, what its goal is and how it accomplishes it. Instead of testing with random data, mentally walk through all the possibilities and branches.
This is akin to making a mental model of what you are trying to do, and then verifying that the model is correct, and that your code matches the model.