Would a physics engine actually make it any easier to solve this inverse problem? Simulating where the ball ends up relative to the hoop after bouncing off the board is the kind of forward problem that I tend to think of physics engines as designed for, but the problem here is to (repeatedly) determine what angle of board would yield the desired post-bounce trajectory. Does something like Bullet have a flexible enough inverse kinematics solver to handle this kind of problem?
There are obvious ways that the physics engine might be able to capture more effects than a simple analytic solution of parabolic trajectories. But if it only lets you run the simulation forward to get a trajectory from initial conditions, then you now have a numeric optimization problem as the body of the inner loop of another optimization problem.
If the physics engine can't efficiently solve the inverse problem here, it is probably only well-suited for simulating a final solution for validation, not iterating over the rather large space of possible shot trajectories times possible backboard angles at the point of impact.
It's a straightforward integration to get from initial conditions of the ball throw and backboard angle to the post-bounce trajectory. Physics engines like Bullet will get you that solution very quickly. But determining which backboard angle puts the ball in the center of the hoop is not as straightforward if your physics engine is only designed to do numeric integration of the equations of motion to move a simulation forward in time. You have to run the simulation a few hundred times with different backboard angles to determine which setup puts the ball where you want it.
And that process needs to be done for all possible incoming ball trajectories that would hit that point of the backboard in order to figure out what backboard angle will maximize the number of shots that go in.
