The seller is the one who signs the contract with the agent and determines what percentage of the purchase price goes to the agent. It doesn’t matter that the buyer was the source of the money, the seller is who decided what to do with it.
If the seller uses 10% of the sale price to buy a boat, are you going to say that the house buyer bought the boat?
That’s a transaction that occurs after the sale. The paying of agents is something that happens as part of the sale, so yes the buyer is very much paying the agent fees.
Houses aren't sold at a fixed price. Buyers all put in bids and the seller chooses the best one.
The amount paid for the house isn't going to be less if there were no agent fees. The buyer is paying the specific price because there are other buyers who would pay slightly less. It isn't like buyers are adding money to their offer because of agent fees, and sellers aren't going to sell the house for less if they didn't have agent fees. The price point is market equilibrium, which means the agent fees come out of the seller's total.
Now, you might try to argue that more sellers would enter the market if sellers made 6% more on selling their house, which would increase supply and decrease price, but that's a big stretch... sellers are usually selling their house for reasons besides making 6 percent more.
If the seller uses 10% of the sale price to buy a boat, are you going to say that the house buyer bought the boat?