There are malls, markets, things like that. These are just the front end for the customers to buy things, the retailer.
As you said there are centralisation behind. Either through the same company managing all those different malls, or through Wholesalers working in the background selling to a lot of retailers.
Amazon is as you said extremely efficient on their warehouse and shipping because of their centralization and size. They can build extremely expensive warehouse to keep being efficient and surpass the competition.
There would maybe be possible to get an “enhanced” activity pub in order to protect privacy and financial informations. There is already something similar but without activity pub : a multitude of website where it is possible to buy things.
But there still needs to be some centralization, either through the postal service for small companies, or through centralised warehouse companies, with spzcialized equipments. And both of these things are already in place.
However if the OP wants a decentralized Amazon like efficiency, it won’t happen. The costs will increase and it will take more time, because the way the shipment will work will heavily depend on the seller.