There are several reasons.
First among them, as you may have noticed, the devices themselves vary wildly in both size and shape. Some devices are just shaped funny, and don't take a normal battery. One would not look at a cell phone and expect a D-cell battery to fit inside and power it like certain flashlights. That brings up another good point as to why the D-cell if AA or AAA cell batteries can also power a flashlight? We can use AAAs, but a D-cell fits more naturally into an adult's hand, and it makes the flashlight larger and harder to lose. We could just use thicker plastic or metal to make the housing, I suppose, but that's much more expensive than just making a bigger hollow tube to fit a bigger battery. Sometimes your flashlight needs to double as a bludgeoning weapon.
Also, the D-cell operates at the same voltage as the smaller batteries mentioned, but is physically larger. It contains more energy. We have different sizes of batteries because devices have different energy requirements. Batteries are typically hooked up in series, so adding more batteries increases the voltage the device operates at, but not necessarily the length of time it will run. Hooking batteries up in parallel would, but that gets cumbersome of you've chained four batteries together to get 6V. If you want to hook up extra batteries to add more running time to the device, you need to double up each battery. So now we have eight batteries instead of four. Maybe you don't want to put six AA batteries together to get 9V, and would be better off just using a single battery.
Lastly, electronic equipment is manufactured in many different countries. I don't really have time to research where all of those batteries originated, and it's possible every one of them originated in the United States, but consider this:
main power and
wall outlets used around the world. We don't have an international standard for the power going to your wall, nor do we have a standard for the plug you stick into the wall and the hole it jacks into. Now, unless you travel outside North America, that's not going to be an issue for you, because we do have a standard within the continent. A battery-powered radio is a lot easier to ship overseas, though. We've been doing that for several decades now, so it's possible that we have a lot of different batteries designed in different countries that have gotten shuffled around and just haven't died out.