Well to be fair, when you use the forums platform method of adding pictures by clicking on the "Upload a File" button, it takes you to browse your picture files, where you click on the file you want to attach, and then you get the little rectangular link as you said. So that is the sanctioned way the forum gives you to do it.yep..... absolutely.
I guess not many are aware of this feature, myself included.
Instead of adding it that way, if you have a picture you want to add open somewhere else on your computer, you can use the right click and paste method I mentioned above to put it anywhere in your post rather than the upload file button to put it at the bottom. I don't know if there is any reason that the forum wouldn't want it done this way, but it does work.