There's actually different levels of "deleting".
If you simply delete a file in your operating system (windows, macos, linux,...), it is most of the times simply moved into another folder, the "trashcan", and it will stay there untill you ask to put it back into place, or if you empty the trashcan.
If you "permanently delete" a file, or if you empty it from the trashcan, the operating system will mark the entry of the file as being "deleted". This doesn't mean that the file is gone yet at that moment. It has just been marked as deleted, so that the operating system knows that the part of hard drive that was taken by that file can be reused for other files. And as long as no other files have been written to disk, onto that particular spot, the file you've permanently deleted can still be retrieved by disk-rescue software.
If you "safe delete" a file (not available on all operating systems), the file is marked as deleted on the hard drive, and immediately overwritten by random characters for a set amount of times (99 or so). This makes sure that the file cannot be "undeleted" and read by someone else.
You don't physically delete anything on a hard drive. What you do is tell the computer that you do not want this file anymore, and it will then be able to use this storage for other purposes. Most of the time it will just leave the data be, which is why we can often recover it for a certain period of time, but this storage is then marked as available to store new information on.
However, it's also possible to delete data more securely, preventing anybody from recovering it. You still don't "delete" it, but instead the program responsible will write "junk data" on top of the previous data, making it permanently unavailable. This junk data can then be considered free space, and you can store other things on top of it. This obviously removes the junk data, but it's not like you need that anymore anyway.