It seems like the benefits are having the device lock/wipe itself after a set amount of attempts in case of a brute force attack and not having to run software to decrypt the drive on the device you plug it into.
I included a picture of the IronKey Keypad 200 but that's just because it's the first result that came up when I was looking for an example. There seem to be a few other manufacturers and models out there and they probably have different features.
I am curious what do you think of them? Do you think they are useful? Do you find it more a novelty?
It was an ExplainingComputers video titled Very Useful Small Computing Things that made me think of them.
This isn't true. You need your algorithm and your key to be secure. If the key needs to be remembered or entered often it probably can't be secure. So brute force protection becomes very important.
This isn't true. Yes, with enough time and effort it is possible to extract any data from any device. But in practice physical HSMs do an excellent job at raising the cost of key extraction. I would much rather have an attacker steal my Yubikey than a USB with my GPG key lying on it.