[…], from that point the app will be built by f-droid with their own digital signature.
This part of your comment is not quite true. One of the advantages of reproducible builds is that the app can be signed by the developer but fdroid can still verify that it has been built from the correct source code. You can check out the documentation here: https://f-droid.org/docs/Reproducible_Builds/
If an app doesn't support reproducible builds, the version you can download from F-Droid was built and signed by F-Droid, not by the dev