A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
No spam posting.
Posts have to be centered around self-hosting. There are other communities for discussing hardware or home computing. If it's not obvious why your post topic revolves around selfhosting, please include details to make it clear.
Don't duplicate the full text of your blog or github here. Just post the link for folks to click.
Submission headline should match the article title (don’t cherry-pick information from the title to fit your agenda).
No trolling.
Resources:
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
If you are acting as the proxy for the user, then all web requests go through you. A normal proxy would just forward those on to the Amazon (e.g.) server.
In your scenario, the proxy sends a cert to the client saying 'I'm Amazon', and the client believes it. The client talks to you like you're Amazon, and you read all his messages. At the same time, you talk to Amazon pretending to be the user, so you can get all the correct Amazon responses to send back to the user.
the proxy sends a cert to the client saying ‘I’m Amazon’, and the client believes it
Thanks, could you explain this a bit more? I didn't understand what you mean by "sends a cert saying 'I'm Amazon'"
A decrypting proxy acts like a local certificate authority and issues a certificate on behalf of the site to create one encrypted connection between the client and proxy while creating a second on with the original site on the other side.
The client will get an error (or depending on the site and browser may refuse to connect at all in the case of pinned and preloaded certificates) indicating that they're connecting to a site with a certificate from an untrusted issued unless they've been configured to trust the CA held on the proxy.
The 'break' between the client side connection and the server side connection will be in plaintext and can be examined through internal tools or sent to external ones typically through ICAP.
I see. Which would mean that in cases where the application/OS can be made to blindly trust the "fake" CA (by inserting a root certificate into the certificate store, like in Android), the proxy can simply send the certificate and the client will believe that the certificate comes from Amazon. The certificate list can be refreshed by flushing the cache, yes?
Thanks for the explanation
There are some cases where this would not work by the way. It's called certificate pinning and it's basically when an application comes with the trusted certificate for a host built-in. Even if you were to override it with a root certificate in the certificate store, the app simply wouldn't use it.
Shouldn't flushing the cache mitigate this problem?
No, not at all. The request never hits the cache. The certificate is stored within the app and all internet communication is specifically pinned to said certificate. It doesn't even ask your certificate store.
I see. Thanks, I'll have to rethink the idea in that case.
Yeah, unfortunately it's a huge barrier if you're wanting to see why your devices are phoning home and the data being sent. It makes it extremely difficult if not impossible for most people to bypass.
I understand. What other methods would you suggest to be able to snoop on/decrypt the traffic from my device?
Specifically for attempting to bypass certificate pinning you're solidly in the realm of reverse engineering. I haven't attempting it myself but I have read the efforts of others over the years and the process was quite evolved and ever changing. If you are interested in going down this rabbit hole you may use these links as starting points but be prepared to adapt them.
https://gist.github.com/approovm/e550374428065ff1ecafca6a0488d384
https://codeshare.frida.re/browse
Best of luck.
Thanks, I had heard of frida before, but never tried it. I'll save this comment, many thanks for the pointers!
That's the sum of it. Like others and I have noted some mobile apps (and Apple phones in particular have their entire OS configured to not trust any intercepted certs when attempting to speak to Apple home base) are prone to using certificate pinning and will reject the intercepted certs regardless of the trust store. It's mostly beneficial for adjusting the browser.
If I might ask, what's the purpose of this proxy? Functionally there are a lot simpler and more efficient ways to block traffic from a phone. If it's more for traffic inspection I've seen a couple VPN based pcap apps for Android that could get a lot more detail while a DNS filter could both control and give visibility to traffic from the device without all the cert hassles.
Thanks, I didn't realise that certificate pinning was this strict.
This effort is to check if my mobile has a baseband processor that might be communicating with the internet. I want to know if my device has a backdoor in hardware. The idea with a VPN has me intrigued, could you tell me more about that?
https://play.google.com/store/apps/details?id=app.greyshirts.sslcapture
Something like that should ship all traffic through a local VPN adapter and output a standard pcap file.
Another option if you have a bit more fancy networking available is to set up a security onion instance, then mirror a port on the network and just capture everything at an on-wire level. That would also cover things beyond just web traffic to catch other things like ssh or whatever other remote connectivity could be in play. Seeing the content of the connection is different than just seeing the connections existence though. The endpoint generally has the best visibility before data gets pushed into a a connection but unless you start getting into kernel level debugging it can still be hard to see into the behavior of internal applications.
https://play.google.com/store/apps/details?id=eu.faircode.netguard
This is also a local VPN way of seeing all the outgoing traffic along with allowing control of it as a local firewall.
Hi, thanks for the resources. However, I don't think I'll be able to decrypt the traffic from my mobile using this, yes? Using a VPN for this is a great idea though, this also happens to be how NoRoot firewalls work on Android
That's literally it. It sends a cert for amazon.com, that your client trusts, because the CA cert used to sign it is in your trusted store.
I see, thanks for the explanation. Amazon sends the certificate to the proxy, and the proxy sends its own certificate, masquerading as amazon, to the user.
Thanks
I think it's important to understand how a typical SSL certificate is generated. Basically, there are a handful of companies that we have all agreeded to trust. When you download Chrome it comes with a set of trusted root certificates, so does your OS, etc. So when Amazon wants to create an SSL for amazon.com, the only way they can do that is by contacting one of those handful of trusted companies and getting them to issue a certificate that's says Amazon.com. When you go to the site, you see a trusted party generated the cert and your browser is happy.
When you create a new root certificate and install it on your computer, you become one of those companies. So now, you can intercept traffic, decrypt it, read it, reissue a certificate for amazon.com (the same way Amazon would have gotten one from the third parties), reencrypt it, and pass it along to the client. Because the client trusts you it's still a valid certificate. But if you inspect the certificate on the client side the root signer will no longer be GoDaddy or whatever, it will be you.
Basically, if I were to run a CA locally (on my network) and use that certificate in some OS, I should be able to set up a MiTM setup easily. Except that applications like web browsers don't care and will utilise certificate pinning anyway, regardless of which certificates I keep in my certificate store in the OS.
Also, let's say I'm running HAProxy/Caddy/NGINX at home which acts as my SSL-termination point for my network. In this case, if a service wants to access the internet, do these reverse proxies encrypt the outbound traffic with a different certificate?
If I understand the question, the traffic in your local intranet will basically always be encrypted with your root cert. So client -> proxy with your cert, then normal internet encryption from proxy -> internet.
For the apps, it depends on the app, but you can usually insert your cert into their store, it might just be different than the systems store. This could be hard to do on an non-rooted iPhone, idk. My experience is with Linux desktops. For example, in chromium based apps, there is a database in ~/.pki/nssdb that you can insert your cert into. Again, this is something I do at work where we have a very tightly controlled network and application stack. I would not recommend a MiTM proxy for your home environment, it will only cause headaches.
Ah, I see. I suppose I got a bit confused, and wanted to confirm how encryption would work for outbound traffic. Just to be sure: when an application wants to send encrypted data to a website/service on the Internet, it will use the respective certificate for said website. However, if one runs a transparent proxy in the middle and inserts a root certificate in the certificate store of the application, said certificate becomes valid (and the de-facto pick) for all websites, and this is what the client begins to use for everything.
Is that how it works?
Not quite. This might be a better explanation than I'm providing: https://chat.openai.com/share/c77fc7ed-9d68-4076-ab70-e953a3896bb6
Indeed, that is what I meant to say. Thanks for the link. I'm assuming that transparent proxies have some sort of certificate store in them, since they connect as the user to different websites?
Yes, I think that 'masquerading' is the key bit to grasp. The MITM Proxy isn't just intercepting the traffic, it alters the traffic as it passes through.
Thanks, got it!
Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I've seen in this thread:
| Fewer Letters | More Letters |
|---|---|
| DNS | Domain Name Service/System |
| SSL | Secure Sockets Layer, for transparent encryption |
| VPN | Virtual Private Network |
3 acronyms in this thread; the most compressed thread commented on today has 7 acronyms.
[Thread #237 for this sub, first seen 24th Oct 2023, 15:25] [FAQ] [Full list] [Contact] [Source code]