This is a rant about how so many apps on many different platforms (TVs, mobile devices, computers, etc...) have decided to not actually show detailed errors any more. Instead, we get something along the lines of:
Oops, somehting went wrong. Please try again later
.... and then, well, we get to figure out what just happened and what in the world we need to do about it. And good luck with that, since you have no idea what just failed.
Why software developers?!? Why have you forsaken us?
EDIT 24 hours later: I feel like I need to clarify a few things:
I've worked for 8 software companies over 30+ years. I know why putting a DB error into the message users see is a bad idea. I know that makes me uncommon, but I still want more info from these messages.
You all are answering as if there are only two ways this can work: (a) what we have now (which is useless), and (b) a detailed error listing showing a full stack trace. I think the developers could meet me half-way.
What I want is either (a) "Something went wrong on the server, you can't fix it, but we will" or (b) "Something on your end didn't work. Check your network or restart the app or do something differently and then try the same thing again". And if they're blocking me because I'm using a VPN, fucking say so (but that's a whole separate thing...)
Some apps do provide enough info so I have a clue what I should do next, and I appreciate the effort they put into helping me. I think what I am really ranting about is I want more developers to take the time to do this instead of reporting all errors with "Oops, try again". (If the error is in their server, why should I try again?) Give me a hint as to the problem, so I have something to go on.
Cheers y'all. Still love you my techy brothers and sisters.
If it's an error code I've worked around before, apply same troubleshooting.
If its a new errror code, search the error code to see how other people solved it.
If no one else has solved the error code, try analogous troubleshooting, post results online with the error code name, successful or not.
I agree with Nouveau_Brunswick here.
And to add: to @[email protected] , are you not also a user of software and do you not see room for improvement in many apps? That's where I am rn: I just want them to try harder to communicate a tiny bit more info when things go so wrong that a message has to be displayed on my screen. Telling me "There's nothing you can do to fix the problem" would be a big help, for instance. Make sense?
I am a developer of software. I can guarantee you that what you’re asking for would make my job harder, because I’ve done it, and it has made my job harder. If an error is transient (like, a caching layer error, a db connection error, an external API error, an endpoint connectivity error, etc), giving the user an error code will make it more likely that they’ll file a useless bug report or support ticket. The errors are all logged internally, and we can see when there is a spike in the error count. There’s no reason to give the user an error code, because there’s nothing helpful that the user can do with it, and there’s a lot of unhelpful things a user can do with it.
There are times where a message to the user is appropriate, like if they made a mistake with their input. But there are so many things that could go wrong that the user can’t do anything about. You’re not going to work around your DB shard going down, and a replica will replace it in a few seconds anyway, so giving you an error code does more harm than good. Telling you to try again later is exactly what I would tell you if you filed a support ticket. I don’t want to deal with useless support tickets, and you don’t want to deal with useless error messages.
Modern software stacks are big, complex systems with lots of failure points. We monitor them, and we can tell when you see these errors. If we chose to not show you a specific error code/message, there’s almost definitely a good reason.
So what you're saying is that your code is garbage and you're hiding it from users because it's too much work to fix it.
What I’m saying is that error messages can be helpful or harmful. Knowing that and how to tell the difference is what makes you an expert. Just firing off any information to the user without thinking about it is what makes you a novice, and will eventually get you fired. We’re talking about systems with millions of daily users. If you cause 2,000 unnecessary support tickets or forum posts every day because you don’t know when to send what information to the user, you won’t get very far in tech.
If you have 2000 daily people getting error messages, your code is garbage rofl
And if your company would rather you avoid those tickets by not giving out error codes, your company is also garbage. Which to be fair, is a lot of tech companies.
I feel like you really don’t understand how big tech works. There’s not some single server running every service perfectly. There are tons of different layers and services running on thousands or hundreds of thousands of hosts.
Let’s say you make a request to something like Facebook. Say you’re liking a post. Here’s what happens:
That request goes in through a PoP (point of presence). These are sometimes called edge servers or edge gateways, but at Facebook we called them PoPs. This is a server that’s physically close to you that’s used to terminate the TLS connection. It doesn’t have any user data. Its job is to take your encrypted request, decrypt it, then pass it on to Facebook’s regional data center on their internal network.
The request enters a webby. These are usually called frontend servers, but again, at Facebook we called them webbies. This is a server that runs the monolithic Facebook web app. Again, it doesn’t have any user data. Its job is to take your request and orchestrate actions on deeper services to fulfill that request.
First it’s going to check a local memory cache server for sitevars. These control system level switches, like AB tests, and whether certain services are brought down. That server returns the sitevars and the webby proceeds, now knowing which logic paths to take.
For a like, which is a write request between your user account and a post, it will create two DB entries (you likes post, post liked by you). It needs to first get the data from the caching layer, so it will make two requests to TAO (Facebook’s caching layer), one for your account, and one for the post.
TAO runs in the same regional data center, and if it doesn’t have the two data objects cached, it will request them from the regional db shards.
These regional db shards also run in the same data center, and they’ll return the data.
TAO returns the data back to the webby.
The webby (after doing some permission checks, which probably hit TAO again) now creates the two relationships, likes and liked by, referencing the two data objects, you and the post. TAO is a write-through cache, so the webby sends the writes to TAO.
TAO now needs to send the requests to the db primary shards, since they are the only ones that can handle writes. Your primary shard and the post’s primary shard are probably in different data centers, so TAO now passes the writes to the regional data centers for each primary shard.
A host running TAO in each regional data center for each primary shard now passes the write to each shard.
Each primary shard now writes the data to the local disk, and waits for the binary log to be written to the local journal before returning a success message.
The success message is passed from the local TAO host back to the original region’s TAO host.
When that TAO host gets both requests back successfully, it returns a success back to the webby handling your request.
The webby then returns a success to the PoP you’re still connected to.
The PoP then returns a success to the client running on your device.
The client doesn’t notify you of anything, because it already showed you a filled in like button right after you pressed it.
This was how it worked back in 2013 when I worked there. It probably hasn’t changed a whole lot, but this is also an extremely simplified overview (I didn’t even touch on any load balancing systems). That request will probably hit hundreds of services. Some of them can fail and the request could still succeed. But some are required to succeed for your request to be considered successful, like the db write operations. Something like a hardware failure on your primary db shard’s disk can’t be overcome with better code. Nor can a lightning strike taking out the cable connecting your PoP be overcome with better code.
These systems are absolutely massive, and there are failures you wouldn’t even think of. When I worked at FB, we had an entire data center go down because the humidity got just high enough that the capacitors in each hosts’ power supplies all failed in a matter of a few minutes. Thousands of users probably got error messages that day, but the automatic failover systems moved all the traffic to a new region and promoted new primary db shards within about ten minutes. The fact that losing an entire data center was mitigated in about ten minutes is actually really impressive. You might think it’s still garbage code, since users got error messages, but I know enough about these systems to be very impressed by that.
If you know a better way to make a system like this that works for billions of users across the planet, you should write a paper and submit it to a local conference. If they approve you for a talk, you can present your designs to an audience there. If the audience is really receptive, your designs could make a big impact in the tech sector. That’s basically what the highest level engineers at these big tech companies do when they design these multi-billion user systems, so it’s definitely possible for you to do it too.
All I'm saying is that the vast majority of "oops" issues happen before step one. Client-side issues. For those, give an error code. All the stuff you talked about, there's little to nothing users can do. And yeah, it could definitely be done better, but it would require abandoning the "ooh shiny new thing" mentality of tech companies. Updates just to boost resumes, deprecation of anything user friendly. It's an endemic cultural problem.
Why do you think the vast majority of these messages come from client side issues? I worked as a Site Reliability Engineer at Facebook. We had data on client side errors too. Crash logs are sent to the servers when a client side error happens. There’s not really one source that constitutes a “vast majority” of these error messages, but I can tell you that the plurality of them come from the caching layer.
How? I mean how is it possible to send a crash log to the server when the problem is that the client can't connect to the server?
You by definition don't see most of these errors except maybe as pings, depending on why it can't connect
If it’s a mobile app, the operating system handles crash logs, and reports them to you through your app management portal. Then for connection issues to the host or handled errors, you can store that in your app’s data store, and upload them once connection is restored.
If it’s a web app, you can save them in local storage through your service worker, then upload them once the connection is restored. If you don’t have a high level error handling function on your web app, that’s an issue with your web app, not your logging infrastructure.
For a network outage error, these aren’t usually reported if the problem is on the client side, since that’s not something we can do anything about. Both mobile apps and service workers can tell if the operating system is disconnected from the network. If it’s an issue connecting to our host (host is unreachable, but network is online), that’s when we’d save the issue and log it later once service is restored.
We can tell when our services go offline, because we have health checks on our hosts. So, technically, we don’t need client side reporting if our hosts are down. But, every place I’ve worked at has had them anyway.
Analytics don’t usually run on the same hosts as services, so if your service goes down, that doesn’t mean your analytics platform is down. I mentioned before how many systems there are in big tech services. Analytics is one of those systems. It’s generally completely separate from user facing services.
These kinds of error messages are almost exclusively used for transient errors. You aren’t going to work around a transient error. The best thing you can do (the only thing you can do, really) is to try again later, hence, the message. It’s not helpful to show you a message like “cache-1234.example.com failed to respond within 300 milliseconds”. What are you going to do about that? By the time you submit a support ticket, that host has already been brought back up automatically. So now you’ve just wasted your time and the support staff’s time. The engineers already have a log of that error and a log of whatever error brought down that host, so you’re not telling them anything new by making a support ticket.