this post was submitted on 23 Apr 2024
68 points (68.5% liked)
World News
32087 readers
989 users here now
News from around the world!
Rules:
-
Please only post links to actual news sources, no tabloid sites, etc
-
No NSFW content
-
No hate speech, bigotry, propaganda, etc
founded 5 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Please avoid citing MBFC as a valid source.
If you’re going to discredit a source, please try to do the legwork of actually discrediting it. A guy with a Bachelors in Physiology and being “fascinated with politics since high school (a long time ago)” cannot be considered a reliable source, nevermind one who claims to follow the “scientific method” which he, presumably, learned while studying to become an occupational therapist or through his 20-year journey of reading political news.
If you have photos of this man, any record of interviews with him, records that support his credibility/the incorporation of his company, records of his job in occupational rehabilitation, details about his team, or anything else, please feel free to share them. Please do not confuse him with Dave E. Van Zandt (Princeton BA Sociology, Yale JD, London School of Economics PhD, ex-managing editor of the Yale Law Journal, ex-Dean of Northeastern’s School of Law, ex-President of The New School).
I don’t understand. Unless you have a degree in journalism or something similar you’re not allowed to be an expert on media outlets? How many professors of practice at universities don’t have a degree related to what they’re teaching?
Don’t get me wrong, I’m super put off by this notion that he had a “super keen eye“ and natural aptitude for spotting “bias.” I also object to the way that people talk about bias, but that’s another discussion. The point is yeah there’s a little bit of bullshit in there, but his background does not discredit the endeavor.
Professors of Practice tend to have experience in the industry they are professors in. Their reputation is hinged on their achievements, and they don't cite their degree as being instrumental to their credibility.
Edit: professors are also, y'know, subject to scrutiny and can't hide behind anonymity when they get things wrong.
The site's history speaks for itself. Because or in spite of him, it's a solid way to at-a-glance assess an outlet. It is not the whole story, it's not even a great story, but it's a start that's pretty solid.
How would you support this claim? It's solid because it exists and people read it?
Burden of proof is on you here. What about the site are you disputing here?
It's credibility and reliability, which I've already done and which you've acknowledged.
Just do the legwork to critique the source, it's not that hard. There's no need to cite bad sources just because they exist.
You need to show it’s a bad source. Discrediting the founder does not satisfy that requirement.
i'll bite:
i went to the media bias fact check page for radio free asia, pushed control-f and typed "cia". there were three hits, as part of the words "politicians", "appreciate" and "social".
radio free asia was literally founded by the cia as an anticommunist us propaganda mouthpiece.
well, maybe they don't exactly use those words but they might basically say the same thing... what does mbfc's rfa history section look like?
well, that's glossing over and avoiding some important points, but at least they're admitting it's promoting "USA interests with a less direct propaganda approach". lets see how they score a source they described as literal government propaganda mouthpiece:
oh, the US government propaganda outfit serving "content in nine Asian languages for audiences in six countries" is left-center and highly factual! Who would have known!
the thing that makes media bias fact check a bad source is that it relies on a one dimensional left-right bias continuum and another one dimensional veracity continuum.
anyone with their head screwed on straight, no matter their personal politics or country of origin can tell without a shadow of a doubt that rfa isn't a good source because it's a propaganda arm of the us government. when evaluated on the metrics of leftness or rightness under the rubric of mbfc though, it shows up as "left-center" and when put to the test of authenticity by mbfc it is determined to be highly factual.
media bias fact check is a bad source. it cannot, by design, communicate the reality of a source's bias because the way it evaluates bias is constrained by and i'd say warped into only what fits it's highschool-in-1999-ass rubric of bias and accuracy!
The OP is using this "source" to discredit other sources. If you're going to disprove another source, prove that your own source is legitimate in spite of the questions regarding its credibility.
It's just an ad hominem with extra steps.
yeah, pretty much. They need to show us an example of why it’s not effective at its mission. Preferably not just pointing to the founder and saying “he doesn’t have the proper degree.“
I don't think you quite understand what an ad hominem attack is. The fact is, the operator of MBFC has no accountability if they get anything wrong because nobody knows who or what he is. The fact is, the operator of MBFC uses his degrees and experience as justification for his "scientific" evaluation of media bias.
I'm not making any claims that the operator isn't making themselves.
Ok you’re right have a good one
You have not done any "legwork" to discredit MBFC. Your personal opinion is that the owner/author doesn't have appropriate credentials/experience, but you haven't actually demonstrated that he is not credible.
A person without credentials, without experience, and without any evidence to prove that their claimed credentials or experience are legitimate... Is a credible source?
Can you find any evidence, any at all that the person actually has the credentials that they themselves claim? This is trivial to do for pretty much any modern journalist, but I've been able to find zero information on him.
Nope, you are making the claim that the information presented on MBFC is not credible, it is up to you to substantiate that claim. Claims made without evidence can be dismissed without evidence.
Your claim is that... Credibility exists unless disproven? Consider that for a minute.
Nope, my claim is that you haven't substantiated your claim with anything more than your own personal opinion. And look at that, my claim is supported by all of your comments continuously failing to present anything more than your personal opinion. QED.
Get some sources. Or get quiet.
"[MBFC's] subjective assessments leave room for human biases, or even simple inconsistencies, to creep in. Compared to Gentzkow and Shapiro, the five to 20 stories typically judged on these sites represent but a drop of mainstream news outlets’ production." - Columbia Journalism Review
"Media Bias/Fact Check is a widely cited source for news stories and even studies about misinformation, despite the fact that its method is in no way scientific." - PolitiFact journalists
Journalists seem to agree with me, which you'd know if you actually read "all of my comments." This isn't the first time I've posted these quotes in this thread.
Link the direct sources you're quoting from. I won't go hunting for you.
Cool.
MBFC is a good enough source for routine information, and its system is accurate enough to give a general idea of who finances, who writes, and whether the articles are sourced according to journalistic standards. It’s a good tool to help with critical evaluation of media sources. But you’re right: it’s not flawless.
Your attack on the founder is an ad hominem attack, and I don’t think it’s relevant. Are you suggesting that people can only learn things through a university education?
Besides, it’s often cited by university sources and experts as being a decent enough indicator of reliability and bias, if not necessarily held up to standards of something like a peer review.
It’s a tool to be used in conjunction with critical thought and evaluation of the source itself, and for that I think it’s rather accurate and useful.
Thing is, even if he is good at media criticism, there’s no stakes for him. Nobody knows who he is, what he looks like, he has nothing on the line, and his credibility in his primary occupation cannot be harmed if he is wrong.
Nevermind that he lacks the credentials nor any legitimate scientific expertise, and yet claims that his Bachelor’s in Physiology was sufficiently advanced to teach him everything he needs to know about the scientific process.
The dataset is seen in academia as being accurate enough to train machine learning models for or to make aggregate claims on. Machine learning models are not the bastions of truth, nor are their datasets.
This reads like an argument against open source projects in general lol
You can trivially verify that an open-source project works. Good luck verifying a subjective rating.
Machine learning has nothing to do with this. I am referring to academics who study journalism, communication, political science, or sociology.
And it’s doesn’t really matter who he is at this point, the product he created works well and continues to be a reliable source to interrogate media sources.
I am happy that a person is able to create such a useful product, maintain it and continue to prove reliability in the product, and maintain anonymity. I certainly would want to remain anonymous if I was creating something that actively worked to check people’s information bias.
But it’s an irrelevant discussion: who he is doesn’t really matter when evaluating the work of the site itself.
"[MBFC's] subjective assessments leave room for human biases, or even simple inconsistencies, to creep in. Compared to Gentzkow and Shapiro, the five to 20 stories typically judged on these sites represent but a drop of mainstream news outlets’ production." - Columbia Journalism Review
"Media Bias/Fact Check is a widely cited source for news stories and even studies about misinformation, despite the fact that its method is in no way scientific." - PolitiFact journalists
MBFC is used when analyzing a large swathe of data because they have ratings for basically every news outlet. There, if a quarter or a third of the data is wrong, you can still generate enough signal to separate from noise.
It absolutely matters who is running a site because there's an inherent accountability for journalism. There's a reason you don't see NYT articles from "Anonymous Ostrich."
I accept your point about why it matters who runs the site. I would just argue that in this case, it’s not as relevant because the goal seems to legitimately be information transparency, which is consistently delivered across its work. Its findings are at least generally reproducible. But no it’s not scientific. I believe I’ve stated that already, however it’s a good indication of reliability of a source.
Yes, human bias creeps in, hence my point of using it alongside general media literacy and critical thinking when evaluating media.
It aggregates and analyzes a ton of sources, and gives generally accurate information about how they are funded, where they are based, and how well the cite original sources. These are all things that can be corroborated by a somewhat systematic reading of the sources themselves.
An LLM also "aggregates and analyzes a ton of sources, and gives generally accurate information about how they are funded, where they are based, and how well the cite original sources."
That doesn't make an LLM a useful source.
We don't allow LLM-generated summaries as news stories. Do the legwork, use these tools to start if you want to, but don't cite them as though they are gospel.
What are you talking about? LLMs have no bearing in this conversation, you brought them up.
Are you saying that you don’t allow people to use tools to evaluate media; and share their reasons for scepticism?
The bit that I quoted from MBFC is factual information (the story’s sponsors and an assessment of reliability), which I used to begin a conversation about the source.
Which upon further discussion was, indeed, ultimately sourced to a Syrian governmental agency, which is then been repeated by various governmental sources. There has not yet been any evidence to support the allegations made by the original source, which supports MBFC assertion that the original news agency does not often provide reliable (by journalistic standards) justification for its news stories. It seems like a really weird idea for you to so vehemently oppose a resource that enables critical thinking.
The news article is an extension of at least one state agency, and there are critiques of its truthfulness. That’s the takeaway from my original comment.
I feel like I’m repeating myself, but I literally cannot fathom a good faith justification for not allowing a widely accepted tool for media literacy to be allowed here. (For clarity, I’m talking about MBFC, not any LLM stuff, which only serves to obfuscates things.)
This is all true, and comes
I cannot fathom a good faith justification for allowing a resource that intentionally obfuscates the media landscape in an effort to compress the entire landscape onto a 2D plane from a person who cannot be found through any conventional means and very well may not exist. Their methodology is bunk for a number of reasons, but we'll focus specifically on how they evaluate factuality.
As you know, op-eds typically fall under different journalistic purview than news stories. This is as true for the NYT and SCMP (newspapers of record) as it is for Breitbart. Mixing the factuality rating for op-eds and news stories is rather questionable.
The rating scheme works by sampling (how? nobody knows) a small number of stories from each paper and evaluating their factuality. This destroys the validity of the data, as different news sources cover different stories and categories of stories vary in factuality. For example, a paper which records the daily weather temperature in Toronto would be "very highly accurate" even if they release a story saying that water is dry and trees are fake once a month. Because of the limitations of sampling, their methodology leads to inherently skewed results.
The definition of propaganda used is... Unclear. This is obvious as statements made by the US government and repeated by other news agencies are not considered propaganda, despite their factual inaccuracy. For example, "40 beheaded babies" (later demonstrated to be false) and "we [the United States] have the most sophisticated semiconductors in the world" (literally, provably, false because TSMC's Taiwan fabs are the clear and undisputed leader).
They fail to do due diligence on sourcing because of a (I assume) lack of experience. For example, in their critique of their article "Fake data - the disease afflicting China's vaccine system," they say that the article is poorly sourced because it lacks hyperlinks. The article in question cites: a Hong Kong microbiologist (by name), a professor at the University of Hong Kong (by name), the WHO, stories published in the China Economic Times, data from the State Drug Administration, a law case against Changsheng Biotech, and an unnamed head of a disease control center in China. This, they claim, is a use of "quotes or sources to themselves rather than providing hyperlinks." Their evaluation of "sourcing" seems to be dependent almost entirely on the usage of hyperlinks.
They fail to consistently apply standards applied to smaller news outlets (such as Al Jazeera) to larger news outlets (such as the New York Times and CNN). Against Al Jazeera, they claim that wordplay is used that is negative towards Israel. However, as covered by The Intercept and The Guardian, the New York Times and others have just as extreme (if not more extreme) policies surrounding wordplay that is used to show Israel in a positive light. In major newspapers, for example, the words "slaughter," "massacre," and "horrific" are reserved almost exclusively for Israeli deaths rather than Palestinian deaths.
MBFC is not consistent with the sources of their fact checks. Against Al Jazeera, they point to "The forgotten massacre that ignited the Kashmir dispute" as not crediting the image correctly. In fact, the caption describes exactly what the image shows, which is exactly what the original source for the image (which they cite) claims.
I can go on...
Again, if it's trivial to do the legwork and discredit a source anyway, then do that. If it's not, then don't outsource the work just because you don't understand it.
We can talk about how it assesses factuality, but it’s not really relevant to my particular use of MBFC, since I quoted how the media of the OP is funded, which is incredibly relevant.
The existence of op-eds and their content is a useful indicator of where a particular media entity sits. Their editorial standards also reflect the kind of language a source routinely allowed. It’s a good indication of what the outlet is willing to publish.
What is your critique with how it states it samples? It’s a sample of a media source for a qualitative and subjective assessment. I, too would like to know more about how it samples, but I can also see the framework that it follows to assess factuality and confirm or dispute it through a quick look at the headlines and by skimming through some stories, if it seems warranted (though I admit, when it comes to sensationalized headlines and incendiary language, or an obvious government agenda I won’t necessarily do all my due diligence to assess a media source… like I did with the OP).
As for your specific concerns about factuality, you chose some random articles and engaged with them specifically but didn’t link them here, so I’m not going to do your job and go and find the thing you’re talking about.
To your last comment: it’s not always trivial to do the legwork. There is a lot of media out there, and it’s just getting more and more overwhelming. MBFC is just a tool. You have to be aware of the dangers when using a tool. Your critiques are all somewhat valid, but you’re advocating for throwing out a useful tool for media literacy because it’s not perfect.
It's entirely relevant. If a source is bad as a whole, the foundation of trust you evidently have for it is built on sand.