Tuesday, September 09, 2008
The 5% solution
I posted this recently in response to UFO Updates, Errol Bruce-Knapp's excellent UFO discussion forum. My submission was because of a thread regarding the Condon Committee's stance that few UFOs were unexplained, yet a review of their data showed that more than 50% of their cases were unexplained. Discussion ensued concerning how skeptics leap on the suggestion that only about 5% of all UFO reports are unexplained, and that even these might be explained away, leaving nothing at all.
My contribution:
=================================================================================
List:
I really hate to wade into this again, but here goes nothing...
Don Ledger noted:
> You, myself and a few others have been making the point for some
> time that the "thrown bone" of 5 percent unexplained is not
> supportable. Never has been in the cases I've personally come
> investigated. I suspect that investigators and to some extent
> researchers have grabbed onto the admitted 5 percent allowed by
> some nebulous authority. Ufology saw this as a thin wedge and
> proof of true unknowns. On the other hand however that authority
> could claim that there wasn't enough detail to prove that they
> were IFOs.
I know that Brad and others have attacked me for daring to publish our results showing that only a small percentage of UFOs are high-quality unknowns, but to quote Stan Friedman, it seems to be a matter of: "I've made up my mind, don't bother me with the facts."
The reality is that when we started doing our case-by-case analysis many years ago, I was actually surprised that when we actually looked at each case in the files, the percentage of high-quality unknowns was really not very high. In fact, the percentage ranged between 2 and 10 per cent. Overall, on first pass, the percentage of unknowns from the raw data us about 15 per cent, but when we sat down and went through the cases, we could whittle that down to a smaller number of "good cases" each year.
(Take a look at our studies available online at: survey.canadianuforeport.com)
[I think the reason why the Condon Report had more than 50 per cent unknowns is because they seemed to only publish details on well-investigated cases. (One case in particular that I have studied and written about extensively, the Falcon Lake case, is Case 22 in the Report, and I note in my lectures that it's listed as "unexplained.")]
I'm working with Geoff Dittman now on a 20-year longitudinal analysis of Canadian UFO reports. It's been two decades since we began compiling case data and publishing the annual Canadian UFO Survey as of next year, and I am going back to combine all the years' data into one file for a thorough analysis. (This is a difficult job for many reasons I can talk about later.) I started with the 1989 reports and began sorting through them one by one. No matter what the source: military, police, UFO groups, NUFORC, Vike, or UFOROM, the majority of reports are simple lights in the sky, stars, planets, fireballs, meteors, aircraft and balloons. Only a small percentage each year are unknowns, and only a smaller number are "good"
cases.
How, then, do we reconcile the difference between the perceived impression that the percentage of unknowns must be higher than 5 per cent with the result of looking at the actual data?
First of all, I must say that I have great respect for Don Ledger. He has been invaluable in helping me with UFO case investigation in the Maritimes year after year. I cannot disagree with his view that the percentage of unknowns within cases he deals with is higher than 5 per cent.
All I can do is note that in the data we have available to us, and this includes RCMP files, National Defence as well as civilian reports, the percentage of unknowns is relatively low. The much-heralded MoD release of British cases supports this too; most of the official British cases aren't that interesting. (Note: In Canada, we have had access to these kinds of "official UFO reports" for many years.)
One obvious solution to the paradox is to suggest that the databases themselves are flawed. Brad Sparks has argued that we're including too many IFOs instead of UFOs, hence the percentages are skewed. But when you look at the labels for the databases (particularly civilian websites), they all describe "UFO" reports. Even the cases we get from RCMP and military sources are actually titled "UFO Sighting" at the top of the standard form, and most of these turn out to be stars or planets.
So, I doubt this is the case. The reality is that when one looks at raw UFO report data, only a relatively small percentage is categorized as "unknown."
As for the problem that debunkers use the small percentage as fuel for their argument that the unknowns are only outliers and "noise" in the data, this is not supported. As Stan Friedman notes in his lectures (and books), even the USAF Special Report tables show that the unknowns are fundamentally different from the knowns, insufficient data cases and explained cases. Remember, we note in our annual surveys that the high-quality unknowns are not filed as "possible explanation" cases or "insufficient information" cases. These are relatively well-investigated UFO cases that do not seem to have prosaic explanations such as stars, planets, aircraft, fireballs, balloons or the like.
I would suggest that ufologists not deny that the percentage of unknowns is low in the body of UFO data. I don't think there's any reason to worry that debunkers will use this fact as ammunition to attack UFO reports in general. Big deal. The facts speak for themselves. There are a dozen or two good cases every year in Canada; probably a factor of ten more than that in the USA. Many more around the world.
These are the cases to focus on. I don't worry about the IFOs, but I need to identify which cases are which through solid case investigation, research and analysis.
Chris Rutkowski
My contribution:
=================================================================================
List:
I really hate to wade into this again, but here goes nothing...
Don Ledger noted:
> You, myself and a few others have been making the point for some
> time that the "thrown bone" of 5 percent unexplained is not
> supportable. Never has been in the cases I've personally come
> investigated. I suspect that investigators and to some extent
> researchers have grabbed onto the admitted 5 percent allowed by
> some nebulous authority. Ufology saw this as a thin wedge and
> proof of true unknowns. On the other hand however that authority
> could claim that there wasn't enough detail to prove that they
> were IFOs.
I know that Brad and others have attacked me for daring to publish our results showing that only a small percentage of UFOs are high-quality unknowns, but to quote Stan Friedman, it seems to be a matter of: "I've made up my mind, don't bother me with the facts."
The reality is that when we started doing our case-by-case analysis many years ago, I was actually surprised that when we actually looked at each case in the files, the percentage of high-quality unknowns was really not very high. In fact, the percentage ranged between 2 and 10 per cent. Overall, on first pass, the percentage of unknowns from the raw data us about 15 per cent, but when we sat down and went through the cases, we could whittle that down to a smaller number of "good cases" each year.
(Take a look at our studies available online at: survey.canadianuforeport.com)
[I think the reason why the Condon Report had more than 50 per cent unknowns is because they seemed to only publish details on well-investigated cases. (One case in particular that I have studied and written about extensively, the Falcon Lake case, is Case 22 in the Report, and I note in my lectures that it's listed as "unexplained.")]
I'm working with Geoff Dittman now on a 20-year longitudinal analysis of Canadian UFO reports. It's been two decades since we began compiling case data and publishing the annual Canadian UFO Survey as of next year, and I am going back to combine all the years' data into one file for a thorough analysis. (This is a difficult job for many reasons I can talk about later.) I started with the 1989 reports and began sorting through them one by one. No matter what the source: military, police, UFO groups, NUFORC, Vike, or UFOROM, the majority of reports are simple lights in the sky, stars, planets, fireballs, meteors, aircraft and balloons. Only a small percentage each year are unknowns, and only a smaller number are "good"
cases.
How, then, do we reconcile the difference between the perceived impression that the percentage of unknowns must be higher than 5 per cent with the result of looking at the actual data?
First of all, I must say that I have great respect for Don Ledger. He has been invaluable in helping me with UFO case investigation in the Maritimes year after year. I cannot disagree with his view that the percentage of unknowns within cases he deals with is higher than 5 per cent.
All I can do is note that in the data we have available to us, and this includes RCMP files, National Defence as well as civilian reports, the percentage of unknowns is relatively low. The much-heralded MoD release of British cases supports this too; most of the official British cases aren't that interesting. (Note: In Canada, we have had access to these kinds of "official UFO reports" for many years.)
One obvious solution to the paradox is to suggest that the databases themselves are flawed. Brad Sparks has argued that we're including too many IFOs instead of UFOs, hence the percentages are skewed. But when you look at the labels for the databases (particularly civilian websites), they all describe "UFO" reports. Even the cases we get from RCMP and military sources are actually titled "UFO Sighting" at the top of the standard form, and most of these turn out to be stars or planets.
So, I doubt this is the case. The reality is that when one looks at raw UFO report data, only a relatively small percentage is categorized as "unknown."
As for the problem that debunkers use the small percentage as fuel for their argument that the unknowns are only outliers and "noise" in the data, this is not supported. As Stan Friedman notes in his lectures (and books), even the USAF Special Report tables show that the unknowns are fundamentally different from the knowns, insufficient data cases and explained cases. Remember, we note in our annual surveys that the high-quality unknowns are not filed as "possible explanation" cases or "insufficient information" cases. These are relatively well-investigated UFO cases that do not seem to have prosaic explanations such as stars, planets, aircraft, fireballs, balloons or the like.
I would suggest that ufologists not deny that the percentage of unknowns is low in the body of UFO data. I don't think there's any reason to worry that debunkers will use this fact as ammunition to attack UFO reports in general. Big deal. The facts speak for themselves. There are a dozen or two good cases every year in Canada; probably a factor of ten more than that in the USA. Many more around the world.
These are the cases to focus on. I don't worry about the IFOs, but I need to identify which cases are which through solid case investigation, research and analysis.
Chris Rutkowski