This study discusses whether or not it’s more useful to wear a parachute when jumping out of an airplane. It comes to the surprising conclusion that it didn’t matter if you wore a parachute or not, you had the same chance of surviving.
The catch, of course, was that the study only looked at drops of 5 feet of height.
Thinking about this in terms of Cybersecurity, this is like doing a study on a very small office and writing it up as if the study was done on the entire Internet. Then at the end of the paper, casually mentioning “This study was done on an office of ten people”, or skipping that part entirely.
Or twisting the words in a write up to make it sound scarier than it is. “This vulnerability will crash every system ever!” sounds terrifying, right? But what if that vulnerability will only work if you have physical possession of the machine as well as having admin access and only on Tuesdays. (I exaggerate here a bit, of course). By the time you have physical possession of the machine and admin access, why would you need to use the vulnerability? You can crash the machine to your heart’s content anyway.
There’s at least two things to take away from this. One, be careful of your study. If you want to be generalizable, then consider your study. Is it generalizable or are you talking about a specific situation? Jumping from an airplane sounds generalizable to all parachuters, until you realize it’s only five feet from the ground. Keep your work relatable.
And try to avoid sensationalism. It’s tempting to attract public attention by making grandiose statements, but really, does that help the situation by crying wolf over minor problems?