In the field of Cybersecurity, we often research in a controlled or simulated environment such as a lab. The problem with the sterile environment of the lab is that this doesn’t model the wild west nature of the Internet.
As an example, consider DDoS (distributed denial of service) attacks. Most practitioners have dealt with a DDoS at some point and would love an effective defense against them. DDoSes have a real business cost, making their cause and remediation an impactful research area.
The question is what the best research target is, that is what kind of DDoS matters. There are volume-based attacks, application-based attacks, and protocol attacks, for example. Not all kinds are equally impactful on practitioners. But researchers may instead gravitate to new, hot problems rather than trying to understand what impacts practitioners. In fact, what types of DDoS attacks incident responders and system admins care about seems like a great (and unstudied) research question.
This leads us to the question of external validity. External validity is the extent to which results generalize to populations outside the lab. Are your results generalizable to the whole of Internet users or some clear subset of them, or do the results hold only in your specific lab setting? For useful results in the field of Digital Threats, lack of external validity is a concern and all researchers and practitioners should aim for producing and consuming externally valid results.