I’m going to digress for a moment into Chemistry because I think it makes my point clearer.
Imagine you’re doing a litmus test on water with the standard paper slips from chemistry class. These paper slips will allow you to determine the ph of the water. You get some water, test it, and record the results. This is part of a larger experiment involving water. At the end of your experiment, you collate your results and report them. Unfortunately, you forgot to include the ph of the water in your results.
Someone else decides to replicate your experiment. Replication lets people validate results and sometimes build on them, so it’s a good thing. The problem is, they can’t get your experiment to work.
Since you neglected the data point that included the ph of the water in the experiment, then the person trying to replicate it used their own water that had a completely different ph.
This seemingly small change in your experimental results completely changed the experiment and the replicators couldn’t make it work. Just omitting that one data point was enough so that replication was broken.
What does this mean in Cybersecurity? It means explain your method completely. If I’m doing malware research and tell you my method and results, but neglect the important part of “I was studying Linux malware and collected it in this manner,” then a replicator might try my method on PE32 malware and fail to get similar results.
DTRAP supports transparency in methods and in data. Tell us how you did it and explain clearly what data you used. Imagine reading a paper and thinking “Yes, that method will solve my problem!” then discovering that the method is missing key steps. Tell me how you did it, what you used to do it, and tell me everything I need to do so I can replicate it.
Submit your paper at https://dtrap.acm.org/ we look forward to reading it.
