[e2e] was double blind, now reproduceable results

[Joe Touch]

Jonathan Stone wrote:

> In message <40ACD7E8.3060004 at isi.edu>Joe Touch writes
> 
> [...]
> 
> 
>>>>I don't disagree with making all the data available, and that is almost 
>>>>always possible, even where reproducibility is hard. 
>>>
>>>With the notable exception of whole-payload packet traces.  If you can
>>>get permission to collect them at all, its well-nigh impossible to
>>>redistribute them. For obvious and excellent reasons.
>>
>>It ought to be possible in those cases - if not required - to provide 
>>reasonable access to run second-party tests to confirm results.
> 
> [...]
> 
> 
> Oh, absolutely. I've been on both sides of that fence, and that sounds
> reasonable to me.  Much of the data I published was from researchers
> (such as Vern) who kindly ran my tools ``under escrow'', against their
> own traces.  I never saw the raw traces. The tools were validated
> against synthetic test data, and against my own traces.
> 
> But getting permission to collect whole-packet traces may well impose
> conditions more restrictive than you, or I, or Jon might wish from
> solely a resarch perspective.
> 
> Stanford stipulated I could not disclose *anything* about my
> whole-packet traces, without first clearing it with the appropriate
> campus authorities.  For example, they got veto rights over early
> drafts of my papers, to ensure I didnt disclose anything untoward.
> That same veto would apply if I were to run third-party tools over
> those traces. Getting permission to run second-party tests might
> be a difficult, time-consuming task.
> 
> Traces collected from ISPs may (if the ISP is also a telco) also have
> to pass the `statistical measurement only' regulatory requirements for
> telcos; but I have no idea what those are, these days.

People in the biological sciences are familiar with these sorts of 
restrictions. I'm not surprised by it either; it matters to me only when 
the restrictions compromise the information shared - i.e.,  if I believe 
that residence location matters for a cancer study, I need to publish 
enough information, or at least allow others _WHO SIGN THE SAME 
AGREEMENTS_ to get at the data.

The issue is not whether the data is truly public, but 'public enough' 
for real scientific validation.

Jo
e
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 254 bytes
Desc: OpenPGP digital signature
Url : http://www.postel.org/pipermail/end2end-interest/attachments/20040520/f39389bb/signature.bin