[e2e] Achieving Scalability in Digital Preservation (yes, this is an e2e topic)

[Joe Touch]

On Jul 17, 2012, at 8:50 AM, Micah Beck wrote:

> 
> On Jul 16, 2012, at 6:23 PM, Joe Touch wrote:
> 
>>> One interpretation of end-to-end tells us that in order to improve
>>> the scalability of our solution, we should do less in the channel,
>>> let corruption go uncorrected, and move the work of overcoming faults
>>> closer to the endpoint.
>> 
>> Scalability depends on your metric - are you concerned with archive size, ongoing restoration maintenance (repeated checking and correcting detected errors), or something else?
> 
> I always get tripped up by this point. Perhaps I shouldn't use the term "scalability" which has so many different connotations. 
> 
> What *I* mean by scalability is a solution that can be widely deployed and widely adopted without undue non-linearity in cost and difficulty....

Archive solutions don't necessarily benefits from widescale adoption. If my archive uses one solution for data maintenance, and yours uses another, there's no inherent benefit besides reuse of the solution, unless merging archives is an issue.

For many of your other concerns, this many not be the best list, e.g., archive duration, durability, etc. For some of your concerns, even archivists aren't always thinking in those terms - some approach that of Danny Hillis (see http://longnow.org/).

> Today, we can deploy IP on a cell phone in the middle of the Sahara dessert and interoperate with servers attached to the North American backbone. Today my telephone (Android) and my laptop (OS X) run operating systems whose kernel interfaces are descended from the one that Ken Thompson designed, and which still have a certain interoperable core. Those are designs that *have* scaled. Call it what you will, that kind of design success is my goal when designing hardware or software infrastructure.

IP is designed as a common interoperation layer. OS interfaces similarly define interoperation. What is the interoperation goal here?

Joe