From: Adam Pease ([email protected])
Date: 01/30/02
I sent this out Monday but it bounced, so I'm resending. Sorry for the delay. Folks, I volunteered to provide a few references of "lessons learned" from HPKB that might be relevant to the DAML experiment. - Example metrics. A key point that Paul Cohen made during HPKB is that many research efforts in AI are just "existence proofs" i.e. did the program work or not. This is very different from quantitative metrics about how well or how fast a system worked. Key HPKB metrics were questions answered correctly in particular domain (that was well-specified and scoped in an experiment document of nearly 100 pages for the final version) as well as rates of knowledge creation. - Bob Shrag's final HPKB presentation gives a good set of example test questions, scores and metrics. <http://reliant.teknowledge.com/HPKB/meetings/FinalPImeeting/Schrag-Results/sld001.htm> - Level of detail. I think there was some consensus during the telecon that although the work done to date to specify the experiment was necessary and helpful, it was not at a sufficient level of detail to ensure that the experiment isn't either (1) very hard, "AI-complete" or open-ended or (2) a trivial thing that could be done in Java or with a relational database. The HPKB Crisis Management spec is a good example of the level of detail and scoping needed in an experiment spec that researchers can act on. A full set of those materials can be found at <http://www.iet.com/Projects/HPKB/Y2/> Additional material in the same vein for the RKF project can be found at <http://www.iet.com/Projects/RKF/> Adam At 12:42 PM 1/28/2002 -0500, John Flynn wrote: >All, > > > >This is a reminder that the teleconference to discuss the DAML Experiment >is scheduled for today at 3:00 pm East Coast time. A briefing to support >these discussions has been posted at: > > > ><http://www.daml.org/JF>http://www.daml.org/JF > > > >Dial in info: > >Domestic: 1-800-403-2024 > >International: 865-673-6716 > >Participant code: 671932 > > > >Thanks, > > > >John > > > >John Flynn > >(703) 284-4612 > >DAML Integration and Transition PM > >BBN Technologies > > Adam Pease Teknowledge (650) 424-0500 x571
This archive was generated by hypermail 2.1.4 : 03/26/03 EST