From keld@dkuug.dk Mon Aug 28 01:44:21 2000 Received: from rap.rap.dk (24.ppp1-17.worldonline.dk [213.237.1.152]) by dkuug.dk (8.9.2/8.9.2) with ESMTP id BAA32453; Mon, 28 Aug 2000 01:44:20 +0200 (CEST) (envelope-from keld@dkuug.dk) Received: by rap.rap.dk (Postfix, from userid 500) id 9F2A75C14; Mon, 28 Aug 2000 01:39:37 +0200 (CEST) Date: Mon, 28 Aug 2000 01:39:37 +0200 From: =?iso-8859-1?Q?Keld_J=F8rn_Simonsen?= To: "D. J. Blackwood" Cc: =?iso-8859-1?Q?Keld_J=F8rn_Simonsen?= , ISO 14766 Mailing List Subject: Re: (iso14766.38) (More) Comments on 14766 WD4 Message-ID: <20000828013937.A945@rap.rap.dk> References: <200007271600.SAA15023@dkuug.dk> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Mailer: Mutt 1.0.1i In-Reply-To: <200007271600.SAA15023@dkuug.dk>; from djblackwood@attcanada.net on Thu, Jul 27, 2000 at 11:59:48AM -0400 On Thu, Jul 27, 2000 at 11:59:48AM -0400, D. J. Blackwood wrote: > The attached Word 6.0 document contains text that should go on page i of the > document immediately before the table of contents, and on pages iii and iv > immediately following the table of contents. The table of contents itself > should be renumbered ii. The page starting with section 1 then becomes page > 1. OK, added and I edited it somewhat. > The following new section should be inserted before the existing section > 6.4: > > "6.4 Conformance Testing > > While a profile in and of itself is not a testable entity, the underlying > infrastructure components are testable and the performance goals of the > profile are measurable and therefore testable. The technical infrastructure > of the profile is testable in both its accuracy of conformance to the > pertinent standards and its ability to successfully address the cultural and > linguistic requirements. > > In testing a claim that a particular implementation conforms to a POSIX > National Profile, a systematic approach should be taken, for example, > > — Testing individual claims of conformance to the base standards or > specifications (Some or all of these claims may have been validated in a > previous testing campaign.) > > — Testing the aggregation of all the claims of conformance to the base > standards or specifications > > This latter case may require that many interactions have test cases that may > give rise to an unworkable number of tests being required. However, testing > would be restricted to observable behaviour, which may be a subset of > behaviour defined in a base specification. > > Conformance testing per se does not guarantee interoperability; it is only a > test of conformance to a set of test assertions based on the standard. One > way to measure conformance is to develop a reference implementation of the > particular standard or standardized profile. > > The system then is “exercised” through the use of test scripts. The > behaviour of the system is monitored and compared with the expected outcome > from the reference implementation. The advantage of this approach is that > many vendor products can be tested, thus spawning competition. If a > reference implementation is not available, other conformance test methods > could be used, such as software unit testing, software qualification > testing, and integrated hardware/software testing. > > Although conformance testing does not ensure interoperability, such > inter-working would be virtually impossible without conformance to > standards. Interoperability testing is a matter for vendors and users, > rather than standards setting organizations. > > Many current standards have registered conformance tests. The JTC1 > administers an index of registers of conformance tests that is generally > available. When a product is found on such a registry, it can be assumed > that it has “passed” a battery of test procedures. The availability of test > registries greatly simplifies the hardware/software qualification testing of > a system, thus saving time and money for the developer." > > The existing section 6.4 then becomes 6.5. The existing section 6.6 should > have the following text added immediately before 6.6.1: > > "Once a POSIX National Profile has been developed, it is still necessary to > document the implementation conventions associated with each of the selected > standards and/or specifications. This final step is required to prevent > interoperability problems." OK. > The following new section should be inserted before the existing section > 6.7: > > "6.7 Using PASs > > PASs are specifications (e.g., industry profiles, de facto standards) that > have been developed outside the approved process for open standards. For the > purpose of this technical report, PASs are widely implemented and the > documentation is available in the public domain. Examples of widely used > PASs are the RFCs and standards from IETF. The JTC1 has recognized two > procedures for incorporating PASs into standardized profiles. The first is > to reference the PAS directly; the second is to convert the PAS into a > formal standard and then reference that formal standard." OK. > Dave > -- > D. J. Blackwood