Testing and Interoperability
The main aim of standardization is to enable interoperability in a multi-vendor, multi-network, multi-service environment. The absence of interoperability must not be the reason why final services for which there is great demand do not come into being.
ETSI is very much aware of these developments and market demands. It knows what the inhibitors to interoperability are that can be encountered during the standards development process.
Clarity of standards
Incomplete, unclear standards with poorly specified options can contribute to the biggest single cause of non-interoperability, namely that the unfortunate implementer is forced to make potentially non-interoperable design decisions on critical parts of the system based on a lack of information. Standards need to be designed for interoperability from the very beginning. Interoperability is not something that will somehow get fixed at the end of the process. Ensuring interoperability is the red thread running through the entire ETSI Standards Making Process from day one. We call this standards engineering.
Natural languages (English, French, etc) are frequently imperfect, and misunderstandings easily occur. Such languages are therefore inadequate for certain parts of specifications and standards, where precision and the elimination of ambiguity are essential. Various specialized languages (often based on mathematical principles) have been developed and, in its aim to produce high-quality standards, ETSI has become a world leader in the use of techniques such as:
- Specification and Description Language (SDL)
- Message Sequence Charts (MSC)
- Abstract Syntax Notation No. 1 (ASN.1)
- Testing and Test Control Notation (TTCN)
- Universal Modelling Language (UML)
- - and so on.
Validation of standards
Many standards are complex documents and, even with the use of specification languages, ambiguities and errors may occur. ETSI relies on feedback from validation and testing activities, and from routine use of the standards, to correct and improve the documents. The Institute's procedures permit rapid updating of its standards in a controlled manner.
The development of standardized test specifications is an integral part of the ETSI strategy for ensuring interoperability. Much of the industry we serve recognizes, and places great importance in, the ability to test products and services in a standardized manner. For example, the use of ETSI conformance test specifications in the Global Certification Forum (GCF) certification of GSM and UMTS™ handsets guarantees interoperability of these terminals over the air interface.
Reflecting the principle: test the components first, then test the system, ETSI focuses on the development of two types of test specifications:
- conformance test specifications
- interoperability test specifications.
Verifying implementations and validating standards
Since interoperability is what most standards are about, interoperability testing events, such as ETSI's 'Plugtests™ events, are popular among the industry. These events enable developers from different (and competing) companies to get together to test their companies' own implementations and ensure interoperability between products. In addition, these events are immensely valuable in validating a draft standard, and identifying and removing ambiguities and misinterpretations that may exist.
The organization of interoperability events is challenging. An absolute must for a successful event is a flawlessly working test-bed which, depending on the standard being tested, can be complex. ETSI's Plugtests™ service has the experience and resources to manage such events, even for technologies that are outside the scope of our standardization activity.
ETSI Centre for Testing and Interoperability
The ETSI Centre for Testing and Interoperability (CTI) provides direct support and assistance to ETSI technical committees on the application of these techniques in standards and other documents. The CTI has wide experience in the specification of information and communication technologies and can offer assistance with:
- Specification methods and the use of good technical practices in standards
- The application of validation, analysis and simulation techniques
- Testing methodology (based on ISO/IEC 9646)
- Methodology and language tutorials
- Instruction on the use of tools used at ETSI
- Reviews of standards that contain formal methods, etc.
The CTI works closely with ETSI's MTS technical committee (Methods for Testing and Specification).
The principal task of the CTI is the planning and development of conformance and interoperability test specifications. Many of these are used in external certification schemes such as those of the GCF, the DECT™ Forum and the WiMax Forum. The tests are validated either in-house (as in the case of our IPv6 test specifications), in collaboration with commercial test laboratories and recognized test tool suppliers, or among the ETSI membership (e.g. GSM, UMTS).
The CTI also contains the ETSI Plugtests™ Service. This service specializes in the organization of interoperability events for any telecommunication / Internet / Information Technology standard, and is highly respected within the industry world-wide.
Free ETSI White Papers on Interoperability and Testing
"Achieving Interoperability – The ETSI Approach" gives an overview of the approach that ETSI has taken to address inhibitors to interoperability and how the Institute ensures that interoperable standards of high quality and relevance to the marketplace are developed.
"Experiences of Using TTCN-3 for Automated Interoperability Testing" motivates the automation of interoperability testing for distributed systems and summarises a methodology for automated interoperability testing using the standardised testing language TTCN-3. This White Paper also shows the applicability of the approach to interoperability events and test beds based on practical experiences gained through the application of the methodology in real-life IMS Plugtests.
Presentation on Achieving Interoperable Standards
ETSI's Presentation on Achieving Interoperable Standards gives an overview of what means Interoperability to ETSI, why Interoperability is important and how the Institute ensures that interoperable standards of high quality and relevance to the marketplace are developed.