The Data Documentation Initiative (DDI) Metadata Specification

The Data Documentation Initiative (DDI) is an international program to produce a metadata specification for the description of social science data resources. The program was initiated in 1994 by the Inter-University Consortium for Political and Social Research (ICPSR). Contributors to the efforts of the DDI come from social science data archives and libraries in USA, Canada and Europe and from major producers of statistical data (like the US Bureau of the Census, the US Bureau of Labour statistics, Statistics Canada and Health Canada). Until recently the work has been carried out by a committee based on voluntary participation. A reorganization to a more strongly knit consortium model will be put in place by the end of 2002. The original aim of the DDI committee was to replace the existing and widely used OSIRIS Codebook/data dictionary standard with a more modern and Web-aware specification that could be used to structure the description of the content of social science data archives. The first preliminary version of the DDI specification came in the form of an SGML DTD, which in 1997 it was converted into an XML DTD. i The migration to XML happened just a few months after the W3C released the very first working draft of the XML specification. The DDI was consequently one of the very first major metadata initiatives using the new framework. The first official version of the DDI specification (version 1.0) was published in March 2000. Publication followed extensive beta-testing in a variety of environments as well as software implementation by major projects like Nesstar.ii Since the first public release,several extensions have been made to the specification, most noteworthy the inclusion of a sub-model for description of aggregated data structured as multidimensional tables (cubes). Several other extensions, like more precise descriptions of data coming out of CATI/CAPI-like environments are on the drawing table. Plans do also exists for a DDI II process that will revisit the entire DDI-specification from a modeling perspective and possibly produce alternative syntactical representations in frameworks like RDF Scheme and/or XML-Schema. The uptake of the specification in the data archive community has been quite remarkable. Several major European data archives have already migrated their entire holdings to the DDI and an effort to create a harmonized and integrated interface to all the national social science data archives in Europe based on the new standard has been initiated. In North America, the worlds biggest data archive, ICPSR, as well as several minor data centers and libraries, are in the process of moving to the DDI. The acceptance of the DDI is also growing outside the data archive community. Health Canada is, as an example, making their new data dissemination tool webDAIS compatible with the DDI specification.