The Committee on National Statistics under the U.S. Academy of Sciences, Engineering, and Medicine released a report on November 18, 2021 from the Panel on Transparency and Reproducibility in Federal Statistics called Transparency in Statistical Information for the National Center for Science and Engineering Statistics and All Federal Statistical Agencies (Report). You can download a copy for free by following the link. The panel included several people familiar to the DDI community.
The focus of the report is on transparency of federal statistics, where transparency is defined in the report as follows:
Transparency is the provision of sufficiently detailed documentation of all the processes of producing official estimates. The goal of transparency is to enable consumers of federal statistics to accurately understand and evaluate how estimates are generated.
The emphasis on documentation, understanding, and evaluation should be familiar to all who use the Data Documentation Initiative (DDI) metadata standards. This emphasis is about metadata.
The Report is divided into 7 chapters and 2 substantive appendices, with Chapter 5 devoted to metadata, systems, and standards. Several of the standards described are DDI-Codebook, DDI-Lifecycle, and the soon to be released DDI-CDI (Cross Domain Integration). SDTL, XKOS, and Controlled Vocabularies are briefly described, too. Descriptions of other standards, such as SDMX, DCAT, PROV, GSIM, and GSBPM are included and divided between Chapter 5 and Appendix A. In Appendix B, the case for how transparency can be achieved through the use of and conformance to standards is presented.
Every chapter deals with a particular aspect of transparency, and for many in the DDI community, Chapters 2 and 3 will be interesting, as archiving is discussed. Each Chapter includes a short set of recommendations for the federal statistical agencies, and the ones listed here include text about the need among the agencies to adopt standards and manage metadata. Excerpts from each follow:
3.1 "The metadata that accompany such data should also be preserved using broadly accepted metadata standards appropriate to the data at hand."
5.1 "The Interagency Council on Statistical Policy (ICSP) should develop and implement a multi-agency pilot project to explore and evaluate employing existing metadata standards and tools to accomplish data sharing, data access, and data reuse."
5.2 "The ICSP should: (1) prioritize and emphasize the importance and benefits of federal statistical agency staff engaging in international metadata standards and tool development, and (2) organize a discussion among statistical agencies that leads to an effective, coordinated, and accountable approach for staff in agencies that produce federal statistics to contribute to international metadata standards and tool development."
The Report provides an opportunity for the DDI Alliance to position itself as the source of many of the metadata standards needed by the federal statistical agencies to achieve transparency. The expertise that many representatives of member organizations in the Alliance can bring to discussions is vital to furthering the adoption of DDI. Communication among the Alliance, the ICSP, and the agencies should begin soon.