News

Public Review: Structured Data Transformation Language (SDTL)

The DDI Alliance is pleased to announce the Public Review of Structured Data Transformation Language (SDTL), a new product in the DDI suite of products that provides a key component of an automated metadata production process.
SDTL is an independent intermediate language for representing data transformation commands.  Statistical analysis packages (e.g., SPSS, Stata, SAS, and R) provide similar functionality, but each one has its own language and syntax.  SDTL consists of JSON schemas for common operations, such as RECODE, MERGE FILES, and VARIABLE LABELS.  SDTL provides machine-actionable descriptions of variable-level data transformation histories derived from any data transformation language.  Provenance metadata represented in SDTL can be added to documentation in Data Documentation Initiative (DDI) and other metadata standards.

SDTL greatly enhances the value of DDI; currently, DDI metadata is almost always created by data repositories but not by data producers.  Even when data are born digital, data producers discard provenance information that could be transported into DDI, because they perform data management and variable transformations in statistical packages that offer minimal metadata capabilities.  SDTL and the tools created by the C2Metadata Project are designed to create a metadata life cycle that parallels the research data life cycle. The same scripts that are used to transform and manage variables and data files can be used to update metadata files.  As a result, data producers can create more accurate and complete DDI metadata with less time and effort for them and for data repositories.

Links to the specification and instructions for comment are found at https://ddi-alliance.atlassian.net/wiki/spaces/DDI4/pages/1120370729/SDTL+Review.  We are eager to obtain feedback from a broad community on this specification. The comment period is open until August 31, 2020, and we hope to hear from you. Comments are requested on the content and scope of the:

  • SDTL Command Language
  • Function Library
  • Pseudo-Code Library
  • Documentation

DDI Alliance Virtual Annual Meetings

Dear DDI community,

You are invited to DDI Alliance virtual annual meetings next month.  As Steve McEachern, Chair of the DDI Executive Board, indicated last month, the in-person DDI Alliance annual meeting scheduled for IASSIST is not feasible this year.  In its place, we will host two separate, shorter virtual meetings.  This is an opportunity to learn about Alliance activities and provide feedback on priorities for the coming year.  We invite anyone who is interested in DDI activities to attend these virtual meetings.   
 
DDI Alliance Scientific Board Annual Meeting
May 18, 2020 (Monday) 13:00-15:00 UTC

Join Zoom Meeting: https://us02web.zoom.us/j/86002275516?pwd=eVV2dmJzQkhXZ1Rja2FRVUZ2UGFGQT09
Meeting ID: 860 0227 5516
Password: 499786

Find your local number: https://us02web.zoom.us/u/kcySGSOm9
 
 
The Scientific Board meeting will discuss several important new DDI technical and scientific developments, including:
DDI Alliance Annual Meeting of Members
May 26, 2020 (Tuesday) 13:00-15:00 UTC
 
Join Zoom Meeting: https://us02web.zoom.us/j/86033483859?pwd=THlKRytQZ3hnV0JDeWNiOVhpUlBqdz09
Meeting ID: 860 3348 3859
Password: 285310
Find your local number: https://us02web.zoom.us/u/kb05UPoDvg
 
 
The Meeting of Members will discuss the activities of the standing DDI committees and working groups, the budget, and planning for the upcoming fiscal year. 
 
Summary reports for each of the discussion items will be linked to the agendas the week before the respective meetings.  
 
We recognize that this is an unprecedented and unique situation and appreciate your flexibility.  Our thoughts are with everyone affected in the DDI community.  If you have any questions or concerns, please do not hesitate to contact me.  Looking forward to seeing many of you (virtually) next month!  
 
Sincerely,
Jared
 
 

Jared Lyle
Executive Director, DDI Alliance
ICPSR, University of Michigan
lyle@umich.edu

Community Input Requested: DDI-Codebook Enhancements

The DDI Technical Committee has had a number of requests for enhancements to DDI-Codebook, but is aware that there is probably far more content that could usefully be incorporated into a new version, especially for those with existing systems that have new demands, eg DataVerse & NADA.  In addition, DDI is in the process of describing both the overall coverage of DDI and the applications and coverage of individual products. YOUR HELP IS NEEDED!
 
The Technical Committee is looking for people interested in contributing to the short term goals of the next version of DDI-Codebook and/or the long-term development rules for DDI-Codebook so that it can remain relevant for the needs of our current and future users.
 
The Technical Committee has prepared a fuller description of what they are looking at both short and long-term
 
They would like users of DDI-Codebook to review the current set of issues (viewed from the link to DDI-Codebook version 2.6 from this page), then:
 
  • file any new issues (there is a link from the description page)
  • volunteer to spend a bit of time in the next few months to provide feedback on a new version of Codebook
If you are interested in spending some time over the next year exploring the future of DDI Codebook in terms of its structure, development rules, and requirements, please respond to Wendy Thomas, Chair of the Technical Committee (wlt@umn.edu).
 
Thanks for your input,
Wendy Thomas, for the DDI-Alliance Technical Committee

Public Release of DDI Lifecycle 3.3

The DDI Alliance is pleased to announce the public release of DDI Lifecycle 3.3. This updated version of DDI Lifecycle has moved through the development and review process and is now being published as the current version of DDI Lifecycle.

Version 3.3 contains additional coverage for:

  • Non-survey data collection (Measurements)
  • Sampling
  • Weighting
  • Questionnaire Design
  • Classification management (based on GSIM/Neuchatel)
  • Support for DDI as a Property Graph (properties on items and references)
  • Quality Statement improvements (useful for Eurostat reporting)

DDI Lifecycle 3.3 also introduces changes in the types and organization of documentation for the specification. These changes allow for the publication of updates and additions as they occur rather than requiring the reissuing of the specification itself. We continue to welcome comments on the coverage and format of this documentation.

Changes and updates in version 3.3 reflect work on 162 issues filed on version 3.2 which reflected specific bugs, continuity issues, and improving alignment with other standards. A full list of changes as well as instructions for those needing to translate instances or systems from version 3.2 to 3.3 is available: https://ddi-alliance.atlassian.net/wiki/spaces/DDI4/pages/805142559/DDI+3.3+Change+Log.

Thanks to everyone who contributed to DDI Lifecycle 3.3, especially the DDI Technical Committee, the Study Development Working Group, and the Copenhagen Profile.

Public Review: DDI - Cross Domain Integration (DDI-CDI)

The DDI Alliance is pleased to announce the Public Review of DDI - Cross Domain Integration (DDI-CDI), a specification aimed at helping implementers integrate data across domain and institutional boundaries.  The new DDI-CDI is an application of the model which emerged from many years of work on a "next generation" DDI specification, known as "DDI 4."

Since then, their family name has become an integral part of the tabloids, and Kim and her sisters have become regular Hollywood socialites.In addition to participating in the family reality show, Kimberly has also taken part in other television projects. So, Kim Kardashian playboy in 2008, her fans will remember her appearance Kardashian on the popular show "Dancing with the Stars", where Kim's partner on stage was a professional dancer Mark Ballas. A year earlier, in December 2007, "Playboy" magazine published a series of nude photos of Kardashian, and after a while Kim signed a contract with a famous clothing manufacturer and became the face of the company. Since then, Kardashian has been involved from time to time in fashion shows, as well as advertising campaigns for numerous fashion brands, including "Quick Trim", "Kotex" and "Sketchers"

DDI-CDI focuses on a uniform approach to describing a range of needed data formats (traditional wide/rectangular data, long [event] data, multi-dimensional data, and NoSQL/key-value data) which allows them to be connected and understood to support transformation and processing for integrated use. DDI-CDI is aligned with other DDI specifications (DDI-Codebook, DDI-Lifecycle) to support integration of external data in systems which use DDI. It is explicitly designed to work with many popular generic technology standards such as PROV-O, BPMN, DCAT, SDMX, DataCube, SSN/SOSA and Schema.org to allow for easy integration into systems which support them.

DDI-CDI offers an extension to the suite of DDI work products which will help those in the SBE domains and outside of them integrate the expanding range of data required by today's research. 

Links to the specification and instructions for comment are found at https://ddi-alliance.atlassian.net/wiki/x/IQBPMw. We are eager to obtain feedback from a broad community on this specification. The comment period is open until July 31, 2020, and we hope to hear from you. To assist you in your review we recommend the following approach: 

Read the Overview section of the DDI-CDI review page. This provides a guide to what you will find in the download package. There are four documents linked from the Overview section. Looking at these in the order presented will provide you with information on the purpose, organization, use, and structure of DDI-CDI. Depending on your areas of interest and expertise, you may be focusing on specific aspects of the package. These documents will help guide you to the relevant material. If you have questions about the package contents and how to review them, use the issue form or email option to ask for clarification. Find these options through the link in the upper right frame of the review page.

Your feedback to DDI-CDI is appreciated.

Update: The DDI Alliance is collaborating with CODATA on a series of webinars to support the public review of DDI-CDI.

DDI Alliance FY2021 funding requests

Dear DDI community,
 
Each year, the DDI Executive Board considers funding requests to support Alliance activities.  Funding requests typically come from DDI working groups and committees to support their activities.  
 
Any member may submit a funding request for the Executive Board to consider.  If you have a funding request for the next fiscal year (FY2021, July 2020 through June 2021), please email the request to secretariat@ddialliance.org by April 17, 2020.
 
The Alliance's membership dues are used to fund these requests.  The budget for the current 2020 fiscal year was finalized in June 2019 and is described in the Executive Board minutes: https://ddialliance.org/sites/default/files/20190625_Executive_Board.pdf.  The FY2019 financial report is part of the 2019 Annual Meeting minutes (see Appendix 3): https://ddialliance.org/sites/default/files/20190601_DDI_Alliance_Meeting_Minutes_0.pdf
 
Feel free to contact me with questions.
 
Sincerely,
Jared
 
 

Jared Lyle
Executive Director, DDI Alliance
ICPSR, University of Michigan
lyle@umich.edu

 

Welcome New Member: Qualiservice

The DDI Alliance welcomes Qualiservice (Noemi Betancort Cabrera, member and Scientific Board representative) as an Associate Member!  Qualiservice archives and provides qualitative social science research data from various disciplines for subsequent scientific use.  Prof. Dr. Betina Hollstein, Director of the SOCIUM Research Center on Inequality and Social Policy (Univ. Bremen), is the head of Qualiservice.

Cancellation of DDI in-person annual meeting, and virtual meeting planning

Dear DDI Member and Scientific Board Representatives,

In consultation with the DDI Executive Board, we have made a decision to cancel the in-person annual meeting scheduled for May 18, 2020 in Gothenburg, Sweden (the Monday of IASSIST). This decision was based on the most recent information about the COVID-19 pandemic and in an abundance of caution.

In place of the in-person meeting, we will hold a virtual meeting instead. We are currently liaising as to the timing and content of the meeting given the international spread of our membership. More information about the virtual meeting will be shared in the coming month.

We recognise that this is an unprecedented and unique situation and appreciate your flexibility.  Our thoughts are with everyone affected in the DDI community.  If you have any questions or concerns, please do not hesitate to contact the Alliance.

Regards,
Steve McEachern
Chair, on behalf of the DDI Executive Board

2020 North American DDI conference cancelled, to be replaced by virtual presentations

Given the severity of the COVID-19 pandemic, the 2020 North American Data Documentation Initiative Conference (NADDI) scheduled for June 17-19, 2020 in Minneapolis, Minnesota has been cancelled.  Instead, virtual presentations will be hosted in June 2020.  The virtual presentations will showcase how metadata standards, including and especially DDI, are making data Findable, Accessible, Interoperable, and Reusable (FAIR).  More information will be distributed in the coming months.

Special thanks to the 2020 hosts, IPUMS at the University of Minnesota Institute for Social Research and Data Innovation, for their support and hard work.

Sent on behalf of the NADDI 2020 Organizing Committee

Welcome New Member: Center for Socio-Political Data (CDSP), Sciences Po

The DDI Alliance welcomes Center for Socio-Political Data (CDSP), Sciences Po (Nicolas Sauger, member and Scientific Board representative) as a Full Member!