What's New

Webinars: Data Description with DDI-CDI

The Data Documentation Initiative Alliance recently announced the public review of the new specification, DDI – Cross Domain Integration (DDI-CDI). In order to introduce potential reviewers to the specification, a series of general introductory webinars have been held. Perhaps you already took part in one of them or would like to have a look at them? Please review the slides and recordings.

To follow up on this, we are now launching a second series of virtual events 1) for specific audiences, and 2) about specific topics.

We are writing to you to invite you to an event focusing on data description with DDI-CDI, which will be held on Wednesday 12th August at 14:00 UTC and we plan to host a follow-up workshop, a week later, to allow more detailed discussion on 19th August 2020 at 14:00 UTC. Attendees are asked to register using the following links:

Presentation and discussion, 12 August: https://register.gotowebinar.com/register/2796701692183077903

Discussion workshops, 19 August: https://register.gotowebinar.com/register/3437775245292851727

The webinar on data description focuses on:

  • understanding the meaning of data;
  • cross-domain exchange;
    • harmonization, including e.g. the same measurement represented in different data structures, and
    • different measurements are being compared;
  • transformation across structures and platforms

About DDI-CDI

DDI-CDI is a model-driven specification that is designed to provide support for the wide number of use cases that use data from multiple sources and domains. It originates from the  Social, Behavioral, and Economic (SBE) sciences, but is designed to be applicable to data coming from any domain. Integrating data across domain and disciplinary boundaries requires a flexible mechanism for describing disparate data sources, and their provenance and processing. DDI-CDI is designed to meet these emerging needs for the integration of data in old and new forms, coming from a variety of domains.

Cross Domain (Data) Integration is particularly important for grand challenge research areas that necessarily combine data of many types. CODATA sees great potential in DDI-CDI as a contribution to the International Science Council-endorsed Decadal Programme ‘Making Data Work for Cross-Domain Grand Challenges’. Consequently, CODATA through the Decadal Programme will partner with DDI in the public review process, assisting in getting feedback.

DDI-CDI aims to be technology agnostic and to be adaptable to any platform or representation. At the moment the DDI-CDI specification is provided as a formal UML model and as an XML syntax representation of that model: we are keen to adapt it to other technologies and representations, including RDF, JSON, Python etc.

Some resources:

To prepare for the event

At the event, data description using DDI-CDI will be introduced, followed by a structured discussion between the audience and people from the DDI-CDI team, with the event lasting for an hour. Please take a look at the draft presentation.

We would also kindly ask you to think about the following questions in advance of the event:

  • Are there data structures that you commonly use that we don’t seem to have covered?
    • What are they?
    • Do you see potential in DDI-CDI for needs that you currently haven’t met?
  • What have we missed?

We are looking forward to meeting you at the events!

Thanks,

Joachim Wackerow
Chair, DDI Alliance Scientific Board

DDI work products listed in FAIRsharing

Did you know that DDI work products are listed in FAIRsharing, a curated resource on data and metadata standards.  DDI work products cataloged in FAIRsharing include:

FAIRsharing is used by curators, librarians, developers, societies, funders, and researchers to explore what standards, databases, repositories, and data policies exist and are interrelated.

Feedback Requested: Scientific Board restructuring proposal

A temporary working group has been meeting since February to discuss how to improve the structure and organization of the Scientific Board, which is the scientific and technical body of the Alliance.  This past week, that group finalized their recommendations to propose a restructuring and to draft changes to the Alliance Bylaws.  Those recommendations are here: https://ddi-alliance.atlassian.net/wiki/spaces/DDI4/pages/850853895/Scientific+Board+Revision+-+temporary+working+group

The recommendations include:

  • ScientificBoard restructuring.docx.  This short note is intended to explain the thinking behind a new plan to organize, operate, and engage with the Scientific Board of the DDI Alliance.
  • Bylaws-2020-Final_Draft-20200619.docx.  This contains the proposed changes to the DDI Alliance Bylaws to accommodate the reorganized Scientific Board.
  • ScientificBoardOperationalGuidelines-draft.docx.  This specifies the operational guidelines of the reorganized Scientific Board.

As a next step, we ask that you review the recommendations and send any comments to Ingo Barkow <Ingo.Barkow@fhgr.ch>, chair of the temporary working group, by 17 July.  

Ingo and the temporary working group will finalize their recommendations and then send them to the Executive Board, which under the Bylaws can propose amendments.  "Amendments must be adopted by a two‐thirds majority vote of the Designated Member Representatives after written electronic notice of the vote of at least sixty days" (DDI Alliance Bylaws, (Section XVII).  

Public Review: Structured Data Transformation Language (SDTL)

The DDI Alliance is pleased to announce the Public Review of Structured Data Transformation Language (SDTL), a new product in the DDI suite of products that provides a key component of an automated metadata production process.
SDTL is an independent intermediate language for representing data transformation commands.  Statistical analysis packages (e.g., SPSS, Stata, SAS, and R) provide similar functionality, but each one has its own language and syntax.  SDTL consists of JSON schemas for common operations, such as RECODE, MERGE FILES, and VARIABLE LABELS.  SDTL provides machine-actionable descriptions of variable-level data transformation histories derived from any data transformation language.  Provenance metadata represented in SDTL can be added to documentation in Data Documentation Initiative (DDI) and other metadata standards.

SDTL greatly enhances the value of DDI; currently, DDI metadata is almost always created by data repositories but not by data producers.  Even when data are born digital, data producers discard provenance information that could be transported into DDI, because they perform data management and variable transformations in statistical packages that offer minimal metadata capabilities.  SDTL and the tools created by the C2Metadata Project are designed to create a metadata life cycle that parallels the research data life cycle. The same scripts that are used to transform and manage variables and data files can be used to update metadata files.  As a result, data producers can create more accurate and complete DDI metadata with less time and effort for them and for data repositories.

Links to the specification and instructions for comment are found at https://ddi-alliance.atlassian.net/wiki/spaces/DDI4/pages/1120370729/SDTL+Review.  We are eager to obtain feedback from a broad community on this specification. The comment period is open until August 31, 2020, and we hope to hear from you. Comments are requested on the content and scope of the:

  • SDTL Command Language
  • Function Library
  • Pseudo-Code Library
  • Documentation

DDI Alliance Virtual Annual Meetings

Dear DDI community,

You are invited to DDI Alliance virtual annual meetings next month.  As Steve McEachern, Chair of the DDI Executive Board, indicated last month, the in-person DDI Alliance annual meeting scheduled for IASSIST is not feasible this year.  In its place, we will host two separate, shorter virtual meetings.  This is an opportunity to learn about Alliance activities and provide feedback on priorities for the coming year.  We invite anyone who is interested in DDI activities to attend these virtual meetings.   
 
DDI Alliance Scientific Board Annual Meeting
May 18, 2020 (Monday) 13:00-15:00 UTC

Join Zoom Meeting: https://us02web.zoom.us/j/86002275516?pwd=eVV2dmJzQkhXZ1Rja2FRVUZ2UGFGQT09
Meeting ID: 860 0227 5516
Password: 499786

Find your local number: https://us02web.zoom.us/u/kcySGSOm9
 
 
The Scientific Board meeting will discuss several important new DDI technical and scientific developments, including:
DDI Alliance Annual Meeting of Members
May 26, 2020 (Tuesday) 13:00-15:00 UTC
 
Join Zoom Meeting: https://us02web.zoom.us/j/86033483859?pwd=THlKRytQZ3hnV0JDeWNiOVhpUlBqdz09
Meeting ID: 860 3348 3859
Password: 285310
Find your local number: https://us02web.zoom.us/u/kb05UPoDvg
 
 
The Meeting of Members will discuss the activities of the standing DDI committees and working groups, the budget, and planning for the upcoming fiscal year. 
 
Summary reports for each of the discussion items will be linked to the agendas the week before the respective meetings.  
 
We recognize that this is an unprecedented and unique situation and appreciate your flexibility.  Our thoughts are with everyone affected in the DDI community.  If you have any questions or concerns, please do not hesitate to contact me.  Looking forward to seeing many of you (virtually) next month!  
 
Sincerely,
Jared
 
 

Jared Lyle
Executive Director, DDI Alliance
ICPSR, University of Michigan
lyle@umich.edu

Community Input Requested: DDI-Codebook Enhancements

The DDI Technical Committee has had a number of requests for enhancements to DDI-Codebook, but is aware that there is probably far more content that could usefully be incorporated into a new version, especially for those with existing systems that have new demands, eg DataVerse & NADA.  In addition, DDI is in the process of describing both the overall coverage of DDI and the applications and coverage of individual products. YOUR HELP IS NEEDED!
 
The Technical Committee is looking for people interested in contributing to the short term goals of the next version of DDI-Codebook and/or the long-term development rules for DDI-Codebook so that it can remain relevant for the needs of our current and future users.
 
The Technical Committee has prepared a fuller description of what they are looking at both short and long-term
 
They would like users of DDI-Codebook to review the current set of issues (viewed from the link to DDI-Codebook version 2.6 from this page), then:
 
  • file any new issues (there is a link from the description page)
  • volunteer to spend a bit of time in the next few months to provide feedback on a new version of Codebook
If you are interested in spending some time over the next year exploring the future of DDI Codebook in terms of its structure, development rules, and requirements, please respond to Wendy Thomas, Chair of the Technical Committee (wlt@umn.edu).
 
Thanks for your input,
Wendy Thomas, for the DDI-Alliance Technical Committee

Public Release of DDI Lifecycle 3.3

The DDI Alliance is pleased to announce the public release of DDI Lifecycle 3.3. This updated version of DDI Lifecycle has moved through the development and review process and is now being published as the current version of DDI Lifecycle.

Version 3.3 contains additional coverage for:

  • Non-survey data collection (Measurements)
  • Sampling
  • Weighting
  • Questionnaire Design
  • Classification management (based on GSIM/Neuchatel)
  • Support for DDI as a Property Graph (properties on items and references)
  • Quality Statement improvements (useful for Eurostat reporting)

DDI Lifecycle 3.3 also introduces changes in the types and organization of documentation for the specification. These changes allow for the publication of updates and additions as they occur rather than requiring the reissuing of the specification itself. We continue to welcome comments on the coverage and format of this documentation.

Changes and updates in version 3.3 reflect work on 162 issues filed on version 3.2 which reflected specific bugs, continuity issues, and improving alignment with other standards. A full list of changes as well as instructions for those needing to translate instances or systems from version 3.2 to 3.3 is available: https://ddi-alliance.atlassian.net/wiki/spaces/DDI4/pages/805142559/DDI+3.3+Change+Log.

Thanks to everyone who contributed to DDI Lifecycle 3.3, especially the DDI Technical Committee, the Study Development Working Group, and the Copenhagen Profile.

Public Review: DDI - Cross Domain Integration (DDI-CDI)

The DDI Alliance is pleased to announce the Public Review of DDI - Cross Domain Integration (DDI-CDI), a specification aimed at helping implementers integrate data across domain and institutional boundaries.  The new DDI-CDI is an application of the model which emerged from many years of work on a "next generation" DDI specification, known as "DDI 4."

DDI-CDI focuses on a uniform approach to describing a range of needed data formats (traditional wide/rectangular data, long [event] data, multi-dimensional data, and NoSQL/key-value data) which allows them to be connected and understood to support transformation and processing for integrated use. DDI-CDI is aligned with other DDI specifications (DDI-Codebook, DDI-Lifecycle) to support integration of external data in systems which use DDI. It is explicitly designed to work with many popular generic technology standards such as PROV-O, BPMN, DCAT, SDMX, DataCube, SSN/SOSA and Schema.org to allow for easy integration into systems which support them.

DDI-CDI offers an extension to the suite of DDI work products which will help those in the SBE domains and outside of them integrate the expanding range of data required by today's research. 

Links to the specification and instructions for comment are found at https://ddi-alliance.atlassian.net/wiki/x/IQBPMw. We are eager to obtain feedback from a broad community on this specification. The comment period is open until July 31, 2020, and we hope to hear from you. To assist you in your review we recommend the following approach: 

Read the Overview section of the DDI-CDI review page. This provides a guide to what you will find in the download package. There are four documents linked from the Overview section. Looking at these in the order presented will provide you with information on the purpose, organization, use, and structure of DDI-CDI. Depending on your areas of interest and expertise, you may be focusing on specific aspects of the package. These documents will help guide you to the relevant material. If you have questions about the package contents and how to review them, use the issue form or email option to ask for clarification. Find these options through the link in the upper right frame of the review page.

Your feedback to DDI-CDI is appreciated.

Update: The DDI Alliance is collaborating with CODATA on a series of webinars to support the public review of DDI-CDI.

DDI Alliance FY2021 funding requests

Dear DDI community,
 
Each year, the DDI Executive Board considers funding requests to support Alliance activities.  Funding requests typically come from DDI working groups and committees to support their activities.  
 
Any member may submit a funding request for the Executive Board to consider.  If you have a funding request for the next fiscal year (FY2021, July 2020 through June 2021), please email the request to secretariat@ddialliance.org by April 17, 2020.
 
The Alliance's membership dues are used to fund these requests.  The budget for the current 2020 fiscal year was finalized in June 2019 and is described in the Executive Board minutes: https://ddialliance.org/sites/default/files/20190625_Executive_Board.pdf.  The FY2019 financial report is part of the 2019 Annual Meeting minutes (see Appendix 3): https://ddialliance.org/sites/default/files/20190601_DDI_Alliance_Meeting_Minutes_0.pdf
 
Feel free to contact me with questions.
 
Sincerely,
Jared
 
 

Jared Lyle
Executive Director, DDI Alliance
ICPSR, University of Michigan
lyle@umich.edu

 

Welcome New Member: Qualiservice

The DDI Alliance welcomes Qualiservice (Noemi Betancort Cabrera, member and Scientific Board representative) as an Associate Member!  Qualiservice archives and provides qualitative social science research data from various disciplines for subsequent scientific use.  Prof. Dr. Betina Hollstein, Director of the SOCIUM Research Center on Inequality and Social Policy (Univ. Bremen), is the head of Qualiservice.