1. About
  2. News

News

DDI-CDI Version 1.0 Vote Results: Approved

Submitted by lyle on Mon, 2024-11-25 10:27
Dear DDI Community,

The voting period for Version 1.0 of DDI Cross-Domain Integration (DDI-CDI) concluded on Friday, November 22nd.  The vote passed with 19 of 26 designated member representatives from full members participating (73% turnout).  All votes cast supported the statement: “YES, the validity and usefulness of Version 1.0 of DDI-CDI has been demonstrated and it should now be accepted as part of the DDI standard.”

About DDI-CDI
DDI-CDI extends traditional DDI metadata to describe data beyond the social, behavioral, and economic (SBE) domains, addressing the need for broader capabilities. It supports descriptions of event and sensor data ("long" data), key-value data (often associated with "big" data and no-SQL data), and multi-dimensional data. By integrating these with traditional "wide" (or "rectangular") DDI data descriptions, DDI-CDI enables the management and production of integrated data sets from diverse sources. Additionally, DDI-CDI facilitates the documentation of generic processes, allowing for clear descriptions of the steps involved in data integration. To ensure compatibility with varying technologies and standards used across domains, DDI-CDI is designed to interoperate with a wide range of web standards, supporting both XML or RDF-based syntax representations.  For a Webinar explaining DDI-CDI, please see the YouTube recording: https://youtu.be/mITmwqcbais. Slides for the presentation: https://doi.org/10.5281/zenodo.13961159.
 
Next Steps
The Technical Committee will prepare the specification for publication.  A separate announcement will be shared with this community and beyond once the Version 1.0 DDI-CDI specification has been formally published.
 
Acknowledgments
Special thanks go to Arofan Gregory (chair) and the DDI-CDI Working Group for developing and shepherding the specification, as well as the Technical Committee for their efforts in preparing for this vote. DDI-CDI work represents contributions from across the DDI community. From the DDI-CDI contributors page: "DDI-CDI evolved from the earlier DDI Moving Forward (DDI 4) effort. That effort involved multiple teams and in-person meetings (Sprints), both in Europe and North America. The first Moving Forward sprint took place at Schloss Dagstuhl, the Leibniz Center for Informatics, in October 2012....In time, the effort became extremely broad. The development of DDI Moving Forward (DDI 4) involved 23 multi-day working events since 2012 (total of 109 days), 14 working groups, 115 participants coming from 65 organizations."  DDI Alliance membership funds also played a significant role in supporting this development.
 
Thank you for your continued support and contributions in advancing the work of DDI.

Sincerely,

Jared
 
 

Jared Lyle
Executive Director, DDI Alliance

DDI Lifecycle 4.0 BETA2 Review

Submitted by lyle on Tue, 2024-11-05 14:03

The Technical Committee is happy to announce the availability of the second beta version of DDI Lifecycle 4.0.

https://github.com/ddialliance/ddimodel/releases/tag/v4.0-beta.2

We continue to look for feedback on the community's preference between the different OWL serializations and general comments about the XML and JSON. Users can provide questions and comments to the TC using github issues with a free github account at:

https://github.com/ddialliance/ddimodel/issues

Comments on this Beta 2.0 are requested by December 5th.

DDI Lifecycle 4.0 is managed using a Model First development approach. Content additions and changes are added to the model and the serialization formats are automatically generated. The metadata content of DDI 3.3 has been converted into the new 4.0 model. Issues raised during the first beta version review have been addressed.

The ddi-lifecycle-4.0-beta.2.zip package includes:

  • schema.xsd - XML Schema
  • jsonSchema.json - JSON Schema
  • ddi4.xmi - OMG's UML XMI 2.5
  • ddi4.owl - RDF OWL (closed with restrictions)
  • ddi4.owl.ttl - RDF OWL (open)
  • ddi4.shex - ShEx
  • ddi4.shacl- SHACL

Other formats are also available but are not included in the official release. The beta 2 package is available at: https://github.com/ddialliance/ddimodel/releases

The documentation for DDI Lifecycle 4.0 is also auto-generated in real time from the model, and can be viewed at: https://docs.ddialliance.org/DDI-Lifecycle/dev/model/

What's Changed

  • Add Ordered attribute to properties which were implicitly ordered in the model

  • Check cardinalities for choices from DDI to cogs conversion

  • Upload development version builds to docs.ddialliance.org automatically

  • Missing 3.3 elements with extension base Reference (In Person TC meeting activity)

Full Changelog: v4.0-beta.1...v4.0-beta.2
Issues Resolved: https://github.com/ddialliance/ddimodel/issues?q=is%3Aissue+closed%3A%3E2023-12-21

Webinar: The Impact of FAIR and AI on Standards in Official Statistics

Submitted by lyle on Wed, 2024-10-09 15:35

This webinar, held on 26 September 2024, looked at the changing data landscape, and how it is and will continue to impact the development of standards in the world of official statistics. The advent of large language models and associated AI approaches, combined with the increasing demand for and use of a wider range of data for both research and policy purposes — the driver behind the FAIR data principles — alters the way standards are used to support data systems in official statistics. This webinar looked at these drivers for change, and the recent and upcoming developments around DDI, SDMX, and a range of other standards from WorldFAIR, W3C, and the UN/ECE Modernisation of Statistics community. Further, it suggested some further avenues of exploration and work.

  1. Introduction (Arofan Gregory, Chair, DDI-CDI Working Group)
  2. Trends in the Data Landscape (Mercè Crosas, Head of Computational Social Sciences, Barcelona Supercomputing Centre and President, CODATA). 
  3. Standards and Approaches (Arofan Gregory, Chair, DDI-CDI Working Group). 
  4. Interoperability with SDMX, DDI and other OS standards (Flavio Rizzolo, Senior Data Architect, Statistics Canada). 
  5. Example: UN SDG Indicators Example: DDI-CDI from SDMX Services for Google Data Commons (Luis Gerardo Gonzalez Morales, Chief of the Data Innovation Section. UN Statistics Division). 

The recording and slides are now available: https://codata.org/the-impact-of-fair-and-ai-on-standards-in-official-statistics-webinar-thursday-26-september/

DDI Community Newsletter, Fall 2024

Submitted by lyle on Thu, 2024-10-03 00:26

The latest edition of the DDI community newsletter is now available:

https://mailchi.mp/umich/ddi-community-newsletter-fall-2024

Registration is open for the European DDI Conference 2024

Submitted by lyle on Mon, 2024-09-30 16:54

We are pleased to announce that registration is now open for the 16th European DDI Users Conference, to be held at the heart of the Alps in Chur, Switzerland from 2-6 December 2024. The conference is being generously hosted by the University of Applied Sciences of the Grisons.

Details of how to register are at: https://events.geant.org/event/1637/page/17-registration

There will be 2 workshops/tutorials on the Monday followed by ~40 presentations, several posters and tow birds-of-a-feather discsussions on the Tuesday/Wednesday from a wide range of organisations and individuals on all things DDI, and related issues.

Side meetings will take place on Monday and Thursday.

The DDI Alliance Developers Group Hackathon will be on Thursday and Friday (separate registration needed).

The draft program is available at: https://events.geant.org/event/1637/timetable/#20241202.detailed

Details of getting to the venue and accommodation are available at: https://events.geant.org/event/1637/page/14-travel-accommodation

We look forward to welcoming you to Chur!

Mari Kleemola & Jon Johnson

EDDI 2023 co-chairs

 

Publication of XKOS Best Practice

Submitted by lyle on Wed, 2024-09-25 14:23

The XKOS Working Group is excited to share the new XKOS Best Practices guide. XKOS, which is one of the RDF products of the DDI Alliance, is an extension of the SKOS standard, which organizes and describes concepts and their relationships. XKOS adds features specifically for managing statistical classifications and nomenclatures. It offers detailed ways to show relationships within a hierarchy, such as broader or narrower categories. XKOS also adds specific notes typical of nomenclatures: notes to define the core content of a classification item, notes to specify additional content, or notes that refer to a ruling concerning the classification of certain statistical objects/units. Finally, XKOS helps create clear links between different classification systems, making it easier to work with multiple data sets.

This XKOS Best Practices guide (a ReSpec document) provides examples of how to use the XKOS vocabulary. The expected uses of RDF properties are specified using SHACL constraints. You can see examples of the implementation of our nomenclatures in XKOS via INSEE's SPARQL EndPoint. Here are the links:

Christophe Dzikowski & Éric Sigaud
XKOS Working Group Co-Chairs

DDI-CDI Vote + Informational Webinar

Submitted by lyle on Tue, 2024-09-24 23:28

Dear DDI Alliance community,

 

Today, the Technical Committee received Version 1.0 of DDI Cross-Domain Integration (DDI-CDI) from the DDI-CDI working group. Over the coming weeks, the Technical Committee will review the package for completeness and prepare it for a formal vote on publication. Once preparations are complete, Designated Member Representatives from DDI Alliance Full Members will receive a link to cast their votes. It is important to vote. 

An initial public review was conducted from June to December 2022, with the issue tracker remaining open for comments through 2023. During this time, the working group actively addressed all submitted issues.  The next candidate review was submitted to the Technical Committee in April 2024, but no full public review was requested, as changes were limited to minor refinements and documentation updates. An additional technical review of the production process took place from May 1 to August 16, 2024. All issues have been addressed and the product is ready for a final vote on publication.

To assist you in evaluating this product for inclusion in the DDI suite, we invite you to attend a one-hour webinar, which will cover key DDI-CDI features, its relationship with DDI-Codebook and DDI-Lifecycle, and early implementation cases within the DDI and FAIR communities.  

Webinar Details:

After registering, you will receive a confirmation email with information on how to join the webinar.

For additional background information on DDI-CDI, please visit: https://ddialliance.org/Specification/ddi-cdi.

DDI-CDI is a new product for DDI and we are asking you to vote on its inclusion in the product line of the DDI Alliance. DDI-CDI extends traditional DDI metadata to describe data beyond the social, behavioral, and economic (SBE) domains, addressing the need for broader capabilities. It supports descriptions of event and sensor data ("long" data), key-value data (often associated with "big" data and no-SQL data), and multi-dimensional data. By integrating these with traditional "wide" (or "rectangular") DDI data descriptions, DDI-CDI enables the management and production of integrated data sets from diverse sources. As cross-domain and multi-disciplinary research grows, this capability becomes increasingly vital. 

Additionally, DDI-CDI facilitates the documentation of generic processes, allowing for clear descriptions of the steps involved in data integration. To ensure compatibility with varying technologies and standards used across domains, DDI-CDI is designed to interoperate with a wide range of web standards, supporting both XML or RDF-based syntax representations. 

We would like to thank Arofan Gregory, the DDI-CDI Working Group, and the Technical Committee for their dedicated efforts in preparing the DDI-CDI product for publication.

 

Jared Lyle
Executive Director, DDI Alliance
ICPSR, University of Michigan
lyle@umich.edu

DDI Developers Hackathon III - Registration and Submission of Challenges now open

Submitted by lyle on Fri, 2024-09-13 17:48

After the successful Hackathons in Gothenburg and Cologne the DDI Developers Group is organizing its third Hackathon at the University of Applied Sciences in Chur (Switzerland) from Thursday December 5th till Friday December 6th 2024 in the rooms surrounding and within the Campus Brewery of this location (Building D, Comercialstrasse 20, Chur). This event is directly following the European DDI Users Conference (EDDI) in the same week. Therefore, we believe some participants could already be at the location saving travel costs. Hotel information can also be found on the EDDI webpage.

https://www.eddi-conferences.eu/

 

Please note that EDDI and the DDI Developers Hackathon are two seperate events though they take place at the same location after each other. You must register for both events if you want to participate.

 

If you are a developer, software engineer or programmer using or implementing tools around the DDI suite of metadata standards this event might be the chance to exchange ideas with similar people plus during the two days we would like to create some prototypical software implementations of current pain points or needed features for DDI tools.

 

If you are engaged in DDI related work (e.g. in working groups or any of the boards) you are welcome to submit challenges (i.e. implementation ideas) for the participating hackers, even if you do not participate in person.

 

The event is sponsored by the DDI Alliance and the University of Applied Sciences of the Grisons (FHGR) which will provide catering and the location for the whole event. For a limited number of people also sponsorships for travel costs can be provided if the member organizations cannot sponsor it.

 

Please register here for the Hackathon
 

https://docs.google.com/forms/d/1QI7KSc8J15D_-UCkXCX8eQJcFOICvnh3qgm6pzdOCps/viewform?pli=1&pli=1

 

Please submit ideas for topics (so called challenges) for the participating hackers here

 

https://github.com/ddi-developers/

 

During the event there will be a Hackathon Bar which provides snacks and drinks for the participating hackers at all times. Additionally there will be lunch and dinner served at fitting times during the implementation phases so hackers can fully concentrate on their tasks. The event will also start with a welcome dinner on Wednesday evening and end with a thank you dinner on Friday evening. As a special the Campus Brewery will also provide craft beers especially brewed for this event. All catering will be free of cost.

 

The results from the Hackathon will be available as Open Source software under a license (e.g. MIT, LGPL) which will allow the use in other Open Source projects as well as commercial products.

 

We hope to re-create some lasting interest for more IT-related topics within the DDI community. Therefore we are happy for every developer, software engineer, programmer or DevOps person who can make it to Chur. Of course we are also happy about any new member who would like to join our steadily growing DDI Developer group.
 

Best Regards
 
The DDI Developers Group Chairs
 
Ingo Barkow

Johan Fihn Marberg

Oliver Hopt

Olof Olsson

Call for Expression of Interest: Dagstuhl DDI-CDI Workshop Oct 7-11, 2024 ‘Aligning Technology Architectures with Cross-Domain Metadata Models’

Submitted by lyle on Mon, 2024-08-12 13:33
To Whom It May Interest:
 
We are organising a workshop at Schloss Dagstuhl on October 7 - 11, 2024, in Wadern, Germany with a focus on DDI-CDI (https://ddialliance.org/Specification/DDI-CDI/) and its implementation. The title of the workshop is ‘Aligning Technology Architectures with Cross-Domain Metadata Models’ (https://www.dagstuhl.de/24413). While the workshop is strictly limited for space, we would welcome participation from interested individuals. The topics under consideration include:
 
- Syntax Representation of the DDI-CDI Model
- Modularity of the DDI-CDI Model and W3C-DDI Collaboration
- DDI-CDI Tools Development and Integration
- Describing Non-Quantitative Data
 
 
The workshop will include the members of the current DDI-CDI Working Group, but will also include outside experts. If you are implementing DDI-CDI or wish to learn more about it by being involved in a working group on the proposed topics, and want to attend the workshop, please send an expression of interest along with any questions to Arofan Gregory, chair of the DDI-CDI Working Group (ilg21@yahoo.com). We would appreciate a short statement of your background in working with metadata standards and data systems, and of your interest in DDI-CDI. We will prioritize those applicants who make a strong case for their contribution to the listed topics and indicate an interest in ongoing participation in the DDI-CDI WG.
 
Please contact us by 23 August, with the above information if you are interested in participating.
 
Some general information about the costs and the venue are given below. 
 
Practical Information
 
The workshop will be an in-person event, lasting for the entire week. Total numbers will be 25 people.  Schloss Dagstuhl is an immersive environment, and it is requested that participants be available for the entire week.
 
Travel and Accommodation Costs 
 
Accommodation and subsistence costs at Dagstuhl are subsidized and very reasonably priced (420 € for the week).  It will be the responsibility of applicants to fund their participation.
 
 
The Organisers: Arofan Gregory, Simon Hodson, Steve McEachern, Hilde Orten, Joachim Wackerow, and Michelle Edwards.

UNECE workshop on standards and implementations, 21-22 October 2024 in Geneva, Switzerland

Submitted by lyle on Wed, 2024-07-17 12:40
The UNECE is organizing a ModernStats World Workshop on 21-22 October 2024 in Geneva, Switzerland. The workshop will explore how the work of national and international statistical offices can be streamlined and modernized using conceptual models describing data/metadata, architectures, business processes and activities. The programme of the workshop includes: improving interoperability using standards, enhancing transparency with standards, tools for automating metadata-driven processes, and the future of statistical production in 2025 and beyond.

https://unece.org/statistics/events/MWW2024

 
The workshop is a forum discussing implementation issues and ideas in four content areas and providing input to the UNECE group for future planning. Many DDI community members will attend and present, but more participants would bring new perspectives and enhance the discussion. This could also inform the work of the Alliance and offer insights from major statistical agencies on using implementation standards, especially DDI. Integrating DDI (specifically, Lifecycle, CDI, and XKOS) with SDMX to create interoperable pipelines across the GSBPM is a current R&D focus for NSOs and will be the topic of a side meeting after the workshop.