Tag metadata

From “Data Quality for AI” to “AI for Data Quality Management”: Insights from BIR 2025

Anastasija Nikiforova Leave a comment

Back in September, our paper “From Data Quality for AI to AI for Data Quality: A Systematic Review of Tools for AI-Augmented Data Quality Management in Data Warehouses” was presented at the BIR Business Informatics Research Conference (BIR) in my hometown, Riga.

The paper, authored by my former student Heidi Carolina Tamm (now DW Lead Designer at Swedbank Estonia) and myself, explores a central question: can AI do more than just consume high-quality data—can it actively create and maintain it?

From “Data Quality for AI” to “Data Quality for AI & AI for Data Quality Management”

High-quality data underpins analytics, AI performance, and regulatory compliance, yet DQ management remains complex, resource-intensive, and often manual. With global data projected to reach 175 zettabytes by 2025, poor DQ carries real business costs—up to 19% of companies report customer loss due to inaccurate or incomplete data.

Traditionally, data quality (DQ) management focuses on ensuring AI has reliable, clean data. We propose a complementary perspective: using AI itself to enhance and automate data quality management. This is particularly relevant for enterprise data warehouses (DWs), which remain central despite the growth of decentralized and domain-driven architectures.

To this end, we systematically reviewed 151 DQ tools, evaluating their support for AI-augmented DQM, particularly automation of rule detection and anomaly identification. Key findings:

  • only 10 tools fully support AI-augmented DQM in data warehouses;
  • most tools focus on data cleansing rather than leveraging AI for rule discovery or explainable quality improvements;
  • metadata + rule-based + ML hybrid approaches show promise but remain underutilized;
  • SQL and natural language rule definitions are rarely supported, yet essential for practical use;
  • explainable AI and governance features are critical for trust;
  • cloud scalability with GDPR compliance is a must for enterprise adoption.

For organizations, AI-driven DQM goes beyond cleaning and profiling, enabling detection, enforcement, and explanation of data quality issues. This leads to better compliance, efficiency, and trust in enterprise data ecosystems.

For researchers and tool developers, there’s a clear need for next-generation DQ tools that are explainable, support SQL/natural language rules, integrate reconciliation logic, and operate under real-world regulatory and architectural constraints. LLMs and AI models can play a role, though they are not a silver bullet; early research shows strong potential.

Our study highlights that while AI has mainly been seen as a consumer of data quality, there is an untapped opportunity to use AI to actively improve DQ, shifting from “Data Quality for AI” to “Data Quality for AI and AI for Data Quality Management.” This research provides practical guidance for organizations and sets the stage for next-generation AI-augmented data governance and DQM solutions.

Read more in the article.

Data Quality as a prerequisite for sustainable success of business and not only: no data quality, no party!

Anastasija Nikiforova Leave a comment

In response to several requests, I finally completed the very first blog post on data quality management. In this post, I’ve mainly “set the scene” with the intention of publishing several more in-depth blog posts related to specific areas of my interest within data quality management, including but not limited to AI-augmented data quality management.

As for now, this post is focused primarily on my personal opinion (experience-based) what influences the choice of the data quality management approach (i.e., smth very similar to what I talked about at HackCodeX Forum I posted about earlier, and where the photo comes from).

View at Medium.com

Our – EOSC TF “FAIR Metrics and Data Quality” – whitepaper “Community-driven Governance of FAIRness Assessment: An Open Issue, an Open Discussion” is published by European Commission!

Anastasija Nikiforova Leave a comment

Proud to be part of the EOSC Task Force on FAIR Metrics and Data Quality and present our whitepaper “Community-driven Governance of FAIRness Assessment: An Open Issue, an Open Discussion” (Mark D. Wilkinson; Susanna-Assunta Sansone; Eva Méndez; Romain David; Richard Dennis; David Hecker; Mari Kleemola; Carlo Lacagnina; Anastasija Nikiforova; Leyla Jael Castro) published by European Commission, of course, in an open access, here.

“Although FAIR Research Data Principles are targeted at and implemented by different communities, research disciplines, and research stakeholders (data stewards, curators, etc.), there is no conclusive way to determine the level of FAIRness intended or required to make research artefacts (including, but not limited to, research data) Findable, Accessible, Interoperable, and Reusable.

The FAIR Principles cover all types of digital objects, metadata, and infrastructures. However, they focus their narrative on data features that support their reusability. FAIR defines principles, not standards, and therefore they do not propose a mechanism to achieve the behaviours they describe in an attempt to be technology/implementation neutral.

FAIR is evolving in some expected and some unexpected ways. FAIR “Reusability” sub-principle R1.3 states that “(meta)data should meet domain-relevant community standards,” which predicts a proliferation of FAIR interpretations by individual communities as they select their preferred approach to FAIRness. Similarly, as expected, there is an active movement around the adaptation of the FAIR Principles to digital objects other than data (e.g., software and workflows), again with individual communities interpreting what FAIRness means in these expanded contexts. However, there have also been attempts to expand the FAIR Principles themselves in recent years, including features of digital objects beyond reusability, including popularity (reuse/citation), reproducibility, reliability, data quality, etc. All of this is occurring with no overall coordination or planning.

A range of FAIR assessment metrics and tools have been designed that measure FAIRness. Unfortunately, the same digital objects assessed by different tools often exhibit widely different outcomes because of these independent interpretations of FAIR. This results in confusion among the publishers, the funders, and the users of digital research objects. Moreover, in the absence of a standard and transparent definition of what constitutes FAIR behaviours, there is a temptation to define existing approaches as being FAIR-compliant rather than having FAIR define the expected
behaviours. While it is anticipated that communities will define domain-specific FAIR metrics and tests, it is desirable to avoid “gaming the system” and have broadly agreed-upon approaches to FAIRness that do not favour a specific implementation of technology.

These observations suggest a growing need to align the different interpretations of the FAIR Principles. However, this whitepaper does not suggest that the FAIR Principles themselves require governance. Indeed, the document argues that the Principles should remain untouched. Specialised communities should extend/edit those Principles to adapt and make them more relevant to their community and their specific research outcome intended to be FAIR.

This whitepaper identifies three high-level stakeholder categories -FAIR decision and policymakers, FAIR custodians, and FAIR practitioners – and provides examples outlining specific stakeholders’ (hypothetical but anticipated) needs. It also examines possible models for governance based on the existing peer efforts, standardisation bodies, and other ways to acknowledge specifications and potential benefits. This whitepaper can serve as a starting point to foster an open discussion around FAIRness governance and the mechanism(s) that could be used to implement it, to be trusted, broadly representative, appropriately scoped, and sustainable”

Cite as: Mark D. Wilkinson, Susanna-Assunta Sansone, Eva Méndez, Romain David, Richard Dennis, David Hecker, Mari Kleemola, Carlo Lacagnina, Anastasija Nikiforova, & Leyla Jael Castro. (2022). Community-driven Governance of FAIRness Assessment: An Open Issue, an Open Discussion [version 1; peer review: awaiting peer review]. Open Res Europe 2022, 2:146 (https://doi.org/10.12688/openreseurope.15364.1)

Want to know more? Read more here!

A keynote for the Innovation and Smart Government Conference 2022 (ISGOV2022)

Anastasija Nikiforova Leave a comment

This September I was exceptionally proud to be invited to serve as one of two keynotes for the Innovation and Smart Government Conference 2022 (ISGOV2022) and open this conference, delivering my talk titled “Data as an asset for Sustainable Development of data-driven Smart Cities and Smart Society”. Another keynote was delivered by J. Ramon Gil-Garcia with the talk devoted to the use of Artificial Intelligence in the Public Sector.

ISGOV2022 keynotes

Some key takeaways or the main points of this talk (see slides above):
💡 Open data became a daily phenomenon, but making government data publicly available is not enough – there is time and need for the next steps towards sustainable and smart data ecosystems, requiring transformations at all levels;
💡 Open data is not only about OGD (B2G, crowdsourced data, sensor generated data etc.);
💡 Open data is not about data availability and accessibility. The data should –> must be qualitative (?), well-documented, valuabl (high value data (?)), smart (?), while the entry point from which they are available should be sufficiently user-friendly and interactive. Here, the concepts after which “?” follows is a subject for another 1.5h long talk :);
💡Open data quality is not only about metadata, their completeness and accuracy, it is also about the quality of content of dataset, where completeness of data is not the only criteria to be assessed! (reliability, internal and external consistency, timeliness, currency / up-to-date’ness and many more!);
💡{open, geospatial, smart, …} data ecosystem is not only about the platform (e.g., OGD portal), from which data can be downloaded, and not only about data governance or management. It is also not only about the data, which is, however, the key asset. It is many more…
💡Open data users – individuals, businesses (SME) etc., are not only end-users, they are (should be) an integral part of open data ecosystem. In other words, public data (eco)system is also about “public” –> stakeholders and actors of OD ecosystems should be identified (their role, needs etc.), which should then imply in requirements for newer and sustainable data ecosystems, which will facilitate interaction of (multiple) stakeholders
💡All in all, it’s all about data, portal, service, policy…
💡At the same time users and the whole society should be educated and acquire knowledge and skills needed to interact with the OD artifacts –> digital literacy, and open data literacy, where the latter term is not well-defined at this moment, while it is obvious that we need it!

Open Data as an asset for sustainable development of data-driven smart cities and smart society (isgovc2022)Download

I am very grateful ISGOV organizers, namely, the Autonomous University of Tamaulipas (UAT), the Autonomous University of the State of Mexico (UAEM), the University of Guadalajara (UDG), the I-Lab Public Innovation and Artificial Intelligence Laboratory – for having me as a part of this conference!!!

I am also grateful to all participants, especially those who participated in a discussion with so many relevant questions. Hope that this talk was interesting and contributed to the general understanding of the topic!

Editorial Board Member of eJournal of eDemocracy and Open Government (JeDEM)

Anastasija Nikiforova 1 Comment

Since June 2022, I am an Editorial Board Member of the eJournal of eDemocracy and Open Government (JeDEM) – a platinum/diamond scholarly-led Open Access e-journal managed by an interdisciplinary team of scholars at the Department for E-Governance and Administration at Danube University Krems, Austria. Its new Chief Editor – an external scholar in the journal’s key area, who is invited every 4 years to advice on the journal’s strategy – is Anneke Zuiderwijk with whom I was proud to collaborate very actively as part of my research visit to Delft University of Technology, Faculty Technology Policy and Management.

JeDEM is interested in both theoretical, practical and empirical research in the categories Research Papers, Invited Papers, Project Descriptions and Reflections. Within this scope, JeDEM particularly welcomes, but is not limited to, submissions related to the following topics:

  • e-Democracy
    • ICT and communication technologies to promote democracy or (re-)democratization;
    • Digital Divide, social inclusion and related political strategies;
    • Data Divide and algorithmic accountability;
    • policy analysis;
    • the role of security and privacy;
    • democratic innovation, governance models and alternative solutions.
  • e-Society and e-Participation
    • civic technologies and platforms (e.g. evaluation, critical and innovative approaches, national or international solutions);
    • collaborative decision-making and participatory budgeting;
    • the role of civil society and organizations;
    • stakeholder analysis, tool assessment and evaluation (e.g. political parties, government);
    • analysis of platform engagement (e.g. semantic analysis, computational or discourse analytical approaches);
    • co-decision, co-creation, co-production, decision-making and e-voting.
  • e-Government
    • general government services, evaluation of public policies (e.g. platforms for digital communication, virtual organizations and solutions, organizational training);
    • decision-making, Artificial Intelligence and automatization;
    • environmental, social and smart governance solutions;
    • governmental innovation.
  • Open data, including both social and technical aspects and the intersection between them 
    • open data policy, governance, decision-making and co-production;
    • technical frameworks for open data and metadata (e.g. ontologies, data formats, standards and APIs; data visualization; data quality);
    • evidence and impacts of open data: on society and/or public administration; value of real-life applications based on open data, costs and benefits of providing or using open data; emerging good practices; value generation (e.g. transparency, accountability, economic value, public service provision).
  • Data sharing and use, including but not limited to:
    • data with different levels of openness;
    • the role of public, private and societal stakeholders in data sharing and use, data end-users and intermediaries;
    • challenges and solutions for data sharing and use by various actors, including governments, researchers, companies, citizens, journalists, students, NGOs, librarians and intermediaries.
  • Open science, open access and open source software, including but not limited to:
    • best practices of open science;
    • benefits and challenges of scholarly publication, publishing data, information, articles and code through portals and platforms with different levels of openness;
    • safe and responsible sharing of data, information, articles and code with others
    • communication platforms to get more exposure and enhance usability of (open) data information, publications and code.

We encourage a diversity of methods and theoretical lenses, including critical studies in the above-mentioned thematic fields. It is the journal’s mission to encourage interdisciplinarity, unconventional ideas and multiple perspectives, and to connect leading thinkers and young scholars in inspiring reflections. JeDEM is an innovative journal that welcomes submissions from all disciplines and approaches. We publish both theoretical and empirical research, both qualitative and quantitative.

For the types of contribution, they are:

  • Research papers (double blind peer-review):
    • Regular submissions (submitted throughout the year, unrelated to a specific call for papers);
    • Special issue submissions (related to a call for papers);
    • Enhanced conference papers;
  • Reflections (reviews, comments, discussions, interviews, project descriptions) (editorial review).

JeDEM provides full open access to its authors and readers. Publishing with and reading JeDEM is free of charge. We ask authors to register with JeDEM to manage the publishing process. To gain all the benefits of the JeDEM community we recommend authors, readers, editors and reviewers to register their interest with JeDEM. JeDEM is a peer-reviewed, open-access journal (ISSN: 2075-9517). All journal content, except where otherwise noted, is licensed under the Creative Commons Attribution Licence.

Editorial JeDEM Vol. 14, No. 1 (2022)Download

Read more about JeDEM and consider submitting you contribution in it!