Skip to content

Add Data Structures & Organization pillar (inspector tab + readiness report) #162

Description

@biqar

Is your feature request related to a problem? If so, please describe.

Yes. AIDRIN’s current metrics focus on value quality (completeness, outliers), feature relationships (correlation), fairness, and governance, but there is no dedicated place to assess whether a dataset is structurally ready for AI workflows — i.e. easy to parse, navigate, and link.

Example workflow: I upload a wide Excel export or denormalized CSV. Data Quality may look acceptable, but columns use inconsistent types, there is no clear row identifier, the file format is inefficient for pipelines, and load time is poor. I have no single inspector view or readiness scorecard that answers “is this data organized well enough for modeling and integration?”

Describe the solution you'd like

Add a new Data Structures & Organization capability in two places:

  1. Inspector menu — a new sidebar tab/pillar where users can explore structural signals interactively (similar to Data Quality or Impact on AI).
  2. Readiness Report — an automated, scored section derived from the same engines, with KPIs, needs-attention items, and PDF export alignment.

Organize the feature around four themes (from our readiness framework):

  • Data types — appropriateness and consistency of column types
  • Schema organization — row/column identity, redundancy, naming, and linkability (table-level proxies on flat uploads)
  • File format & storage — suitability and efficiency of the uploaded representation
  • Access performance — ingest-time load speed and reliability

The solution should reuse existing signals where possible (e.g. duplicate rows, redundant columns) and add new structural checks where gaps exist.

Describe alternatives you've considered

  • Extend Dataset Overview only — adds context but keeps structural assessment ungraded and easy to miss.
  • Fold into Data Quality — mixes “clean values” with “well-organized structure,” which are different readiness questions.
  • Readiness report only (no new tab) — improves automation but leaves no interactive deep-dive path for practitioners.
  • Defer until multi-table / DB connectors exist — more complete for relational schema, but delays useful flat-file signals users need today.

Additional Information

N/A

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions