Skip to content

docs: add DataFusion security guidance - #26064

Open
efegokdemir wants to merge 4 commits into
apache:mainfrom
efegokdemir:docs/securing-datafusion
Open

efegokdemir wants to merge 4 commits into
apache:mainfrom
efegokdemir:docs/securing-datafusion

Conversation

@efegokdemir

@efegokdemir efegokdemir commented Oct 5, 2026 •

Copy link
Copy Markdown

Which issue does this PR close?

Rationale for this change

Applications embedding DataFusion may accept SQL from users, but the library guide does not explain security-relevant defaults or how available controls contribute to hardening.

What changes are included in this PR?

Adds a library guide covering SQL statement restrictions, opt-in local-file access, query memory limits, spill storage location and size limits, and the authorization and isolation responsibilities of the hosting application.

What is the testing strategy for this PR?

  • ./ci/scripts/doc_prettier_check.sh (passed after the documentation update).

Are there any user-facing changes?

Adds a “Securing DataFusion” page to the Library User Guide and links it from the guide and documentation navigation. The page now explains that memory limits do not cap spill disk use and names the runtime settings for controlling temporary storage.

AI assistance was used in preparing this contribution.

@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Oct 5, 2026
@alamb

alamb commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

@efegokdemir can you please confirm that you have reviewed the description and documentation of this PR?

The description seems like it may be unreviewed LLM output -- for example this line seems unrelated to this PR (and you can install it if you follow the README directions).

Sphinx HTML build not run: sphinx-build is not installed in this environment.

@alamb

alamb commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

We are working on better AI policies

@efegokdemir

Copy link
Copy Markdown
Author

I reviewed the PR description and the new documentation page. I removed the unrelated Sphinx installation note from the testing section and retained only the documentation check that was run. AI assistance is disclosed in the PR description.

@samueleresca

Copy link
Copy Markdown
Member

@efegokdemir The description of the PR not renders correctly after last update.

Comment thread docs/source/library-user-guide/securing-datafusion.md Outdated

@alamb alamb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @efegokdemir -- this is looking close, I left some suggestions

Comment thread docs/source/library-user-guide/securing-datafusion.md Outdated
Comment thread docs/source/library-user-guide/securing-datafusion.md Outdated
Comment thread docs/source/library-user-guide/securing-datafusion.md Outdated
Comment thread docs/source/library-user-guide/securing-datafusion.md Outdated
Comment thread docs/source/library-user-guide/securing-datafusion.md
Comment thread docs/source/library-user-guide/securing-datafusion.md Outdated
Comment thread docs/source/library-user-guide/securing-datafusion.md Outdated
Comment thread docs/source/library-user-guide/securing-datafusion.md Outdated
Comment thread docs/source/library-user-guide/securing-datafusion.md Outdated
Comment thread docs/source/library-user-guide/securing-datafusion.md Outdated
@efegokdemir

Copy link
Copy Markdown
Author

Updated in 8a146c79fc72ab4c7e693bb9888c68807c186375: the page now points to the project security policy, links DDL/DML examples and SessionContext::sql_with_options, uses the runtime settings guide for memory/spill controls, and removes repeated or vague guidance. ./ci/scripts/doc_prettier_check.sh --write --allow-dirty and git diff --check pass.

@alamb alamb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good -- thank you @efegokdemir

Signed-off-by: Efe Gökdemir <gokdemirefe1903@gmail.com>
@efegokdemir

Copy link
Copy Markdown
Author

Fixed the Sphinx reference warning in 6be87571: the guide now links directly to the repository security policy instead of treating the root SECURITY.md as a Sphinx source document. ./ci/scripts/doc_prettier_check.sh --write --allow-dirty and git diff --check pass. A local full docs build was unavailable here (cargo-depgraph and the Sphinx Python module are not installed). The new Docs workflow is currently action_required with zero jobs, so there is no post-fix remote Sphinx result yet.

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.74%. Comparing base (b40d696) to head (6be8757).
⚠️ Report is 49 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #26064      +/-   ##
==========================================
+ Coverage   82.69%   82.74%   +0.05%     
==========================================
  Files        1147     1147              
  Lines      447204   449767    +2563     
  Branches   447204   449767    +2563     
==========================================
+ Hits       369793   372154    +2361     
+ Misses      55001    54944      -57     
- Partials    22410    22669     +259     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a "securing DataFusion" documentation

4 participants