diff --git a/docs/source/index.rst b/docs/source/index.rst index ea6ebb74c08b1..c994ae65249e6 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -137,6 +137,7 @@ To get started, see :caption: Library User Guide library-user-guide/index + library-user-guide/securing-datafusion library-user-guide/upgrading/index library-user-guide/extensions library-user-guide/using-the-sql-api diff --git a/docs/source/library-user-guide/index.md b/docs/source/library-user-guide/index.md index fd126a1120edf..0dffc279ab06a 100644 --- a/docs/source/library-user-guide/index.md +++ b/docs/source/library-user-guide/index.md @@ -29,6 +29,9 @@ for details on how to contribute to DataFusion. If you haven't reviewed the [architecture section in the docs][docs], it's a useful place to get the lay of the land before starting down a specific path. +For guidance on running DataFusion with SQL from untrusted users, see +[Securing DataFusion](securing-datafusion.md). + DataFusion is designed to be extensible at all points, including - [x] User Defined Functions (UDFs) diff --git a/docs/source/library-user-guide/securing-datafusion.md b/docs/source/library-user-guide/securing-datafusion.md new file mode 100644 index 0000000000000..968310a3e2598 --- /dev/null +++ b/docs/source/library-user-guide/securing-datafusion.md @@ -0,0 +1,70 @@ + + +# Securing DataFusion + +As described in the [DataFusion security policy](https://github.com/apache/datafusion/blob/main/SECURITY.md), +the end application is responsible for security decisions. The settings below +can help control what a query can do. + +## Restrict SQL statements + +[`SQLOptions`] allows an application to reject classes of SQL statements when +creating a `DataFrame`. DDL (such as [`CREATE TABLE`](../user-guide/sql/ddl.md)) +and DML (such as [`INSERT`](../user-guide/sql/dml.md)) are allowed by default. +Disable the classes that the application does not need and pass the options to +[`SessionContext::sql_with_options`] for every user-provided query: + +```rust +use datafusion::prelude::*; + +let options = SQLOptions::new() + .with_allow_ddl(false) + .with_allow_dml(false) + .with_allow_statements(false); + +let dataframe = ctx.sql_with_options(sql, options).await?; +``` + +## Limit file access + +[`SessionContext::enable_url_table()`] is an opt-in feature that lets SQL query +local files by path. Leave it disabled when users should only query tables +registered by the application. If it is needed, run DataFusion with filesystem +permissions limited to the files the application intends to expose. + +## Set query memory limits + +Set an appropriate `datafusion.runtime.memory_limit` for the workload using the +[runtime configuration settings](../user-guide/configs.md#runtime-configuration-settings). +This limits memory used by DataFusion's query execution memory pool; use +process- or container-level resource limits as well when a hard bound on total +application memory is required. + +## Bound spill storage + +When an execution operator supports spilling, DataFusion may write intermediate +query data to temporary files under memory pressure. Use +`datafusion.runtime.temp_directory` and +`datafusion.runtime.max_temp_directory_size` to configure the location and size +of that temporary storage; see the [runtime configuration settings](../user-guide/configs.md#runtime-configuration-settings). + +[sqloptions]: https://docs.rs/datafusion/latest/datafusion/execution/context/struct.SQLOptions.html +[sessioncontext::sql_with_options]: https://docs.rs/datafusion/latest/datafusion/execution/context/struct.SessionContext.html#method.sql_with_options +[sessioncontext::enable_url_table()]: https://docs.rs/datafusion/latest/datafusion/execution/context/struct.SessionContext.html#method.enable_url_table