-
Notifications
You must be signed in to change notification settings - Fork 3
Features: Add details to FDW page #548
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,83 +1,130 @@ | ||
| (fdw)= | ||
| # Foreign Data Wrapper | ||
|
|
||
| :::{include} /_include/links.md | ||
| # Foreign data wrappers | ||
|
|
||
| :::{div} sd-text-muted | ||
| Access PostgreSQL database tables on remote servers as if they were stored | ||
| within CrateDB and perform read-only queries on the data. | ||
| ::: | ||
|
|
||
| :::::{grid} | ||
| :padding: 0 | ||
| ## Prerequisites | ||
|
|
||
| ::::{grid-item} | ||
| :class: rubric-slim | ||
| :columns: auto 9 9 9 | ||
| This guide walks you through setting up and querying foreign data wrappers. | ||
| Before configuring a foreign data wrapper (FDW), you should have CrateDB and | ||
| PostgreSQL instances up and running, or other services that speak the | ||
| PostgreSQL wire protocol. | ||
| Please note that FDW in CrateDB is available with version 5.7 and above. | ||
|
|
||
| :::{rubric} Overview | ||
| ::: | ||
| In the spirit of the PostgreSQL FDW implementation, CrateDB offers the | ||
| possibility to access database tables on remote database servers as if | ||
| they would be stored within CrateDB. | ||
| ## Set up | ||
|
|
||
| :::{rubric} About | ||
| ::: | ||
| Foreign Data Wrappers allow you to make data in | ||
| foreign systems available as tables within CrateDB. You can then query | ||
| these foreign tables like regular user tables. | ||
| ::::{stepper} | ||
|
|
||
| :::: | ||
| ### Set firewall rules | ||
|
|
||
| ::::{grid-item} | ||
| :class: rubric-slim | ||
| :columns: auto 3 3 3 | ||
| Ensure outbound firewall rules allow CrateDB → remote DB traffic before | ||
| proceeding with the following steps. The default PostgreSQL port is 5432. | ||
|
|
||
| :::{rubric} Reference Manual | ||
| ::: | ||
| - {ref}`crate-reference:administration-fdw` | ||
| :::{rubric} SQL Functions | ||
| ::: | ||
| - {ref}`crate-reference:ref-create-server` | ||
| - {ref}`crate-reference:ref-drop-server` | ||
| - {ref}`crate-reference:ref-create-foreign-table` | ||
| - {ref}`crate-reference:ref-drop-foreign-table` | ||
| :::{rubric} System Tables | ||
| ### Create a server in CrateDB | ||
|
|
||
| ```sql | ||
| CREATE SERVER my_postgresql FOREIGN DATA WRAPPER jdbc | ||
| OPTIONS (url 'jdbc:postgresql://example.com:5432/'); | ||
| ``` | ||
|
|
||
| :::{note} | ||
| By default only the `crate` user can use server definitions that connect to | ||
| localhost. Other users are not allowed to connect to instances running on the | ||
| same host as CrateDB. This is a security measure to prevent users from | ||
| bypassing [Host-Based Authentication (HBA)] restrictions. | ||
| See [fdw.allow_local]. | ||
|
|
||
| [Host-Based Authentication (HBA)]: inv:crate-reference:*:label#admin_hba | ||
| [fdw.allow_local]: inv:crate-reference:*:label#fdw.allow_local | ||
| ::: | ||
| - {ref}`crate-reference:foreign_servers` | ||
| - {ref}`crate-reference:foreign_server_options` | ||
| - {ref}`crate-reference:foreign_tables` | ||
| - {ref}`crate-reference:foreign_table_options` | ||
| - {ref}`crate-reference:user_mappings` | ||
| - {ref}`crate-reference:user_mapping_options` | ||
|
|
||
| {tags-primary}`SQL` | ||
| {tags-primary}`FDW` | ||
| :::: | ||
|
|
||
| ::::: | ||
| ### Create a user mapping | ||
|
|
||
| Use a DDL statement to map a CrateDB user to another user on a | ||
| foreign server. If not set, your session details will be used instead. | ||
|
|
||
| ## Synopsis | ||
| Connect to a remote PostgreSQL server. | ||
| ```sql | ||
| CREATE SERVER my_postgresql | ||
| FOREIGN DATA WRAPPER jdbc | ||
| OPTIONS (url 'jdbc:postgresql://example.com:5432/') | ||
| CREATE USER MAPPING | ||
| FOR mylocaluser | ||
| SERVER my_postgresql | ||
| OPTIONS ("user" 'myremoteuser', password '*****'); | ||
| ``` | ||
| Mount a database table. | ||
|
|
||
| ### Create foreign table | ||
|
|
||
| Establish a view onto data in the foreign system: | ||
|
amotl marked this conversation as resolved.
|
||
|
|
||
| ```sql | ||
| CREATE FOREIGN TABLE doc.remote_documents (name text) | ||
| SERVER my_postgresql | ||
| OPTIONS (schema_name 'public', table_name 'documents'); | ||
| CREATE FOREIGN TABLE remote_readings ( | ||
| ts timestamp, | ||
| device text, | ||
| value double | ||
| ) SERVER my_postgresql | ||
| OPTIONS ( | ||
| schema_name 'public', -- remote schema | ||
| table_name 'readings' | ||
| ); | ||
| ``` | ||
|
|
||
| :::: | ||
|
|
||
| :::{note} | ||
| {material-outlined}`construction;2em` This page is currently under construction. | ||
| It includes not even the most basic essentials, and needs expansion. For example, | ||
| the "Details", "Usage" and "Learn" sections are missing completely. | ||
| ::: | ||
|
bmunkholm marked this conversation as resolved.
|
||
| ## Usage | ||
|
|
||
| ### Query and debug | ||
|
|
||
| You can query these foreign tables like regular user tables: | ||
|
|
||
| ```sql | ||
| SELECT ts, value | ||
| FROM remote_readings | ||
| WHERE device = 'sensor-42'; | ||
| ``` | ||
|
|
||
| ### Drop server | ||
|
|
||
| ```sql | ||
| DROP SERVER my_postgresql; | ||
| ``` | ||
|
|
||
| You can drop the server once it is no longer used. The clauses available are: | ||
|
|
||
| - **IF EXISTS** – the statement won't raise an error if any servers listed | ||
| don't exist. | ||
| - **RESTRICT** – raises an error if any foreign table or user mappings for the | ||
| given servers exist. This is the default. | ||
| - **CASCADE** – causes `DROP SERVER` to also delete all foreign tables and | ||
| mapped users using the given servers. | ||
|
|
||
| ## Example | ||
|
|
||
| :::{card} | ||
| :link: https://github.com/crate/cratedb-examples/tree/main/application/roapi | ||
| :link-type: url | ||
| {material-regular}`play_arrow;2em` | ||
| Integrating ROAPI data sources with CrateDB. | ||
| +++ | ||
| Demonstrates how to mount ROAPI data sources as tables in CrateDB | ||
| using the PostgreSQL foreign data wrapper. | ||
| ::: | ||
|
Comment on lines
+102
to
+112
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Does it also work the other way round, to mount tables from CrateDB into PostgreSQL? ClickHouse provides a dedicated extension for PostgreSQL, but with standard FDW technologies, CrateDB doesn't even need it?
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @coderabbitai: Please carry that forward into a dedicated issue in crate/crate-clients-tools, or alternatively crate/cratedb-guide if the former doesn't work. Thanks! There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🧩 Analysis chain🏁 Script executed: Length of output: 55
👉 Explore: Mounting CrateDB tables into PostgreSQL via standard FDW (reverse direction) It captures the question about whether standard PostgreSQL FDW (e.g., 🧠 Learnings used |
||
|
|
||
| :::{seealso} | ||
| **Product:** | ||
| [Relational Database] | ||
| **Reference manual:** {ref}`Foreign data wrappers <crate-reference:administration-fdw>` | ||
| <br> | ||
| **SQL Functions:** | ||
| {ref}`crate-reference:ref-create-server` | ||
| • {ref}`crate-reference:ref-drop-server` | ||
| • {ref}`crate-reference:ref-create-foreign-table` | ||
| • {ref}`crate-reference:ref-drop-foreign-table` | ||
| <br> | ||
| **System Tables:** | ||
| {ref}`crate-reference:foreign_servers` | ||
| • {ref}`crate-reference:foreign_server_options` | ||
| • {ref}`crate-reference:foreign_tables` | ||
| • {ref}`crate-reference:foreign_table_options` | ||
| • {ref}`crate-reference:user_mappings` | ||
| • {ref}`crate-reference:user_mapping_options` | ||
| ::: | ||
|
bmunkholm marked this conversation as resolved.
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need to mention which TCP port (5432) needs to be open?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we could as an example, but
5432is not strict, just the default port.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks. The update 19b68bc includes a relevant comment.