Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 0 additions & 19 deletions docs/_docs/SQL/JDBC/jdbc-driver.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,6 @@ For the list of security parameters, refer to the <<Using SSL>> section.
- `jdbc:ignite:thin://myHost:11900;user=ignite;password=ignite` - connect to myHost on custom port 11900 with user credentials for authentication.
- `jdbc:ignite:thin://myHost:11900;distributedJoins=true&autoCloseServerCursor=true` - connect to myHost on custom port 11900 with enabled distributed joins and autoCloseServerCursor optimization.
- `jdbc:ignite:thin://myHost:11900/myschema;` - connect to myHost on custom port 11900 and access to MYSCHEMA.
- `jdbc:ignite:thin://myHost:11900/"MySchema";lazy=false` - connect to myHost on custom port 11900 with disabled lazy query execution and access to MySchema (schema name is case sensitive).

=== Multiple Endpoints

Expand Down Expand Up @@ -676,21 +675,3 @@ The table below lists all the link:https://en.wikipedia.org/wiki/SQLSTATE[ANSI S
|50000| Internal error.
The code is not defined by ANSI and refers to an Ignite specific error. Refer to the `java.sql.SQLException` error message for more information.
|=======================================================================


















41 changes: 4 additions & 37 deletions docs/_docs/SQL/sql-tuning.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -52,43 +52,10 @@ Avoid having too many columns in the result set of a `SELECT` query. Due to limi

== Lazy Loading

By default, Ignite attempts to load the whole result set to memory and send it back to the query initiator (which is usually your application).
This approach provides optimal performance for queries of small or medium result sets.
However, if the result set is too big to fit in the available memory, it can lead to prolonged GC pauses and even `OutOfMemoryError` exceptions.

To minimize memory consumption, at the cost of a moderate performance hit, you can load and process the result sets lazily by passing the `lazy` parameter to the JDBC and ODBC connection strings or use a similar method available for Java, .NET, and C++ APIs:

[tabs]
--

tab:Java[]
[source,java]
----
SqlFieldsQuery query = new SqlFieldsQuery("SELECT * FROM Person WHERE id > 10");

// Result set will be loaded lazily.
query.setLazy(true);
----
tab:JDBC[]
[source,sql]
----
jdbc:ignite:thin://192.168.0.15?lazy=true
----
tab:C#/.NET[]
[source,csharp]
----
var query = new SqlFieldsQuery("SELECT * FROM Person WHERE id > 10")
{
// Result set will be loaded lazily.
Lazy = true
};
----
tab:C++[]
--

////
*TODO* Add tabs for ODBC and other programming languages - C# and C++
////
Ignite always operates in lazy loading mode. Query results are not loaded entirely into memory — data is streamed to
the client in pages as the result set is iterated. The page size determines how many rows are fetched per network round
trip, allowing you to balance memory consumption and performance and eliminating the risk of prolonged GC pauses
and `OutOfMemoryError` exceptions for large result sets.

== Querying Colocated Data

Expand Down
4 changes: 1 addition & 3 deletions docs/_docs/binary-client-protocol/sql-and-scan-queries.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -267,7 +267,6 @@ UPDATE = 2
|bool| Replicated only - Whether query contains only replicated tables or not.
|bool| Enforce join order.
|bool| Collocated - Whether your data is co-located or not.
|bool| Lazy query execution.
|long| Timeout (milliseconds).
|bool| Include field names.
|===
Expand Down Expand Up @@ -355,7 +354,7 @@ out.writeBoolean(false);
out.writeBoolean(false);
// Lazy
out.writeBoolean(false);
out.writeBoolean(true);
// Timeout
writeLongLittleEndian(5000, out);
Expand Down Expand Up @@ -631,4 +630,3 @@ readResponseHeader(in);
----

--

43 changes: 4 additions & 39 deletions docs/_docs/perf-and-troubleshooting/sql-tuning.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -78,45 +78,10 @@ with 100+ columns may perform worse than expected.

== Lazy Loading

By default, Ignite will set Lazy Loading enabled, this will minimize memory consumption at the cost of moderate performance degradation.

Otherwise, Ignite attempts to load the whole result set to memory and send it back to the query initiator (which is usually your application). This approach provides optimal performance for queries of small or medium result sets, and minimizes the duration of internal database locks, thus increasing concurrency.

WARNING, if the result set is too big to fit in the available memory, it can lead to prolonged GC pauses and even `OutOfMemoryError` exceptions. This property is deprecated since Ignite 2.15.0.

To change the default behavior:

[tabs]
--

tab:Java[]
[source,java]
----
SqlFieldsQuery query = new SqlFieldsQuery("SELECT * FROM Person WHERE id > 10");

// Result set will be loaded eagerly.
query.setLazy(false);
----
tab:JDBC[]
[source,sql]
----
jdbc:ignite:thin://192.168.0.15?lazy=false
----
tab:C#/.NET[]
[source,csharp]
----
var query = new SqlFieldsQuery("SELECT * FROM Person WHERE id > 10")
{
// Result set will be loaded eagerly.
Lazy = false
};
----
tab:C++[]
--

////
*TODO* Add tabs for ODBC and other programming languages - C# and C++
////
Ignite always operates in lazy loading mode. Query results are not loaded entirely into memory — data is streamed to
the client in pages as the result set is iterated. The page size determines how many rows are fetched per network round
trip, allowing you to balance memory consumption and performance and eliminating the risk of prolonged GC pauses
and `OutOfMemoryError` exceptions for large result sets.

== Querying Colocated Data

Expand Down
2 changes: 0 additions & 2 deletions docs/_docs/thin-clients/python-thin-client.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -422,7 +422,6 @@ The `sql()` method supports a number of parameters that
| `replicated_only` |
| `enforce_join_order` |
| `collocated` |
| `lazy` |
| `include_field_names` |
| `max_rows` |
| `timeout` |
Expand Down Expand Up @@ -485,4 +484,3 @@ If you still want to use authentication without securing the connection, simply
----
include::{sourceFileDir}/auth.py[tag=no-ssl,indent=0]
----

Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ public QueryRegistryImpl(GridKernalContext ctx) {
RunningQueryManager qryMgr = kctx.query().runningQueryManager();

long locId = qryMgr.register(rootQry.sql(), GridCacheQueryType.SQL_FIELDS, rootQry.context().schemaName(),
false, createCancelToken(qry), rootQry.initiatorId(), false, true, false);
false, createCancelToken(qry), rootQry.initiatorId(), false, false);

rootQry.localQueryId(locId);

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@

import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.List;
Expand All @@ -27,7 +28,6 @@
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.function.Consumer;
import java.util.regex.Pattern;

import org.apache.ignite.Ignite;
import org.apache.ignite.Ignition;
import org.apache.ignite.cache.query.FieldsQueryCursor;
Expand Down Expand Up @@ -227,10 +227,16 @@ public void testJdbcV2InitiatorId() throws Exception {
final UUID grid0NodeId = grid(0).cluster().localNode().id();

GridTestUtils.runAsync(() -> {
try (Connection conn = DriverManager.getConnection(CFG_URL_PREFIX + "lazy=false:nodeId="
try (Connection conn = DriverManager.getConnection(CFG_URL_PREFIX + "nodeId="
+ grid0NodeId + "@modules/clients/src/test/config/jdbc-security-config.xml")) {
try (Statement stmt = conn.createStatement()) {
stmt.execute(sql);

ResultSet rs = stmt.getResultSet();

while (rs != null && rs.next()) {
// No-op.
}
}
}
catch (SQLException e) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,6 @@
import org.junit.Test;

import static org.apache.ignite.IgniteJdbcDriver.CFG_URL_PREFIX;
import static org.apache.ignite.cache.query.SqlFieldsQuery.DFLT_LAZY;

/**
* Connection test.
Expand Down Expand Up @@ -270,7 +269,6 @@ public void testSqlHints() throws Exception {
assertTrue(((JdbcConnection)conn).isEnforceJoinOrder());
assertFalse(((JdbcConnection)conn).isDistributedJoins());
assertFalse(((JdbcConnection)conn).isCollocatedQuery());
assertEquals(DFLT_LAZY, ((JdbcConnection)conn).isLazy());
assertFalse(((JdbcConnection)conn).skipReducerOnUpdate());
}

Expand All @@ -279,7 +277,6 @@ public void testSqlHints() throws Exception {
assertFalse(((JdbcConnection)conn).isEnforceJoinOrder());
assertTrue(((JdbcConnection)conn).isDistributedJoins());
assertFalse(((JdbcConnection)conn).isCollocatedQuery());
assertEquals(DFLT_LAZY, ((JdbcConnection)conn).isLazy());
assertFalse(((JdbcConnection)conn).skipReducerOnUpdate());
}

Expand All @@ -288,23 +285,14 @@ public void testSqlHints() throws Exception {
assertFalse(((JdbcConnection)conn).isEnforceJoinOrder());
assertFalse(((JdbcConnection)conn).isDistributedJoins());
assertTrue(((JdbcConnection)conn).isCollocatedQuery());
assertEquals(DFLT_LAZY, ((JdbcConnection)conn).isLazy());
assertFalse(((JdbcConnection)conn).skipReducerOnUpdate());
}

try (final Connection conn = DriverManager.getConnection(CFG_URL_PREFIX + "lazy=" + (!DFLT_LAZY) + "@" + configURL())) {
assertFalse(((JdbcConnection)conn).isEnforceJoinOrder());
assertFalse(((JdbcConnection)conn).isDistributedJoins());
assertFalse(((JdbcConnection)conn).isCollocatedQuery());
assertEquals(!DFLT_LAZY, ((JdbcConnection)conn).isLazy());
assertFalse(((JdbcConnection)conn).skipReducerOnUpdate());
}
try (final Connection conn = DriverManager.getConnection(CFG_URL_PREFIX + "skipReducerOnUpdate=true@"
+ configURL())) {
assertFalse(((JdbcConnection)conn).isEnforceJoinOrder());
assertFalse(((JdbcConnection)conn).isDistributedJoins());
assertFalse(((JdbcConnection)conn).isCollocatedQuery());
assertEquals(DFLT_LAZY, ((JdbcConnection)conn).isLazy());
assertTrue(((JdbcConnection)conn).skipReducerOnUpdate());
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,6 @@
import static java.sql.ResultSet.TYPE_FORWARD_ONLY;
import static java.sql.Statement.NO_GENERATED_KEYS;
import static java.sql.Statement.RETURN_GENERATED_KEYS;
import static org.apache.ignite.cache.query.SqlFieldsQuery.DFLT_LAZY;
import static org.apache.ignite.testframework.GridTestUtils.assertThrows;
import static org.apache.ignite.testframework.GridTestUtils.assertThrowsAnyCause;
import static org.apache.ignite.testframework.GridTestUtils.getFieldValue;
Expand Down Expand Up @@ -376,44 +375,32 @@ public void testPartitionAwarenessPartitionDistributionsCacheSizePropertySemicol
@Test
public void testSqlHints() throws Exception {
try (Connection conn = DriverManager.getConnection(urlWithPartitionAwarenessProp)) {
assertHints(conn, false, false, false, false, DFLT_LAZY,
false, partitionAwareness);
assertHints(conn, false, false, false, false, false, partitionAwareness);
}

try (Connection conn = DriverManager.getConnection(urlWithPartitionAwarenessProp + "&distributedJoins=true")) {
assertHints(conn, true, false, false, false, DFLT_LAZY,
false, partitionAwareness);
assertHints(conn, true, false, false, false, false, partitionAwareness);
}

try (Connection conn = DriverManager.getConnection(urlWithPartitionAwarenessProp + "&enforceJoinOrder=true")) {
assertHints(conn, false, true, false, false, DFLT_LAZY,
false, partitionAwareness);
assertHints(conn, false, true, false, false, false, partitionAwareness);
}

try (Connection conn = DriverManager.getConnection(urlWithPartitionAwarenessProp + "&collocated=true")) {
assertHints(conn, false, false, true, false, DFLT_LAZY,
false, partitionAwareness);
assertHints(conn, false, false, true, false, false, partitionAwareness);
}

try (Connection conn = DriverManager.getConnection(urlWithPartitionAwarenessProp + "&replicatedOnly=true")) {
assertHints(conn, false, false, false, true, DFLT_LAZY,
false, partitionAwareness);
}

try (Connection conn = DriverManager.getConnection(urlWithPartitionAwarenessProp + "&lazy=" + (!DFLT_LAZY))) {
assertHints(conn, false, false, false, false, !DFLT_LAZY,
false, partitionAwareness);
assertHints(conn, false, false, false, true, false, partitionAwareness);
}

try (Connection conn = DriverManager.getConnection(urlWithPartitionAwarenessProp + "&skipReducerOnUpdate=true")) {
assertHints(conn, false, false, false, false, DFLT_LAZY,
true, partitionAwareness);
assertHints(conn, false, false, false, false, true, partitionAwareness);
}

try (Connection conn = DriverManager.getConnection(urlWithPartitionAwarenessProp + "&distributedJoins=true&" +
"enforceJoinOrder=true&collocated=true&replicatedOnly=true&lazy=" + (!DFLT_LAZY) + "&skipReducerOnUpdate=true")) {
assertHints(conn, true, true, true, true, !DFLT_LAZY,
true, partitionAwareness);
"enforceJoinOrder=true&collocated=true&replicatedOnly=true&skipReducerOnUpdate=true")) {
assertHints(conn, true, true, true, true, true, partitionAwareness);
}
}

Expand All @@ -425,39 +412,32 @@ public void testSqlHints() throws Exception {
@Test
public void testSqlHintsSemicolon() throws Exception {
try (Connection conn = DriverManager.getConnection(urlWithPartitionAwarenessPropSemicolon + ";distributedJoins=true")) {
assertHints(conn, true, false, false, false, DFLT_LAZY,
false, partitionAwareness);
assertHints(conn, true, false, false, false, false, partitionAwareness);
}

try (Connection conn = DriverManager.getConnection(urlWithPartitionAwarenessPropSemicolon + ";enforceJoinOrder=true")) {
assertHints(conn, false, true, false, false, DFLT_LAZY,
false, partitionAwareness);
assertHints(conn, false, true, false, false, false, partitionAwareness);
}

try (Connection conn = DriverManager.getConnection(urlWithPartitionAwarenessPropSemicolon + ";collocated=true")) {
assertHints(conn, false, false, true, false, DFLT_LAZY,
false, partitionAwareness);
assertHints(conn, false, false, true, false, false, partitionAwareness);
}

try (Connection conn = DriverManager.getConnection(urlWithPartitionAwarenessPropSemicolon + ";replicatedOnly=true")) {
assertHints(conn, false, false, false, true, DFLT_LAZY,
false, partitionAwareness);
assertHints(conn, false, false, false, true, false, partitionAwareness);
}

try (Connection conn = DriverManager.getConnection(urlWithPartitionAwarenessPropSemicolon + ";lazy=" + (!DFLT_LAZY))) {
assertHints(conn, false, false, false, false, !DFLT_LAZY,
false, partitionAwareness);
try (Connection conn = DriverManager.getConnection(urlWithPartitionAwarenessPropSemicolon)) {
assertHints(conn, false, false, false, false, false, partitionAwareness);
}

try (Connection conn = DriverManager.getConnection(urlWithPartitionAwarenessPropSemicolon + ";skipReducerOnUpdate=true")) {
assertHints(conn, false, false, false, false, DFLT_LAZY,
true, partitionAwareness);
assertHints(conn, false, false, false, false, true, partitionAwareness);
}

try (Connection conn = DriverManager.getConnection(urlWithPartitionAwarenessPropSemicolon + ";distributedJoins=true;" +
"enforceJoinOrder=true;collocated=true;replicatedOnly=true;lazy=" + (!DFLT_LAZY) + ";skipReducerOnUpdate=true")) {
assertHints(conn, true, true, true, true, !DFLT_LAZY,
true, partitionAwareness);
"enforceJoinOrder=true;collocated=true;replicatedOnly=true;skipReducerOnUpdate=true")) {
assertHints(conn, true, true, true, true, true, partitionAwareness);
}
}

Expand All @@ -469,18 +449,18 @@ public void testSqlHintsSemicolon() throws Exception {
* @param enforceJoinOrder Enforce join order.
* @param collocated Co-located.
* @param replicatedOnly Replicated only.
* @param lazy Lazy.
* @param skipReducerOnUpdate Skip reducer on update.
* @throws Exception If failed.
*/
private void assertHints(Connection conn, boolean distributedJoins, boolean enforceJoinOrder, boolean collocated,
boolean replicatedOnly, boolean lazy, boolean skipReducerOnUpdate, boolean partitionAwarenessEnabled)throws Exception {
private void assertHints(
Connection conn, boolean distributedJoins, boolean enforceJoinOrder, boolean collocated,
boolean replicatedOnly, boolean skipReducerOnUpdate, boolean partitionAwarenessEnabled
) throws Exception {
for (JdbcThinTcpIo io: ios(conn)) {
assertEquals(distributedJoins, io.connectionProperties().isDistributedJoins());
assertEquals(enforceJoinOrder, io.connectionProperties().isEnforceJoinOrder());
assertEquals(collocated, io.connectionProperties().isCollocated());
assertEquals(replicatedOnly, io.connectionProperties().isReplicatedOnly());
assertEquals(lazy, io.connectionProperties().isLazy());
assertEquals(skipReducerOnUpdate, io.connectionProperties().isSkipReducerOnUpdate());
assertEquals(partitionAwarenessEnabled, io.connectionProperties().isPartitionAwareness());
}
Expand Down
Loading
Loading