Skip to content

Classic Batch Data Loader fails when loading a Druid Datasource with quantilesDoublesSketch #19036

New issue
New issue
Open
Open
Classic Batch Data Loader fails when loading a Druid Datasource with quantilesDoublesSketch#19036
Labels
Uncategorized problem report
@JRobTS

Description

@JRobTS
JRobTS
opened on Feb 19, 2026

The Unified Console classic-batch-data-loader fails when reindexing data from existing Druid segments that contain a quantilesDoublesSketch fails with error: Failed to sample data: Unhandled type: COMPLEX

Affected Version

33.0.0

Description

In the Connect step of the classic-batch-data-loader, with source type druid, select a Druid datasource that contains a quantilesDoublesSketch and an appropriate interval. Upon clicking Apply we see: Failed to sample data: Unhandled type: COMPLEX.

The coordinator log contains a stack trace:

2026-02-19T17:24:31,503 INFO [qtp615774072-197] org.apache.druid.segment.loading.LocalDataSegmentPuller - Copied 791578758 bytes from [/data/druid/deep-storage/segments/datasource/2025-01-01T00:00:00.000Z_2025-01-02T00:00:00.000Z/2025-01-02T00:15:10.922Z/0/index] to [/data/druid/apache-druid-33.0.0/var/tmp/druid11891466104252998799/datasource/2025-01-01T00:00:00.000Z_2025-01-02T00:00:00.000Z/2025-01-02T00:15:10.922Z/0]
2026-02-19T17:24:31,510 WARN [qtp615774072-197] org.apache.druid.indexing.input.DruidSegmentInputEntity - Could not clean temporary segment file: var/tmp/druid11891466104252998799/datasource/2025-01-01T00:00:00.000Z_2025-01-02T00:00:00.000Z/2025-01-02T00:15:10.922Z/0
2026-02-19T17:21:17,014 ERROR [qtp615774072-197] org.apache.druid.indexing.overlord.sampler.SamplerExceptionMapper - Failed to sample data: Unhandled type: COMPLEX
org.apache.druid.indexing.overlord.sampler.SamplerException: Failed to sample data: Unhandled type: COMPLEX
        at org.apache.druid.indexing.overlord.sampler.InputSourceSampler.sample(InputSourceSampler.java:290) ~[druid-indexing-service-33.0.0.jar:33.0.0]
        at org.apache.druid.indexing.overlord.sampler.IndexTaskSamplerSpec.sample(IndexTaskSamplerSpec.java:90) ~[druid-indexing-service-33.0.0.jar:33.0.0]
        at org.apache.druid.indexing.overlord.sampler.SamplerResource.post(SamplerResource.java:84) ~[druid-indexing-service-33.0.0.jar:33.0.0]
        at jdk.internal.reflect.GeneratedMethodAccessor718.invoke(Unknown Source) ~[?:?]
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
        at java.base/java.lang.reflect.Method.invoke(Method.java:569) ~[?:?]
        at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) ~[jersey-server-1.19.4.jar:1.19.4]
        at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$TypeOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:185) ~[jersey-server-1.19.4.jar:1.19.4]
        at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) ~[jersey-server-1.19.4.jar:1.19.4]
        at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:302) ~[jersey-server-1.19.4.jar:1.19.4]
        at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) ~[jersey-server-1.19.4.jar:1.19.4]
        at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) ~[jersey-server-1.19.4.jar:1.19.4]
        at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) ~[jersey-server-1.19.4.jar:1.19.4]
        at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1542) ~[jersey-server-1.19.4.jar:1.19.4]
        at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1473) ~[jersey-server-1.19.4.jar:1.19.4]
        at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1419) ~[jersey-server-1.19.4.jar:1.19.4]
        at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1409) ~[jersey-server-1.19.4.jar:1.19.4]
        at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409) ~[jersey-servlet-1.19.4.jar:1.19.4]
        at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558) ~[jersey-servlet-1.19.4.jar:1.19.4]
        at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733) ~[jersey-servlet-1.19.4.jar:1.19.4]
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:790) ~[javax.servlet-api-3.1.0.jar:3.1.0]
        at com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:290) ~[guice-servlet-5.1.0.jar:?]
        at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:280) ~[guice-servlet-5.1.0.jar:?]
        at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:184) ~[guice-servlet-5.1.0.jar:?]
        at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:89) ~[guice-servlet-5.1.0.jar:?]
        at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85) ~[guice-servlet-5.1.0.jar:?]
        at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:121) ~[guice-servlet-5.1.0.jar:?]
        at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:133) ~[guice-servlet-5.1.0.jar:?]
        at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) ~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626) ~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.apache.druid.server.http.RedirectFilter.doFilter(RedirectFilter.java:73) ~[druid-server-33.0.0.jar:33.0.0]
        at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) ~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626) ~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.apache.druid.server.security.PreResponseAuthorizationCheckFilter.doFilter(PreResponseAuthorizationCheckFilter.java:84) ~[druid-server-33.0.0.jar:33.0.0]
        at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) ~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626) ~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.apache.druid.server.initialization.jetty.StandardResponseHeaderFilterHolder$StandardResponseHeaderFilter.doFilter(StandardResponseHeaderFilterHolder.java:164) ~[druid-server-33.0.0.jar:33.0.0]
        at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) ~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626) ~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.apache.druid.server.security.AllowHttpMethodsResourceFilter.doFilter(AllowHttpMethodsResourceFilter.java:78) ~[druid-server-33.0.0.jar:33.0.0]
        at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) ~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626) ~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.apache.druid.server.security.AllowOptionsResourceFilter.doFilter(AllowOptionsResourceFilter.java:74) ~[druid-server-33.0.0.jar:33.0.0]
        at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) ~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626) ~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.apache.druid.server.security.AllowAllAuthenticator$1.doFilter(AllowAllAuthenticator.java:84) ~[druid-server-33.0.0.jar:33.0.0]
        at org.apache.druid.server.security.AuthenticationWrappingFilter.doFilter(AuthenticationWrappingFilter.java:59) ~[druid-server-33.0.0.jar:33.0.0]
        at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) ~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626) ~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.apache.druid.server.security.SecuritySanityCheckFilter.doFilter(SecuritySanityCheckFilter.java:77) ~[druid-server-33.0.0.jar:33.0.0]
        at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193) ~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626) ~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:552) ~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) ~[jetty-server-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1624) ~[jetty-server-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233) ~[jetty-server-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440) ~[jetty-server-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188) ~[jetty-server-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505) ~[jetty-servlet-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1594) ~[jetty-server-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186) ~[jetty-server-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355) ~[jetty-server-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) ~[jetty-server-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:772) ~[jetty-server-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:59) ~[jetty-server-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127) ~[jetty-server-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.server.Server.handle(Server.java:516) ~[jetty-server-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487) ~[jetty-server-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732) [jetty-server-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479) [jetty-server-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277) [jetty-server-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) [jetty-io-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) [jetty-io-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) [jetty-io-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) [jetty-util-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) [jetty-util-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) [jetty-util-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) [jetty-util-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) [jetty-util-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) [jetty-util-9.4.57.v20241219.jar:9.4.57.v20241219]
        at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) [jetty-util-9.4.57.v20241219.jar:9.4.57.v20241219]
        at java.base/java.lang.Thread.run(Thread.java:840) [?:?]
Caused by: org.apache.druid.java.util.common.IAE: Unhandled type: COMPLEX
        at org.apache.druid.segment.AutoTypeColumnIndexer$FieldIndexer.processValue(AutoTypeColumnIndexer.java:745) ~[druid-processing-33.0.0.jar:33.0.0]
        at org.apache.druid.segment.AutoTypeColumnIndexer$1.processField(AutoTypeColumnIndexer.java:105) ~[druid-processing-33.0.0.jar:33.0.0]
        at org.apache.druid.segment.nested.StructuredDataProcessor.processFields(StructuredDataProcessor.java:73) ~[druid-processing-33.0.0.jar:33.0.0]
        at org.apache.druid.segment.AutoTypeColumnIndexer.processAuto(AutoTypeColumnIndexer.java:213) ~[druid-processing-33.0.0.jar:33.0.0]
        at org.apache.druid.segment.AutoTypeColumnIndexer.processRowValsToUnsortedEncodedKeyComponent(AutoTypeColumnIndexer.java:160) ~[druid-processing-33.0.0.jar:33.0.0]
        at org.apache.druid.segment.incremental.IncrementalIndex.toIncrementalIndexRow(IncrementalIndex.java:560) ~[druid-processing-33.0.0.jar:33.0.0]
        at org.apache.druid.segment.incremental.IncrementalIndex.add(IncrementalIndex.java:493) ~[druid-processing-33.0.0.jar:33.0.0]
        at org.apache.druid.indexing.overlord.sampler.InputSourceSampler.sample(InputSourceSampler.java:174) ~[druid-indexing-service-33.0.0.jar:33.0.0]
        ... 81 more

When describing the datasource in question, there is only one complex data type: COMPLEX<quantilesDoublesSketch>

It seems to be just the sampler that fails. By manually submitting a job with an appropriately crafted spec (including an aggregation of the quantilesDoublesSketch field), the job succeeds.
e.g. including this in the metricsSpec is fine:

        {
          "type": "quantilesDoublesSketch",
          "name": "ds_ResponseTime",
          "fieldName": "ds_ResponseTime",
          "k": 1024,
          "maxStreamLength": 1000000000
        },

Metadata

Metadata

Assignees

No one assigned

    Labels

    Uncategorized problem report

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions