HDFS-16842. DatanodeHttpServer does not update config after refreshNamenodes#8291
Open
balodesecurity wants to merge 8 commits intoapache:trunkfrom
Open
HDFS-16842. DatanodeHttpServer does not update config after refreshNamenodes#8291balodesecurity wants to merge 8 commits intoapache:trunkfrom
balodesecurity wants to merge 8 commits intoapache:trunkfrom
Conversation
…menodes. When DataNode.refreshNamenodes() is called to add a new nameservice it reloads configuration via setConf(new Configuration()), but the DatanodeHttpServer kept holding a stale reference to the old Configuration. Subsequent WebHDFS requests for blocks on the newly added nameservice would fail because the DFSClient created per-request could not resolve the namenode addresses for that nameservice. Fix: make the `conf` and `confForCreate` fields in DatanodeHttpServer volatile (so Netty I/O threads see writes atomically) and expose a new updateConf(Configuration) method that refreshes both fields plus the embedded Jetty infoServer's servlet-context attributes. Call httpServer.updateConf(getConf()) at the end of DataNode.refreshNamenodes() after the new configuration has been loaded. The Netty ChannelInitializer creates a fresh URLDispatcher per connection; because it accesses the DatanodeHttpServer instance fields at connection-time, new connections automatically pick up the updated configuration with no pipeline restart required. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
Author
|
CI failed due to Jenkins Docker infrastructure issue (apt-get failure and yetus/hadoop image pull denied) — unrelated to the patch. Requesting retest. /retest |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When
DataNode.refreshNamenodes()is called to add a new nameservice it reloads configuration viasetConf(new Configuration()), butDatanodeHttpServerkept holding a stale reference to the oldConfiguration. Subsequent WebHDFS requests for blocks on the newly-added nameservice would fail because theDFSClientcreated per-request could not resolve the namenode addresses for that nameservice from the stale config.Root cause:
DatanodeHttpServer.confandconfForCreatewerefinalfields set once at construction time.DataNode.refreshNamenodes()never called any update method onhttpServer.Fix
confandconfForCreateinDatanodeHttpServervolatile(non-final) so Netty I/O threads see updates atomically.DatanodeHttpServer.updateConf(Configuration newConf)that refreshes both fields and the embedded JettyinfoServer's servlet-context attributes (CONF_CONTEXT_ATTRIBUTE,CURRENT_CONF).httpServer.updateConf(getConf())at the end ofDataNode.refreshNamenodes()after the new configuration has been loaded.The Netty
ChannelInitializercreates a freshURLDispatcherper connection and readsDatanodeHttpServer.confat that point, so new connections automatically pick up the updated configuration without requiring a pipeline or server restart.Testing
TestDatanodeHttpServerUpdateConf.testUpdateConfRefreshesFields— a unit test (no cluster required) that:DatanodeHttpServerwith an initial configuration.updateConf(newConf).confandconfForCreateinstance fields now reference the updated configuration.infoServerservlet-context attributes have been updated.Related