AGENT_TOOLSDIRECTORY override environment variable in self-hosted linux vm doesn't work

[xinyuxu1026]

Hi,

I have a 1es hosted pool with RHEL 8, and I installed python under /opt/hostedtoolscache, currently I need to run UsePythonVersion@0 job with the override AGENT_TOOLSDIRECTORY as instructed.

Here is my job:

- task: UsePythonVersion@0
  name: UsePythonVersion
  displayName: Use Python 3.x
  env:
    AGENT_TOOLSDIRECTORY: /opt/hostedtoolscache

But it still complaining under the default tool directory:

##[error]Version spec 3.x for architecture x64 did not match any version in Agent.ToolsDirectory.
Versions in /mnt/vss/_work/_tool:

How should I override the AGENT_TOOLSDIRECTORY environment variable? Thanks.

[ivanduplenskikh]

@xinyuxu1026, according to the documentation, it is necessary for you to specify directory structure as follows:

├─ $AGENT_TOOLSDIRECTORY/
|   ├─ Python/
|   |   ├─ {version number}/
|   |   |   ├─ {platform}/
|   |   |   |   ├─ {tool files}
|   |   |   ├─ {platform}.complete

Could you please confirm that the directory structure is specified as in the above example?

[xinyuxu1026]

@ivanduplenskikh , thanks, yes I confirm that I have that directory structure.
But how to override the $AGENT_TOOLSDIRECTORY? why it is still looking for /mnt/vss/_work/_tool this directory?

[ivanduplenskikh]

@xinyuxu1026, could you try to override this variable in the variables block?
For example:

variables:
  AGENT_TOOLSDIRECTORY: "path_to_the_custom_directory_with"

steps:
- task: UsePythonVersion@0
  name: UsePythonVersion
  displayName: Use Python 3.x

You can also try to create .env file for the individual agent.
Please refer to this documentation for more details.

[vmapetr]

Hi @xinyuxu1026, thank you for your question!
This is actually an expected behavior. There is three scopes of environment variables:

• The root level, which make the variable available to all jobs in the pipeline (the same scope can be set by defining variable as pipeline variable in UI):

variables:
    AGENT_TOOLSDIRECTORY: /opt/hostedtoolscache

jobs:
- job: Test
  pool: 'Default'
  steps:
    - task: UsePythonVersion@0
      name: UsePythonVersion
      displayName: Use Python 3.x

• The stage level, which make it available only to a specific stage:

jobs:
- job: Test
  pool: 'Default'
  variables:
      AGENT_TOOLSDIRECTORY: /opt/hostedtoolscache
  steps:
    - task: UsePythonVersion@0
      name: UsePythonVersion
      displayName: Use Python 3.x

• And the job level, which make the variable available only to a specific job:

- task: UsePythonVersion@0
  name: UsePythonVersion
  displayName: Use Python 3.x
  env:
    AGENT_TOOLSDIRECTORY: /opt/hostedtoolscache

Please notice that scopes for the environment variables have following priorities:

• root variables are used in case if job step has no definition for environment variables with the same name.
• In case if job variable is defined, it has higher priority than the root and the task variables
• The task level variable have the lowest priority and will be used only in case if there is no variables with the same name at higher scopes
  You can check more details about environment variables in this documentation.

However, please note that AGENT_TOOLSDIRECTORY is not a custom, but predefined variable which is injected by the AzureDevOps Agent during the pipeline Job initialization. Hence, you cannot redefine it at the task level, since the task stage initialization happens after the job initialization and the job variable scope has higher priority than the task - so the variable can be redefined only on the root or the job level.
Please check this documentation for more details about the predefined variables.

We also noticed race condition bug in the agent, when in some cases, the custom AGENT_TOOLSDIRECTORY variable can be overwritten by the default value from the agent - ../_work/_tool, even if the variable is redefined on the root or job scope in YAML pipeline.
We're working on the fix and notify you when it will be done.

[xinyuxu1026]

Thanks @vmapetr , I tried to use the root variables here. The azure pipeline pool(ubuntu) works well, but the 1es hosted pool job is still looking for _work/_tool dir. Have you tested with 1es hosted pool? is there any alternatives to resolve this? Thank you.

[xinyuxu1026]

Hi @vmapetr , is there any update on this issue? Thanks.

[vmapetr]

Hi @xinyuxu1026, sorry for the delayed response.
We weren't aware of this issue previously, thank you for pointing on this.
AFAIK, it wasn't tested for 1ES, only for the default hosted MS and GitHub pools. We will investigate this issue and notify you as soon as possible.