Loading precompiled binaries from cvmfs is not possible?

[Ni-Langguth]

I published two test packages (using ali-bot) to one of our cvmfs test repositories using the aliPublish.conf file below and manually published the BASE/1.0 file below to the same location as the test packages: /cvmfs/sft-nightlies-test.cern.ch/lcg/bits/x86_64-el9-gcc13-opt/Modules/modulefiles, as instructed in the documentation.

Then I added the path to my MODULEPATH, checked with module avail whether the modules were available and tried loading/unloading with module load and module unload, which worked:

export MODULEPATH="/cvmfs/sft-nightlies-test.cern.ch/lcg/bits/x86_64-el9-gcc13-opt/Modules/modulefiles:$MODULEPATH"
[nlanggut@pcepsft17 bits]$ module avail
---------------- /cvmfs/sft-nightlies-test.cern.ch/lcg/bits/x86_64-el9-gcc13-opt/Modules/modulefiles -----------------
alibuild-recipe-tools/0.2.5-1  BASE/1.0  ninja/fortran-v1.11.1.g9-1  

------------------------------------------- /usr/share/Modules/modulefiles -------------------------------------------
dot  module-git  module-info  modules  null  use.own  

Key:
modulepath  
[nlanggut@pcepsft17 bits]$ module load ninja/fortran-v1.11.1.g9-1 
Loading ninja/fortran-v1.11.1.g9-1
  Loading requirement: BASE/1.0
[nlanggut@pcepsft17 bits]$ which ninja
/cvmfs/sft-nightlies-test.cern.ch/lcg/bits/x86_64-el9-gcc13-opt/Packages/ninja/fortran-v1.11.1.g9-1/bin/ninja
[nlanggut@pcepsft17 bits]$ module unload ninja/fortran-v1.11.1.g9-1 
Unloading ninja/fortran-v1.11.1.g9-1
  Unloading useless requirement: BASE/1.0

Then I tried doing the same with bits, specifying my architecture manually and querying which modules were available with bits q, trying to load a module with bits load, which is where I ran into an issue:

[nlanggut@pcepsft17 bits]$ bits -a x86_64-el9-gcc13-opt q
VO_ALICE@BASE::1.0
VO_ALICE@alibuild-recipe-tools::0.2.5-1
VO_ALICE@BASE::1.0
VO_ALICE@ninja::fortran-v1.11.1.g9-1
[nlanggut@pcepsft17 bits]$ bits -a x86_64-el9-gcc13-opt load VO_ALICE@ninja::fortran-v1.11.1.g9-1
Use bits list to list loaded modules.
Loading ninja/fortran-v1.11.1.g9-1
  Loading requirement: BASE/1.0
PATH=/afs/cern.ch/user/n/nlanggut/bits/bits/sw/x86_64-el9-gcc13-opt/ninja/fortran-v1.11.1.g9-1/bin:/afs/cern.ch/user/n/nlanggut/.cargo/bin:/afs/cern.ch/user/n/nlanggut/scripts:/afs/cern.ch/user/n/nlanggut/.cargo/bin:/afs/cern.ch/user/n/nlanggut/.local/bin:/afs/cern.ch/user/n/nlanggut/bin:/usr/sue/bin:/usr/share/Modules/bin:/usr/lib64/ccache:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/var/lib/snapd/snap/bin:/opt/puppetlabs/bin; export PATH;
__MODULES_LMALTNAME=BASE/1.0\&as\|BASE/default\&as\|BASE/latest:ninja/fortran-v1.11.1.g9-1\&as\|ninja/default\&as\|ninja/latest; export __MODULES_LMALTNAME;
__MODULES_LMPREREQ=ninja/fortran-v1.11.1.g9-1\&BASE/1.0; export __MODULES_LMPREREQ;
_LMFILES_=/afs/cern.ch/user/n/nlanggut/bits/bits/sw/MODULES/x86_64-el9-gcc13-opt/BASE/1.0:/cvmfs/sft-nightlies-test.cern.ch/lcg/bits/x86_64-el9-gcc13-opt/Modules/modulefiles/ninja/fortran-v1.11.1.g9-1; export _LMFILES_;
LOADEDMODULES=BASE/1.0:ninja/fortran-v1.11.1.g9-1; export LOADEDMODULES;
BASEDIR=/afs/cern.ch/user/n/nlanggut/bits/bits/sw/x86_64-el9-gcc13-opt; export BASEDIR;
__MODULES_LMTAG=BASE/1.0\&auto-loaded; export __MODULES_LMTAG;
test 0;
[nlanggut@pcepsft17 bits]$ bits -a x86_64-el9-gcc13-opt list
No Modulefiles Currently Loaded.
[nlanggut@pcepsft17 bits]$ which ninja
/usr/bin/which: no ninja in (/afs/cern.ch/user/n/nlanggut/.cargo/bin:/afs/cern.ch/user/n/nlanggut/scripts:/afs/cern.ch/user/n/nlanggut/.cargo/bin:/afs/cern.ch/user/n/nlanggut/.local/bin:/afs/cern.ch/user/n/nlanggut/bin:/usr/sue/bin:/usr/share/Modules/bin:/usr/lib64/ccache:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/var/lib/snapd/snap/bin:/opt/puppetlabs/bin)

aliPublish.conf

---
s3_endpoint_url: https://s3.cern.ch
s3_bucket: bucketofpieces
base_url: https://s3.cern.ch/bucketofpieces/
base_prefix: TARS

http_ssl_verify: True

architectures:
  x86_64-el9-gcc13-opt:
    CVMFS: x86_64-el9-gcc13-opt
include:
  ninja: True

cvmfs_repository: sft-nightlies-test.cern.ch
cvmfs_package_dir: /cvmfs/sft-nightlies-test.cern.ch/lcg/bits/%(arch)s/Packages/%(package)s/%(version)s
cvmfs_modulefile: /cvmfs/sft-nightlies-test.cern.ch/lcg/bits/%(arch)s/Modules/modulefiles/%(package)s/%(version)s

BASE/1.0

#%Module1.0#####################################################################
##
## lcg.bits - versions modulefile
##

proc ModulesHelp { } {
        global version
        puts stderr "Common base module for LCG modules."
}

set     version         1.0

module-whatis   "ALICE versions module for ALICE software."

set base_path [string map "Modules/modulefiles/BASE/$version Packages" $ModulesCurrentModulefile]
if { $base_path == $ModulesCurrentModulefile } {
  set base_path /cvmfs/sft-nightlies-test.cern.ch/lcg/bits/x86_64-el9-gcc13-opt/Packages
}

## -- BASE DIRECTORYS --
setenv          BASEDIR              $base_path

set     osname          [uname sysname]
set     osarchitecture  [uname machine]

if { $osarchitecture == "x86_64" } {
        setenv  ALICE_TARGET       linuxx8664gcc
        setenv  ALICE_TARGET_EXT   linuxx8664gcc
        setenv  SYSTEM_LIBPATH     lib64
} elseif { $osarchitecture == "i686" } {
        setenv  ALICE_TARGET    linux
        setenv  ALICE_TARGET_EXT   linux
        setenv  SYSTEM_LIBPATH     lib
} else {
        puts stderr "\tYour architecture is $osarchitecture. Please use x86_64 or i686 architecture"
        break
}

[pbuncic]

You should never mess with modules or MODULEPATH, simply use 'bits enter' to enter a subshell with properly configured environment (which replaces 'alienv enter' from aliBuild). Here is an example of ROOT:

pbuncic@bits:~/Software/Bits/alice.bits$ bits q | grep ROOT
VO_ALICE@ROOT::latest
VO_ALICE@ROOT::latest-o2
pbuncic@bits:~/Software/Bits/alice.bits$ bits enter ROOT/latest
Loading ROOT/latest
  Loading requirement: BASE/1.0 libffi/v3.2.1-alice1-local1 lz4/v1.9.3-local1
    hdf5/1.14.6-local1 AliEn-Runtime/v2-19-le-local1 sqlite/v3.47.2-local1
    Python/v3.10.19-local1 Python-modules/1.0-local1 boost/v1.83.0-alice2-local1
    protobuf/v29.3-local1 Clang/v20.1.7-local1 utf8proc/v2.6.1-local1
    xsimd/8.1.0-local1 arrow/v20.0.0-alice1-local1 GSL/v2.8-local1
    lhapdf/v6.5.2-local1 HepMC/HEPMC_02_06_10-local1 Vc/1.4.5-local1
    TBB/v2021.5.0-local1 nlohmann_json/v3.11.3-local1 abseil/20240722.0-local1
    pythia/v8315-alice1-local1 XRootD/v5.8.4-local1 FFTW3/v3.3.9-local1
Currently Loaded Modulefiles:
 1) BASE/1.0                       14) arrow/v20.0.0-alice1-local1
 2) libffi/v3.2.1-alice1-local1    15) GSL/v2.8-local1
 3) lz4/v1.9.3-local1              16) lhapdf/v6.5.2-local1
 4) hdf5/1.14.6-local1             17) HepMC/HEPMC_02_06_10-local1
 5) AliEn-Runtime/v2-19-le-local1  18) Vc/1.4.5-local1
 6) sqlite/v3.47.2-local1          19) TBB/v2021.5.0-local1
 7) Python/v3.10.19-local1         20) nlohmann_json/v3.11.3-local1
 8) Python-modules/1.0-local1      21) abseil/20240722.0-local1
 9) boost/v1.83.0-alice2-local1    22) pythia/v8315-alice1-local1
10) protobuf/v29.3-local1          23) XRootD/v5.8.4-local1
11) Clang/v20.1.7-local1           24) FFTW3/v3.3.9-local1
12) utf8proc/v2.6.1-local1         25) ROOT/latest
13) xsimd/8.1.0-local1

Key:
auto-loaded
Use bits list to list loaded modules. Use exit to exit this environment.
[ROOT/latest] ~/Software/Bits/alice.bits $> which root
/home/pbuncic/Software/Bits/sw/ubuntu2510_x86-64/ROOT/v6-36-04-alice3-local1/bin/root
[ROOT/latest] ~/Software/Bits/alice.bits $> root -b
   ------------------------------------------------------------------
  | Welcome to ROOT 6.36.04                        https://root.cern |
  | (c) 1995-2025, The ROOT Team; conception: R. Brun, F. Rademakers |
  | Built for linuxx8664gcc on Dec 16 2025, 14:43:19                 |
  | From tags/v6-36-04-alice3@v6-36-04-alice3                        |
  | With c++ (Ubuntu 15.2.0-4ubuntu4) 15.2.0                         |
  | Try '.help'/'.?', '.demo', '.license', '.credits', '.quit'/'.q'  |
   ------------------------------------------------------------------

root [0] .q
[ROOT/latest] ~/Software/Bits/alice.bits $> exit
exit

[Ni-Langguth]

Alright, so I reset everything I did with module and cloned the alice.bits repo to my docker, just to recreate exactly what you are doing before I try it with our lcg packages.

It seems to me like there is a step missing here, because when I do bits q, I only get the BASE/1.0 module. How can I communicate to bits that it should look in the cvmfs repository where ALICE publish their packages?

[root@lcgapp-alma9-physical alice.bits]# bits q
VO_ALICE@BASE::1.0