Skip to content

dcamarmas/xpn

 
 

Repository files navigation

Logo for the Expand Parallel File System.

Expand Project

Release License Codacy Stars Forks

URL
🌐 Project Website https://xpn-arcos.github.io
📦 Docker Hub https://hub.docker.com/repositories/expandfs
📧 Email xpn.arcos.inf.uc3m.es@gmail.com

Source Code

C MPI Shell

URL
📋 Source Code https://github.com/xpn-arcos/xpn
📦 XPN Docker https://github.com/xpn-arcos/xpn-docker
🔬 Nightly Build https://github.com/xpn-arcos/xpn-beta

Content

  1. Publications
  2. Authors
  3. ChangeLog
  4. Usage
  5. Sponsors

Publications

2025

📰 Improving I/O performance in HPC environments using the Expand Ad-Hoc file system
  • Journal paper: The Journal of Supercomputing
  • Authors: Diego Camarmas-Alonso, Felix Garcia-Carballeira, Alejandro Calderon-Mateos, Jesus Carretero
  • 🔗 Open publication
@article{camarmas2025improving,
  title={Improving I/O performance in HPC environments using the Expand Ad-Hoc file system},
  author={Camarmas-Alonso, Diego and Garcia-Carballeira, Felix and Calderon-Mateos, Alejandro and Carretero, Jesus},
  journal={The Journal of Supercomputing},
  volume={81},
  number={16},
  pages={1--42},
  year={2025},
  publisher={Springer}
}
📰 Malleability and fault tolerance in ad-hoc parallel file systems
  • Journal paper: Cluster Computing
  • Authors: Dario Muñoz-Muñoz, Felix Garcia-Carballeira, Diego Camarmas-Alonso, Alejandro Calderon-Mateos, Jesus Carretero
  • 🔗 Open publication
@article{munoz2025malleability,
  title={Malleability and fault tolerance in ad-hoc parallel file systems},
  author={Mu{\~n}oz-Mu{\~n}oz, Dario and Garcia-Carballeira, Felix and Camarmas-Alonso, Diego and Calderon-Mateos, Alejandro and Carretero, Jesus},
  journal={Cluster Computing},
  volume={28},
  number={13},
  pages={860},
  year={2025},
  publisher={Springer}
}
📰 Hierarchical and distributed data storage for Computing Continuum
  • Journal paper: Future Generation Computer Systems
  • Authors: Elias Del-Pozo-Puñal, Felix Garcia-Carballeira, Diego Camarmas-Alonso, Alejandro Calderon-Mateos
  • 🔗 Open publication
@article{DELPOZOPUNAL2026107931,
  title = {Hierarchical and distributed data storage for computing continuum},
  journal = {Future Generation Computer Systems},
  volume = {174},
  pages = {107931},
  year = {2026},
  issn = {0167-739X},
  doi = {https://doi.org/10.1016/j.future.2025.107931},
  url = {https://www.sciencedirect.com/science/article/pii/S0167739X25002262},
  author = {Elias Del-Pozo-Puñal and Felix Garcia-Carballeira and Diego Camarmas-Alonso and Alejandro Calderon-Mateos}
}
📰 Evaluación del sistema de ficheros Expand Ad-Hoc con aplicaciones de uso intensivo de datos
  • Conference paper: XXXV Jornadas de Paralelismo (JP25)
  • Authors: Diego Camarmas-Alonso, Felix Garcia-Carballeira, Alejandro Calderon-Mateos, Darío Muñoz-Muñoz, Jesus Carretero
  • 🔗 Open publication
@article{DIEGOCAMARMASALONSO_JP25,
  title = {Evaluación del sistema de ficheros Expand Ad-Hoc con aplicaciones de uso intensivo de datos},
  conference = {XXXV Jornadas de Paralelismo (JP25)},
  volume = {1},
  pages = {305-314},
  year = {2025},
  doi = {https://doi.org/10.5281/zenodo.15773123},
  url = {https://zenodo.org/records/15773123},
  author = {Diego Camarmas-Alonso, Felix Garcia-Carballeira, Alejandro Calderon-Mateos, Darío Muñoz-Muñoz, Jesus Carretero}
}
📰 Sistema de almacenamiento para computing continuum: aplicación a sistemas de información ferroviaria
  • Conference paper: XXXV Jornadas de Paralelismo (JP25)
  • Authors: Elías Del-Pozo-Puñal, Félix García-Carballeira, Diego Camarmas-Alonso, Alejandro Calderon-Mateos
  • 🔗 Open publication
@article{DELPOZOPUNAL_JP25,
  title = {Sistema de almacenamiento para computing continuum: aplicación a sistemas de información ferroviaria},
  conference = {XXXV Jornadas de Paralelismo (JP25)},
  volume = {1},
  pages = {315-324},
  year = {2025},
  doi = {https://doi.org/10.5281/zenodo.15773179},
  url = {https://zenodo.org/records/15773179},
  author = {Elías Del-Pozo-Puñal, Félix García-Carballeira, Diego Camarmas-Alonso, Alejandro Calderon-Mateos}
}
📰 LFI: una librería de comunicaciones tolerante a fallos para redes de alto rendimiento
  • Conference paper:XXXV Jornadas de Paralelismo (JP25)
  • Authors: Darío Muñoz-Muñoz, Félix García-Carballeira, Diego Camarmas-Alonso, Alejandro Calderon-Mateos
  • 🔗 Open publication
@article{DARIOMUNOZMUNOZ_JP25,
  title = {LFI: una librería de comunicaciones tolerante a fallos para redes de alto rendimiento},
  conference = {XXXV Jornadas de Paralelismo (JP25)},
  volume = {1},
  pages = {205-214},
  year = {2025},
  doi = {https://doi.org/10.5281/zenodo.15773016},
  url = {https://zenodo.org/records/15773016},
  author = {Darío Muñoz-Muñoz, Félix García-Carballeira, Diego Camarmas-Alonso, Alejandro Calderon-Mateos}
}
📰 Optimización de entornos de Big Data Analytics mediante sistemas de ficheros paralelos Ad-hoc
  • Conference paper: XXXV Jornadas de Paralelismo (JP25)
  • Authors: Gabriel Sotodosos-Morales, Félix García-Carballeira, Diego Camarmas-Alonso, Alejandro Calderón-Mateos, Darío Muñoz-Muñoz, Jesús Carretero
  • 🔗 Open publication
@article{SOTODOSOSMORALES_JP25,
  title = {Optimización de entornos de Big Data Analytics mediante sistemas de ficheros paralelos Ad-hoc},
  conference = {XXXV Jornadas de Paralelismo (JP25)},
  volume = {1},
  pages = {833-842},
  year = {2025},
  doi = {https://doi.org/10.5281/zenodo.15773242},
  url = {https://zenodo.org/records/15773242},
  author = {Gabriel Sotodosos-Morales, Félix García-Carballeira, Diego Camarmas-Alonso, Alejandro Calderón-Mateos, Darío Muñoz-Muñoz, Jesús Carretero}
}

2024

📰 Fault tolerant in the Expand Ad-Hoc parallel file system
  • Conference paper: 30th International European Conference on Parallel and Distributed Computing (Euro-Par)
  • Authors: Dario Muñoz-Muñoz, Felix Garcia-Carballeira, Diego Camarmas-Alonso, Alejandro Calderon-Mateos, Jesus Carretero
  • 🔗 Open publication
@InProceedings{10.1007/978-3-031-69766-1_5,
  author="Mu{\~{n}}oz-Mu{\~{n}}oz, Dario and Garcia-Carballeira, Felix and Camarmas-Alonso, Diego and Calderon-Mateos, Alejandro and Carretero, Jesus",
  title="Fault Tolerant in the Expand Ad-Hoc Parallel File System",
  booktitle="Euro-Par 2024: Parallel Processing",
  year="2024",
  publisher="Springer Nature Switzerland",
  address="Cham",
  pages="62--76",
  isbn="978-3-031-69766-1"
}
📰 Malleability in the Expand Ad-Hoc parallel file system
  • Conference paper: 3rd EuroHPC Workshop on Dynamic Resources in HPC. Euro-Par 2024
  • Authors: Dario Muñoz-Muñoz, Felix Garcia-Carballeira, Diego Camarmas-Alonso, Alejandro Calderon-Mateos, Jesus Carretero
  • 🔗 Open publication
@InProceedings{10.1007/978-3-031-90200-0_26,
  author="Mu{\~{n}}oz-Mu{\~{n}}oz, Dario and Garcia-Carballeira, Felix and Camarmas-Alonso, Diego and Calderon-Mateos, Alejandro and Carretero, Jesus",
  title="Malleability in the Expand Ad-Hoc Parallel File System",
  booktitle="Euro-Par 2024: Parallel Processing Workshops",
  year="2025",
  publisher="Springer Nature Switzerland",
  address="Cham",
  pages="322--333",
  isbn="978-3-031-90200-0"
}
📰 Tolerancia a fallos en el sistema de ficheros paralelo Expand Ad-Hoc
  • Conference paper: XXXIV Jornadas de Paralelismo (JP24)
  • Authors: Dario Muñoz-Muñoz, Diego Camarmas-Alonso, Felix Garcia-Carballeira, Alejandro Calderon-Mateos, Jesus Carretero
  • 🔗 Open publication
@inproceedings{munoz_munoz_2024_12743583,
  author       = {Muñoz-Muñoz, Dario and  Camarmas Alonso, Diego and  Garcia-Carballeira, Felix and Calderon-Mateos, Alejandro and Carretero, Jesus},
  title        = {Tolerancia a fallos en el sistema de ficheros paralelo Expand Ad-Hoc},
  booktitle    = {Avances en Arquitectura y Tecnología de Computadores. Actas de las Jornadas SARTECO},
  year         = 2024,
  pages        = {271-279},
  publisher    = {Zenodo},
  month        = jul,
  venue        = {A Coruña, España},
  doi          = {10.5281/zenodo.12743583},
  url          = {https://doi.org/10.5281/zenodo.12743583},
}
📰 Evaluación del rendimiento de un sistema de ficheros para sistemas IoT
  • Conference paper: XXXIV Jornadas de Paralelismo (JP24)
  • Authors: Elias Del-Pozo-Puñal, Felix Garcia-Carballeira, Diego Camarmas-Alonso, Alejandro Calderon-Mateos
  • 🔗 Open publication
@inproceedings{del_pozo_punal_2024_12094742,
  author       = {Del-Pozo-Puñal, Elias and  Garcia-Carballeira, Felix and Camarmas-Alonso, Diego and Calderon-Mateos, Alejandro},
  title        = {Evaluación del rendimiento de un sistema de ficheros para sistemas IoT},
  booktitle    = {Avances en Arquitectura y Tecnología de Computadores. Actas de las Jornadas SARTECO},
  year         = 2024,
  pages        = {289-298},
  publisher    = {Zenodo},
  month        = jun,
  venue        = {A Coruña, Galicia, España},
  doi          = {10.5281/zenodo.12094742},
  url          = {https://doi.org/10.5281/zenodo.12094742},
}

2023

📰 A new Ad-Hoc parallel file system for HPC environments based on the Expand parallel file system
  • Conference paper: 22nd International Symposium on Parallel and Distributed Computing (ISPDC)
  • Authors: Felix Garcia-Carballeira, Diego Camarmas-Alonso, Alejandro Calderon-Mateos, Jesus Carretero
  • 🔗 Open publication
@INPROCEEDINGS{10272428,
  author={Garcia-Carballeira, Felix and Camarmas-Alonso, Diego and Caderon-Mateos, Alejandro and Carretero, Jesus},
  booktitle={2023 22nd International Symposium on Parallel and Distributed Computing (ISPDC)}, 
  title={A new Ad-Hoc parallel file system for HPC environments based on the Expand parallel file system}, 
  year={2023},
  volume={},
  number={},
  pages={69-76},
  keywords={File systems;Distributed computing;Ad-Hoc File System;HPC;Parallel I/O},
  doi={10.1109/ISPDC59212.2023.00015}
}
📰 Evaluación de rendimiento del sistema de ficheros paralelo Expand Ad-Hoc en MareNostrum 4
  • Conference paper: XXXIII Jornadas de Paralelismo (JP23)
  • Authors: Diego Camarmas-Alonso, Felix Garcia-Carballeira, Alejandro Calderon-Mateos, Jesus Carretero
  • 🔗 Open publication
@misc{diego_camarmas_alonso_2023_8378956,
  author       = {Diego Camarmas-Alonso and Felix Garcia-Carballeira and Alejandro Calderon-Mateos and  Jesus Carretero},
  title        = {Evaluación de rendimiento del sistema de ficheros paralelo Expand Ad-Hoc en MareNostrum 4},
  month        = sep,
  year         = 2023,
  publisher    = {Zenodo},
  doi          = {10.5281/zenodo.8378956},
  url          = {https://doi.org/10.5281/zenodo.8378956},
}
📰 Sistema de Ficheros Distribuido para IoT basado en Expand
  • Conference paper: XXXIII Jornadas de Paralelismo (JP23)
  • Authors: Elias Del-Pozo-Puñal, Felix Garcia-Carballeira, Diego Camarmas-Alonso, Alejandro Calderon-Mateos
  • 🔗 Open publication
@misc{del_pozo_punal_2023_10706248,
  author       = {Del-Pozo-Puñal, Elías and Garcia-Carballeira, Felix andCamarmas-Alonso, Diego andCalderon-Mateos, Alejandro},
  title        = {Sistema de Ficheros Distribuido para IoT basado en Expand},
  month        = sep,
  year         = 2023,
  publisher    = {Zenodo},
  version      = {Version v1},
  doi          = {10.5281/zenodo.10706248},
  url          = {https://doi.org/10.5281/zenodo.10706248},
}

2022

📰 Sistema de almacenamiento Ad-Hoc para entornos HPC basado en el sistema de ficheros paralelo Expand
  • Conference paper: XXXII Jornadas de Paralelismo (JP22)
  • Authors: Diego Camarmas-Alonso, Felix Garcia-Carballeira, Alejandro Calderon-Mateos, Jesus Carretero
  • 🔗 Open publication
@misc{diego_camarmas_alonso_2021_14258796,
  author       = {Diego Camarmas-Alonso and Felix Garcia-Carballeira and Alejandro Calderon Mateos and Jesus Carretero Perez},
  title        = {Sistema de almacenamiento Ad-Hoc para entornos HPC basado en el sistema de ficheros paralelo Expand},
  month        = sep,
  year         = 2021,
  publisher    = {Zenodo},
  doi          = {10.5281/zenodo.14258796},
  url          = {https://doi.org/10.5281/zenodo.14258796},
}

Authors

ChangeLog

:atom: 3.3.1
  • Expand proxy added
  • Support for Apache Spark text and image Dataframes
  • C3 Platform added
  • Minor updates
:atom: 3.3.0
  • Spark connector added
  • IPv6 support added
  • Added support for 32-bits version of Expand
:atom: 3.2.0
  • MQTT support added: mpi_server, sck_server and now mq_server available
  • Cleanup and minor improvements on socket support
  • Improvements on the configuration file reader
  • Minor updates on examples
:atom: 3.1.0
  • Metadata operations improved
  • Minor bug fixes and improvements
:atom: 3.0.0
  • Fault tolerance support based on replication added
  • Maleability support improved
  • Expand deployment based on docker added
  • MXML dependency removed
  • Simplified user experience: more user-friendly way to start and stop ad hoc server
  • Minor bug fixes and improvements
:atom: 2.2.2
  • TCP server rebuilt from scratch
  • New system calls intercepted
  • Preload and flush data operations added
  • Minor bug fixes and improvements
:atom: 2.2.1
  • Minor bug fixes and improvements
:atom: 2.2.0
  • Maleability support
:atom: 2.1.0
  • Spack support added
  • Simplified user experience: now it is easier to start and stop expand ad-hoc servers
  • Code refactoring
:atom: 2.0.0
  • First XPN Ad-Hoc release
  • This version provides:
    • Simplifiyed build system based on build-me script
    • Platforms: MPICH and Intel MPI
    • Benchmarks supported: IOR, MdTest and IO500
    • API: POSIX (through syscall interception library) and native XPN
    • Features: data locality, MPI Ad-Hoc servers, thread on-demand or thread pool for MPI servers

Using Expand

See docs/RUNNING.md.

Sponsors

Expand - High performance storage system for HPC and Big Data environments (TED2021-131798B-I00) funded by MICIU/AEI/ 10.13039/501100011033 and by the European Union NextGenerationEU/PRTR. https://expand-arcos.github.io sponsor logos

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • C 80.3%
  • Shell 8.7%
  • Java 4.0%
  • Python 2.1%
  • Makefile 1.8%
  • RPC 1.7%
  • Other 1.4%