API Inference Scripts

This repository contains Python scripts for running model inference using APIs from the CBorg platform at Lawrence Berkeley National Laboratory (LBL) and the American Science Cloud (AmSC).

Repository Structure

.
└── Inference/
    ├── AmSC
    └── CBorgLBL 

Overview

The scripts in this repository demonstrate how to send prompts to remotely hosted models using API endpoints and retrieve the generated responses. They are intended as simple utilities for experimentation, benchmarking, and automated evaluation workflows.

Requirements

• Python 3.9+
• Python packages:
  • openai
  • argparse

Install dependencies with:

pip install openai

Authentication

Access to the APIs requires authentication keys provided by the respective services.

Example (AmSC):

export AMSC_API_KEY=<your_api_key>

Purpose

These scripts are intended for:

• Testing and benchmarking models hosted on CBorg (LBL) and AmSC
• Running prompt-based experiments
• Automating inference workflows from the command line

Notes

• API keys must be configured through environment variables.
• Model availability depends on the hosting platform.