POKEDEX

A Chinese-language Pokémon Pokédex that can be mounted on a personal website. Open the page on a phone, take a photo with the camera or select an existing image, and a CLIP model running locally in the browser will generate an image embedding. The embedding is compared with pre-generated embeddings of official Pokémon artwork, after which the closest Pokédex entry is displayed and read aloud in Chinese.

Photos are never sent to a server. Matching results are provided solely for entertainment as a measure of visual similarity. They are not assessments of identity, personality, or biological characteristics.

Try it online: POKEDEX

Technical Architecture

• Vite + TypeScript with no UI framework, making the project easy to embed into an existing website.
• Uses getUserMedia to access a mobile device’s camera.
• Runs CLIP through @huggingface/transformers inside a Web Worker.
• Prefers WebGPU and automatically falls back to WASM when WebGPU is unavailable.
• A PokeAPI build script generates Simplified Chinese names, categories, descriptions, types, heights, weights, and image URLs.
• Pokémon reference embeddings are generated before deployment and compressed into a single Int8/Base64 index file.
• Uses the Web Speech API for neutral Mandarin Chinese speech output.
• All configurable parameters are centralized in config/pokedex.config.json.

Running the Project

Node.js 20.19 or later is required. An internet connection is also required during the initial data and embedding preparation process.

The recommended method is to use the project’s built-in setup command. It skips unnecessary CUDA/NuGet downloads from onnxruntime-node, then generates the Pokédex data and reference embeddings:

npm run setup
npm run dev

You can also run the steps individually:

ONNXRUNTIME_NODE_INSTALL=skip npm install
npm run prepare:assets
npm run dev

ONNXRUNTIME_NODE_INSTALL=skip only skips the optional CUDA provider download for Node.js. CPU inference required for generating the reference embeddings remains available, while the mobile web application will continue to use WebGPU or WASM according to its configuration.

Open the local URL displayed in the terminal. Desktop browsers can use the camera directly. Testing on a mobile phone requires HTTPS or a secure preview URL provided by the development environment.

npm run prepare:assets prepares all offline browser assets:

npm run prepare:data        # Generate Chinese Pokédex JSON from PokeAPI
npm run prepare:embeddings  # Download the model and generate Pokémon reference embeddings
npm run prepare:browser-model # Download the quantized browser model into public/models
npm run prepare:runtime      # Copy ONNX Runtime WASM into public/ort

The scripts store raw data, model files, and the intermediate embedding for each Pokémon in .cache/. If the process is interrupted, completed downloads and embeddings can be reused the next time it runs.

Linux / WSL Installation Issues

If a normal npm install fails during the onnxruntime-node stage while trying to access api.nuget.org, the installation process is attempting to download an additional CUDA provider that this project does not require.

Remove the incomplete dependencies and reinstall:

rm -rf node_modules package-lock.json
ONNXRUNTIME_NODE_INSTALL=skip npm install

If public/data/pokemon.zh-CN.json has already been generated successfully, there is no need to retrieve the Pokédex data again. Continue with:

npm run prepare:embeddings
npm run dev -- --host 0.0.0.0

If the model download subsequently fails, make sure the machine can access the Hugging Face model files and PokeAPI sprites. The script will reuse completed downloads and individual Pokémon embeddings already stored in .cache/.

Building and Mounting

npm run build

The build output is placed in dist/ and can be deployed to any static hosting service.

When mounting the application under a website subdirectory, such as https://example.com/pokedex/, only one configuration value needs to be changed:

{
  "deployment": {
    "basePath": "/pokedex/"
  }
}

Then run npm run build again.

An iframe is not required. You can place the contents of dist directly in the corresponding subdirectory. When using an iframe, the parent page must grant camera permission:

<iframe src="/pokedex/" allow="camera" title="PERSONAL DEX"></iframe>

Centralized Configuration

All configurable parameters are located in:

config/pokedex.config.json

Common parameters include:

• data.maxPokemonId: The highest National Pokédex number included in matching.
• model.id: The browser application and preparation scripts must use the same model.
• model.preferredDevice: Defaults to webgpu.
• model.allowMobileWebGpu: Defaults to false, keeping mobile browsers on the more compatible WASM backend.
• model.dtype: Defaults to q4 to reduce memory usage on mobile devices.
• camera.captureSize: The square image size passed to the model.
• matching.topK: The number of candidate matches.
• matching.minimumRawSimilarity: The threshold below which results are discarded.
• matching.displayScoreRawMin/Max: Maps raw model similarity values to the match score displayed in the interface.
• speech.rate/pitch/volume: Controls the speed, pitch, and volume of Chinese speech output.
• matching.autoSpeakBestResult: Determines whether the best result is read aloud automatically.
• ui.theme: Controls the theme colors of the device body and screen.

After changing model.id or the Pokédex range, run:

npm run prepare:assets

Data Field Selection

prepare-data.mjs generates the following structure for each Pokémon:

{
  "id": 25,
  "slug": "pikachu",
  "nameZh": "皮卡丘",
  "nameEn": "Pikachu",
  "categoryZh": "鼠宝可梦",
  "descriptionZh": "……",
  "typesZh": ["电"],
  "heightM": 0.4,
  "weightKg": 6,
  "image": "https://……"
}

Chinese descriptions are selected according to the order defined in preferredDescriptionVersions.

When no Simplified Chinese description is available, the latest English Pokédex description is used instead.

Speech Order

The speech logic is located in src/core/speech.ts:

[nameZh, categoryZh, descriptionZh].join('。')

The Pokédex number, types, height, weight, and match score displayed on the page are not read aloud.

Model and Mobile Performance

The first time the application is used on a device, the browser must download the model files specified in the configuration. Subsequent visits will normally reuse the browser cache.

When WebGPU is unavailable, the application falls back to WASM. Older devices may experience slow loading or insufficient memory. In that case, you can:

1. Switch to a smaller vision model that is compatible with Transformers.js.
2. Keep dtype set to "q8".
3. Temporarily reduce data.maxPokemonId to 151 to shorten the preparation process. The total number of Pokémon has little effect on the time required for a single inference on the phone.

Pokémon reference embeddings are not generated individually on the phone. At runtime, the application only needs to run inference once on the user’s photo and calculate dot products against approximately one thousand small reference vectors.

Privacy and Security

• Camera streams and captured images exist only in the current browser’s memory.
• The project does not include an upload endpoint.
• Photos are not retained after the page is refreshed or closed.
• Camera permission is requested only after an explicit user action.
• Production deployments must use HTTPS.

Content and Copyright

The project’s code and original UI are licensed under the MIT License.

PokeAPI data, model weights, Pokémon names, and character artwork remain subject to their respective licenses and ownership terms.

Before deploying the project publicly, read ATTRIBUTION.md, retain the unofficial-project disclaimer, and do not present the page as an official product.

GitHub Pages: associatedprimeideal.github.io/pokedex/

This version already sets deployment.basePath to /pokedex/ and includes two workflows:

• Prepare Pokédex assets: Run this manually once to generate the Chinese Pokédex data and 1,025 reference embeddings and download the browser model, then commit the generated data and model to the repository. The runtime is generated from the npm dependency during each build.
• Deploy GitHub Pages: Builds and deploys dist/ whenever changes are pushed to main.

Recommended Deployment Order

1. Push the entire project to a public repository named pokedex.

2. In the GitHub repository, open Settings → Actions → General and set Workflow permissions to Read and write permissions.

3. Open Settings → Pages and set Source to GitHub Actions.

4. Open Actions → Prepare Pokédex assets → Run workflow. When the workflow finishes, it will commit public/data and public/models.

5. The asset workflow explicitly triggers Deploy GitHub Pages after committing the generated files. After deployment completes, visit:

   https://associatedprimeideal.github.io/pokedex/

Do not add npm run prepare:assets to the regular GitHub Pages deployment workflow. Reference embeddings only need to be regenerated when the model, number of Pokémon, or reference images change.

After the Action has committed the assets, sync them to a server that cannot access Hugging Face and start local development:

git pull origin main
ONNXRUNTIME_NODE_INSTALL=skip npm ci
npm run prepare:runtime
npm run check:assets
npm run dev -- --host 0.0.0.0

Downloading the Model Through a Local Proxy

First, verify that the model hosting service is accessible:

curl -I https://huggingface.co/Xenova/clip-vit-base-patch32/resolve/main/config.json

When using an HTTP/HTTPS proxy with Node.js 24:

export HTTP_PROXY=http://127.0.0.1:7890
export HTTPS_PROXY=http://127.0.0.1:7890
export NODE_USE_ENV_PROXY=1
npm run prepare:embeddings

Replace the address and port with your own proxy settings.

You can also temporarily set HF_ENDPOINT or change model.remoteHost in config/pokedex.config.json.

Phones and Laptops

GitHub Pages uses HTTPS by default, so both phone and laptop browsers can request camera permission.

Desktop browsers prefer WebGPU and fall back to WASM when WebGPU is unavailable. Users without a camera can still use the Choose Photo option.

The first scan requires downloading the quantized model. Afterward, the model will normally be reused from the browser cache.