Skip to content

docs: add package documentation and 64-bit requirement #6

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
hayabusa-cloud merged 3 commits into from
Jan 1, 2026
Merged

docs: add package documentation and 64-bit requirement #6

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
f70c343
docs: add package-level documentation
hayabusa-cloud Jan 1, 2026
e104183
docs: improve GoDocs for types
hayabusa-cloud Jan 1, 2026
2a1ac78
docs: update READMEs with 64-bit requirement
hayabusa-cloud Jan 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 24 additions & 9 deletions README.es.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,13 @@ Pools de buffers acotados sin bloqueos y eficientes en memoria para Go, optimiza
- **Generación de IoVec sin copia** para llamadas al sistema de I/O vectorizado.
- **Retroceso cooperativo**: Usa `iox.Backoff` para manejar el agotamiento de recursos con elegancia.

## Requisitos

- **Go 1.25+**
- **CPU de 64 bits** (amd64, arm64, riscv64, loong64, ppc64, s390x, mips64, etc.)
Comment thread
hayabusa-cloud marked this conversation as resolved.

> **Nota:** Las arquitecturas de 32 bits no son compatibles debido a las operaciones atómicas de 64 bits en la implementación del pool sin bloqueos.

## Instalación

```bash
Expand Down Expand Up @@ -79,18 +86,22 @@ addr, n := iobuf.IoVecAddrLen(iovecs)

## Niveles de Buffer

Progresión de potencias de 4, comenzando en 16 bytes:
Progresión de potencias de 4, comenzando en 32 bytes (12 niveles, 32 B a 128 MiB):

| Nivel | Tamaño | Caso de Uso |
|-------|--------|-------------|
| Pico | 16 B | Metadatos pequeños, flags |
| Nano | 64 B | Cabeceras pequeñas, tokens |
| Micro | 256 B | Cabeceras de protocolo |
| Small | 1 KiB | Mensajes pequeños |
| Medium | 4 KiB | I/O de tamaño de página |
| Large | 16 KiB | Transferencias grandes |
| Huge | 64 KiB | UDP máximo |
| Giant | 256 KiB | I/O masivo, cargas grandes |
| Pico | 32 B | UUIDs, flags, mensajes de control pequeños |
| Nano | 128 B | Cabeceras HTTP, tokens JSON, payloads RPC pequeños |
| Micro | 512 B | Paquetes DNS, mensajes MQTT, tramas de protocolo |
| Small | 2 KiB | Frames WebSocket, respuestas HTTP pequeñas |
| Medium | 8 KiB | Segmentos TCP, mensajes gRPC, I/O de página |
| Big | 32 KiB | Registros TLS (máx 16 KiB), chunks de stream |
| Large | 128 KiB | Buffer rings io_uring, transferencias de red masivas |
| Great | 512 KiB | Páginas de base de datos, respuestas API grandes |
| Huge | 2 MiB | Alineado a huge pages, archivos mapeados en memoria |
| Vast | 8 MiB | Procesamiento de imágenes, archivos comprimidos |
| Giant | 32 MiB | Frames de video, pesos de modelos ML |
| Titan | 128 MiB | Datasets grandes, buffer máximo seguro para stack |

## Resumen de API

Expand Down Expand Up @@ -119,9 +130,13 @@ func NewNanoBufferPool(capacity int) *NanoBufferBoundedPool
func NewMicroBufferPool(capacity int) *MicroBufferBoundedPool
func NewSmallBufferPool(capacity int) *SmallBufferBoundedPool
func NewMediumBufferPool(capacity int) *MediumBufferBoundedPool
func NewBigBufferPool(capacity int) *BigBufferBoundedPool
func NewLargeBufferPool(capacity int) *LargeBufferBoundedPool
func NewGreatBufferPool(capacity int) *GreatBufferBoundedPool
func NewHugeBufferPool(capacity int) *HugeBufferBoundedPool
func NewVastBufferPool(capacity int) *VastBufferBoundedPool
func NewGiantBufferPool(capacity int) *GiantBufferBoundedPool
func NewTitanBufferPool(capacity int) *TitanBufferBoundedPool
```

### Asignación de Memoria
Expand Down
33 changes: 24 additions & 9 deletions README.fr.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,13 @@ Pools de buffers bornés sans verrou et économes en mémoire pour Go, optimisé
- **Génération IoVec sans copie** pour les appels système d'I/O vectorisées.
- **Recul coopératif** : Utilise `iox.Backoff` pour gérer l'épuisement des ressources avec élégance.

## Prérequis

- **Go 1.25+**
- **CPU 64 bits** (amd64, arm64, riscv64, loong64, ppc64, s390x, mips64, etc.)
Comment thread
hayabusa-cloud marked this conversation as resolved.

> **Note :** Les architectures 32 bits ne sont pas prises en charge en raison des opérations atomiques 64 bits dans l'implémentation du pool sans verrou.

## Installation

```bash
Expand Down Expand Up @@ -79,18 +86,22 @@ addr, n := iobuf.IoVecAddrLen(iovecs)

## Niveaux de Buffer

Progression en puissances de 4, à partir de 16 octets :
Progression en puissances de 4, à partir de 32 octets (12 niveaux, 32 o à 128 Mio) :

| Niveau | Taille | Cas d'Usage |
|--------|--------|-------------|
| Pico | 16 o | Petites métadonnées, drapeaux |
| Nano | 64 o | Petits en-têtes, jetons |
| Micro | 256 o | En-têtes de protocole |
| Small | 1 Kio | Petits messages |
| Medium | 4 Kio | I/O de taille de page |
| Large | 16 Kio | Grands transferts |
| Huge | 64 Kio | UDP maximum |
| Giant | 256 Kio | I/O en masse, grandes charges |
| Pico | 32 o | UUIDs, drapeaux, petits messages de contrôle |
| Nano | 128 o | En-têtes HTTP, jetons JSON, petits payloads RPC |
| Micro | 512 o | Paquets DNS, messages MQTT, trames de protocole |
| Small | 2 Kio | Frames WebSocket, petites réponses HTTP |
| Medium | 8 Kio | Segments TCP, messages gRPC, I/O de page |
| Big | 32 Kio | Enregistrements TLS (max 16 Kio), chunks de flux |
| Large | 128 Kio | Anneaux de tampon io_uring, transferts réseau massifs |
| Great | 512 Kio | Pages de base de données, grandes réponses API |
| Huge | 2 Mio | Aligné sur huge pages, fichiers mappés en mémoire |
| Vast | 8 Mio | Traitement d'images, archives compressées |
| Giant | 32 Mio | Frames vidéo, poids de modèles ML |
| Titan | 128 Mio | Grands ensembles de données, buffer max sûr pour pile |

## Aperçu de l'API

Expand Down Expand Up @@ -119,9 +130,13 @@ func NewNanoBufferPool(capacity int) *NanoBufferBoundedPool
func NewMicroBufferPool(capacity int) *MicroBufferBoundedPool
func NewSmallBufferPool(capacity int) *SmallBufferBoundedPool
func NewMediumBufferPool(capacity int) *MediumBufferBoundedPool
func NewBigBufferPool(capacity int) *BigBufferBoundedPool
func NewLargeBufferPool(capacity int) *LargeBufferBoundedPool
func NewGreatBufferPool(capacity int) *GreatBufferBoundedPool
func NewHugeBufferPool(capacity int) *HugeBufferBoundedPool
func NewVastBufferPool(capacity int) *VastBufferBoundedPool
func NewGiantBufferPool(capacity int) *GiantBufferBoundedPool
func NewTitanBufferPool(capacity int) *TitanBufferBoundedPool
```

### Allocation Mémoire
Expand Down
33 changes: 24 additions & 9 deletions README.ja.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,13 @@
- **ゼロコピーIoVec生成**：ベクトル化I/Oシステムコール用。
- **協調的バックオフ**：`iox.Backoff` を使用してリソース枯渇を優雅に処理。

## システム要件

- **Go 1.25+**
- **64ビットCPU**（amd64、arm64、riscv64、loong64、ppc64、s390x、mips64など）
Comment thread
hayabusa-cloud marked this conversation as resolved.

> **注意：** ロックフリープール実装で64ビットアトミック操作を使用しているため、32ビットアーキテクチャはサポートされていません。

## インストール

```bash
Expand Down Expand Up @@ -79,18 +86,22 @@ addr, n := iobuf.IoVecAddrLen(iovecs)

## バッファ階層

16バイトから始まる4の累乗で増加：
32バイトから始まる4の累乗で増加（12階層、32 B から 128 MiB）：

| 階層 | サイズ | 用途 |
|------|--------|------|
| Pico | 16 B | 小さなメタデータ、フラグ |
| Nano | 64 B | 小さなヘッダ、トークン |
| Micro | 256 B | プロトコルヘッダ |
| Small | 1 KiB | 小さなメッセージ |
| Medium | 4 KiB | ページサイズI/O |
| Large | 16 KiB | 大きな転送 |
| Huge | 64 KiB | 最大UDP |
| Giant | 256 KiB | バルクI/O、大きなペイロード |
| Pico | 32 B | UUID、フラグ、小さな制御メッセージ |
| Nano | 128 B | HTTPヘッダ、JSONトークン、小さなRPCペイロード |
| Micro | 512 B | DNSパケット、MQTTメッセージ、プロトコルフレーム |
| Small | 2 KiB | WebSocketフレーム、小さなHTTPレスポンス |
| Medium | 8 KiB | TCPセグメント、gRPCメッセージ、ページI/O |
| Big | 32 KiB | TLSレコード（最大16 KiB）、ストリームチャンク |
| Large | 128 KiB | io_uringバッファリング、バルクネットワーク転送 |
| Great | 512 KiB | データベースページ、大規模APIレスポンス |
| Huge | 2 MiB | ヒュージページ整列、メモリマップファイル |
| Vast | 8 MiB | 画像処理、圧縮アーカイブ |
| Giant | 32 MiB | ビデオフレーム、MLモデル重み |
| Titan | 128 MiB | 大規模データセット、最大スタック安全バッファ |

## API概要

Expand Down Expand Up @@ -119,9 +130,13 @@ func NewNanoBufferPool(capacity int) *NanoBufferBoundedPool
func NewMicroBufferPool(capacity int) *MicroBufferBoundedPool
func NewSmallBufferPool(capacity int) *SmallBufferBoundedPool
func NewMediumBufferPool(capacity int) *MediumBufferBoundedPool
func NewBigBufferPool(capacity int) *BigBufferBoundedPool
func NewLargeBufferPool(capacity int) *LargeBufferBoundedPool
func NewGreatBufferPool(capacity int) *GreatBufferBoundedPool
func NewHugeBufferPool(capacity int) *HugeBufferBoundedPool
func NewVastBufferPool(capacity int) *VastBufferBoundedPool
func NewGiantBufferPool(capacity int) *GiantBufferBoundedPool
func NewTitanBufferPool(capacity int) *TitanBufferBoundedPool
```

### メモリ割り当て
Expand Down
41 changes: 31 additions & 10 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,13 @@ English | [简体中文](README.zh-CN.md) | [Español](README.es.md) | [日本
- **Zero-copy IoVec generation** for vectored I/O syscalls.
- **Cooperative back-off**: Uses `iox.Backoff` to handle resource exhaustion gracefully.

## Requirements

- **Go 1.25+**
- **64-bit CPU** (amd64, arm64, riscv64, loong64, ppc64, s390x, mips64, etc.)
Comment thread
hayabusa-cloud marked this conversation as resolved.

> **Note:** 32-bit architectures are not supported due to 64-bit atomic operations in the lock-free pool implementation.

## Installation

```bash
Expand All @@ -35,7 +42,7 @@ go get code.hybscloud.com/iobuf
### Buffer Pools

```go
// Create a pool of 1024 small buffers (1 KiB each)
// Create a pool of 1024 small buffers (2 KiB each)
pool := iobuf.NewSmallBufferPool(1024)
pool.Fill(iobuf.NewSmallBuffer)

Expand Down Expand Up @@ -79,18 +86,22 @@ addr, n := iobuf.IoVecAddrLen(iovecs)

## Buffer Tiers

Power-of-4 progression starting at 16 bytes:
Power-of-4 progression starting at 32 bytes (12 tiers, 32 B to 128 MiB):

| Tier | Size | Use Case |
|------|------|----------|
| Pico | 16 B | Tiny metadata, flags |
| Nano | 64 B | Small headers, tokens |
| Micro | 256 B | Protocol headers |
| Small | 1 KiB | Small messages |
| Medium | 4 KiB | Page-sized I/O |
| Large | 16 KiB | Large transfers |
| Huge | 64 KiB | Maximum UDP |
| Giant | 256 KiB | Bulk I/O, large payloads |
| Pico | 32 B | UUIDs, flags, tiny control messages |
| Nano | 128 B | HTTP headers, JSON tokens, small RPC payloads |
| Micro | 512 B | DNS packets, MQTT messages, protocol frames |
| Small | 2 KiB | WebSocket frames, small HTTP responses |
| Medium | 8 KiB | TCP segments, gRPC messages, page I/O |
| Big | 32 KiB | TLS records (16 KiB max), stream chunks |
| Large | 128 KiB | io_uring buffer rings, bulk network transfers |
| Great | 512 KiB | Database pages, large API responses |
| Huge | 2 MiB | Huge page aligned, memory-mapped files |
| Vast | 8 MiB | Image processing, compressed archives |
| Giant | 32 MiB | Video frames, ML model weights |
| Titan | 128 MiB | Large datasets, maximum stack-safe buffer |

## API Overview

Expand Down Expand Up @@ -119,17 +130,27 @@ func NewNanoBufferPool(capacity int) *NanoBufferBoundedPool
func NewMicroBufferPool(capacity int) *MicroBufferBoundedPool
func NewSmallBufferPool(capacity int) *SmallBufferBoundedPool
func NewMediumBufferPool(capacity int) *MediumBufferBoundedPool
func NewBigBufferPool(capacity int) *BigBufferBoundedPool
func NewLargeBufferPool(capacity int) *LargeBufferBoundedPool
func NewGreatBufferPool(capacity int) *GreatBufferBoundedPool
func NewHugeBufferPool(capacity int) *HugeBufferBoundedPool
func NewVastBufferPool(capacity int) *VastBufferBoundedPool
func NewGiantBufferPool(capacity int) *GiantBufferBoundedPool
func NewTitanBufferPool(capacity int) *TitanBufferBoundedPool
```

### Memory Allocation

```go
// Page-aligned memory
func AlignedMem(size int, pageSize uintptr) []byte
func AlignedMemBlocks(n int, pageSize uintptr) [][]byte
func AlignedMemBlock() []byte

// Cache-line-aligned memory (prevents false sharing)
func CacheLineAlignedMem(size int) []byte
func CacheLineAlignedMemBlocks(n int, blockSize int) [][]byte
const CacheLineSize // 64 or 128 depending on architecture
```

### IoVec Generation
Expand Down
33 changes: 24 additions & 9 deletions README.zh-CN.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,13 @@
- **零拷贝 IoVec 生成**：用于向量化 I/O 系统调用。
- **协作式退避**：使用 `iox.Backoff` 优雅处理资源耗尽。

## 系统要求

- **Go 1.25+**
- **64 位 CPU**（amd64、arm64、riscv64、loong64、ppc64、s390x、mips64 等）
Comment thread
hayabusa-cloud marked this conversation as resolved.

> **注意：** 由于无锁池实现中使用 64 位原子操作，不支持 32 位架构。

## 安装

```bash
Expand Down Expand Up @@ -79,18 +86,22 @@ addr, n := iobuf.IoVecAddrLen(iovecs)

## 缓冲区层级

4 的幂次递增，从 16 字节开始：
4 的幂次递增，从 32 字节开始（12 层，32 B 到 128 MiB）：

| 层级 | 大小 | 用途 |
|------|------|------|
| Pico | 16 B | 微型元数据、标志位 |
| Nano | 64 B | 小型头部、令牌 |
| Micro | 256 B | 协议头部 |
| Small | 1 KiB | 小型消息 |
| Medium | 4 KiB | 页大小 I/O |
| Large | 16 KiB | 大型传输 |
| Huge | 64 KiB | 最大 UDP |
| Giant | 256 KiB | 批量 I/O、大型负载 |
| Pico | 32 B | UUID、标志、微型控制消息 |
| Nano | 128 B | HTTP 头部、JSON 令牌、小型 RPC 载荷 |
| Micro | 512 B | DNS 数据包、MQTT 消息、协议帧 |
| Small | 2 KiB | WebSocket 帧、小型 HTTP 响应 |
| Medium | 8 KiB | TCP 分段、gRPC 消息、页 I/O |
| Big | 32 KiB | TLS 记录（最大 16 KiB）、流块 |
| Large | 128 KiB | io_uring 缓冲环、批量网络传输 |
| Great | 512 KiB | 数据库页、大型 API 响应 |
| Huge | 2 MiB | 大页对齐、内存映射文件 |
| Vast | 8 MiB | 图像处理、压缩归档 |
| Giant | 32 MiB | 视频帧、机器学习模型权重 |
| Titan | 128 MiB | 大型数据集、最大栈安全缓冲区 |

## API 概览

Expand Down Expand Up @@ -119,9 +130,13 @@ func NewNanoBufferPool(capacity int) *NanoBufferBoundedPool
func NewMicroBufferPool(capacity int) *MicroBufferBoundedPool
func NewSmallBufferPool(capacity int) *SmallBufferBoundedPool
func NewMediumBufferPool(capacity int) *MediumBufferBoundedPool
func NewBigBufferPool(capacity int) *BigBufferBoundedPool
func NewLargeBufferPool(capacity int) *LargeBufferBoundedPool
func NewGreatBufferPool(capacity int) *GreatBufferBoundedPool
func NewHugeBufferPool(capacity int) *HugeBufferBoundedPool
func NewVastBufferPool(capacity int) *VastBufferBoundedPool
func NewGiantBufferPool(capacity int) *GiantBufferBoundedPool
func NewTitanBufferPool(capacity int) *TitanBufferBoundedPool
```

### 内存分配
Expand Down
Loading