Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/workflows/citest.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ name: citest
on:
push:
branches:
- main
- master
- "release/**"
paths-ignore:
Expand All @@ -16,6 +17,8 @@ on:
- "NOTICE"
- ".github/workflows/lint.yaml"
- ".github/workflows/publish.yaml"
- "javaclients/**"
- ".github/workflows/java-ci.yaml"

pull_request:
paths-ignore:
Expand All @@ -29,6 +32,8 @@ on:
- "NOTICE"
- ".github/workflows/lint.yaml"
- ".github/workflows/publish.yaml"
- "javaclients/**"
- ".github/workflows/java-ci.yaml"

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
Expand Down
5 changes: 5 additions & 0 deletions .github/workflows/citest_npu.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ name: citest-npu
on:
push:
branches:
- main
- master
- "release/**"
paths-ignore:
Expand All @@ -17,6 +18,8 @@ on:
- "NOTICE"
- ".github/workflows/lint.yaml"
- ".github/workflows/publish.yaml"
- "javaclients/**"
- ".github/workflows/java-ci.yaml"

pull_request:
paths-ignore:
Expand All @@ -31,6 +34,8 @@ on:
- "NOTICE"
- ".github/workflows/lint.yaml"
- ".github/workflows/publish.yaml"
- "javaclients/**"
- ".github/workflows/java-ci.yaml"

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
Expand Down
36 changes: 36 additions & 0 deletions .github/workflows/java-ci.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
name: Java Client CI

on:
push:
paths:
- "javaclients/**"
- ".github/workflows/java-ci.yaml"
pull_request:
paths:
- "javaclients/**"
- ".github/workflows/java-ci.yaml"
workflow_dispatch:

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
verify:
name: Maven 构建与测试
runs-on: ubuntu-latest

steps:
- name: 拉取代码
uses: actions/checkout@v4

- name: 配置 JDK 17
uses: actions/setup-java@v4
with:
distribution: temurin
java-version: "17"
cache: maven
cache-dependency-path: javaclients/pom.xml

- name: Maven 构建与测试
run: mvn -B -f javaclients/pom.xml verify
10 changes: 9 additions & 1 deletion .github/workflows/lint.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,14 @@
name: Lint test

on: [push, pull_request]
on:
push:
paths-ignore:
- "javaclients/**"
- ".github/workflows/java-ci.yaml"
pull_request:
paths-ignore:
- "javaclients/**"
- ".github/workflows/java-ci.yaml"

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
Expand Down
19 changes: 19 additions & 0 deletions javaclients/.editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
root = true

[*]
charset = utf-8
end_of_line = lf
insert_final_newline = true
trim_trailing_whitespace = true

[*.java]
indent_style = space
indent_size = 4
ij_java_right_margin = 100
ij_java_use_single_class_import = true
ij_java_class_count_to_use_import_on_demand = 999
ij_java_names_count_to_use_import_on_demand = 999

[*.{xml,yml,yaml,md}]
indent_style = space
indent_size = 2
6 changes: 6 additions & 0 deletions javaclients/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
target/
.idea/
*.iml
.DS_Store
.env
*.log
10 changes: 10 additions & 0 deletions javaclients/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# 更新日志

本项目遵循语义化版本规范。

## 1.0.0-SNAPSHOT

- 新增 Java 17 的 Twinkle HTTP 客户端基础实现。
- 新增会话、心跳、模型训练、采样、训练任务、数据集和远程处理器 API。
- 与 Python `twinkle_client_simple` 对齐 `/twinkle` 管理路由、LoRA 和 DatasetMeta 的核心序列化协议。
- 新增 MockWebServer 协议契约测试与环境变量驱动的服务联调示例。
7 changes: 7 additions & 0 deletions javaclients/CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# 贡献指南

欢迎提交 Issue 和 Pull Request。提交前请使用 IntelliJ IDEA 或本地 Maven 完成编译检查,并保持公开文档、JavaDoc、示例说明和源代码注释为中文。

提交内容不得包含令牌、账号、内网地址、用户数据、本地绝对路径或 IDE 生成文件。API 标识符和协议字段使用英文,以维持 Java 和服务端协议兼容性。

建议的提交格式为:`feat:`、`fix:`、`docs:`、`build:` 或 `chore:`。
63 changes: 63 additions & 0 deletions javaclients/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/

TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION

1. Definitions.

"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.

"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.

"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.

2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.

3. Grant of Patent License. Subject to the terms and conditions of this
License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work.

4. Redistribution. You may reproduce and distribute copies of the Work
or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You meet
the following conditions: You must give any other recipients of the
Work or Derivative Works a copy of this License; cause modified files
to carry prominent notices; retain all copyright, patent, trademark,
and attribution notices; and include a readable copy of attribution
notices from any NOTICE file.

5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
shall be under the terms of this License, without additional terms.

6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor.

7. Disclaimer of Warranty. Unless required by applicable law or agreed
to in writing, Licensor provides the Work on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

8. Limitation of Liability. In no event and under no legal theory,
whether in tort, contract, or otherwise, shall any Contributor be
liable to You for damages arising as a result of this License or use
of the Work.

9. Accepting Warranty or Additional Liability. While redistributing the
Work, You may choose to offer support, warranty, indemnity, or other
liability obligations consistent with this License.
82 changes: 82 additions & 0 deletions javaclients/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# Twinkle Java Client

Twinkle 训练服务的 Java 17 同步客户端。该项目面向国内开发者发布:使用中文文档、中文 JavaDoc 与中文源代码注释;Java API 名称和 HTTP 协议字段保持英文,以便与 Java 生态及 Twinkle 服务端兼容。

## 特性

- 自动创建并维护服务端会话心跳,客户端关闭时自动停止。
- 提供模型训练、LoRA、采样、训练任务、检查点、数据集、DataLoader 与输入处理器 API。
- 所有网络失败和服务端失败均转换为携带上下文的运行时异常。
- 通过 `TWINKLE_SERVER_URL` 与 `TWINKLE_SERVER_TOKEN` 读取默认服务地址和认证令牌。

## 引入依赖

发布到 Maven Central 后,可在项目中引入:

```xml
<dependency>
<groupId>io.github.modelscope</groupId>
<artifactId>twinkle-client-java</artifactId>
<version>1.0.0</version>
</dependency>
```

在发布前,可直接将本项目导入 IntelliJ IDEA 作为 Maven 项目。

## 最小示例

```java
try (TwinkleClient client = TwinkleClient.builder()
.baseUrl(System.getenv("TWINKLE_SERVER_URL"))
.apiKey(System.getenv("TWINKLE_SERVER_TOKEN"))
.build()) {
if (!client.healthCheck()) {
throw new IllegalStateException("Twinkle 服务不可用");
}

var model = client.models().open("Qwen/Qwen3.6-27B");
model.addAdapter("default", new LoraConfig(8, 16, "all-linear", 0.01, "none", null));
model.setLoss("CrossEntropyLoss");
model.setOptimizer("Adam", Map.of("lr", 1e-4));
}
```

## 数据加载与训练

```java
var dataset = client.processors().dataset(
DatasetKind.DATASET,
Map.of("dataset_meta", DatasetMeta.of("ms://your-dataset")));
dataset.setTemplate("Qwen3_5Template", Map.of("model_id", "Qwen/Qwen3.6-27B"));
dataset.encode(false, Map.of("batched", true));

var loader = client.processors().dataLoader(dataset.processorId(), Map.of("batch_size", 4));
for (var batch : loader) {
model.forwardBackward(batch);
model.clipGradAndStep(1.0, 2);
}
```

## 配置

| 配置项 | 默认值 | 说明 |
| --- | --- | --- |
| `TWINKLE_SERVER_URL` | `http://127.0.0.1:8000` | 服务根地址;客户端自动补充 `/api/v1`。 |
| `TWINKLE_SERVER_TOKEN` | `EMPTY_TOKEN` | 服务端认证令牌。 |
| `routePrefix` | `/twinkle` | 会话和训练任务管理 API 的路由前缀。 |

请勿将真实令牌、内网地址、数据集本地路径写入源码、Issue 或提交历史。

## 与旧版原型的迁移

| 原型 API | 新 API |
| --- | --- |
| `new TwinkleClient(url, token)` | `TwinkleClient.builder().baseUrl(url).apiKey(token).build()` |
| `createModel(id)` | `client.models().open(id)` |
| `createSampler(id)` | `client.samplers().open(id)` |
| `createDataset(type, args)` | `client.processors().dataset(type, args)` |
| `Map<String, Object>` 响应 | 稳定字段使用 record,开放字段使用 `JsonObject` / `JsonElement`。 |

## 许可证

本项目采用 [Apache License 2.0](LICENSE)。
5 changes: 5 additions & 0 deletions javaclients/SECURITY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# 安全说明

请不要在公开 Issue、Pull Request、日志或示例中提交 API Token、密码、内网地址和真实训练数据。

发现安全问题时,请使用目标 GitHub 仓库的 **Private Security Advisory** 功能进行私密报告,并说明受影响版本、复现条件和风险范围。
65 changes: 65 additions & 0 deletions javaclients/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>io.github.modelscope</groupId>
<artifactId>twinkle-client-java</artifactId>
<version>1.0.0-SNAPSHOT</version>
<name>Twinkle Java Client</name>
<description>Twinkle 训练服务的 Java 17 客户端</description>
<url>https://github.com/modelscope/twinkle-client-java</url>
<licenses>
<license>
<name>Apache License 2.0</name>
<url>https://www.apache.org/licenses/LICENSE-2.0.txt</url>
</license>
</licenses>
<properties>
<maven.compiler.release>17</maven.compiler.release>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<okhttp.version>4.12.0</okhttp.version>
<gson.version>2.10.1</gson.version>
<junit.version>5.10.2</junit.version>
<mockwebserver.version>4.12.0</mockwebserver.version>
</properties>
<dependencies>
<dependency>
<groupId>com.squareup.okhttp3</groupId>
<artifactId>okhttp</artifactId>
<version>${okhttp.version}</version>
</dependency>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>${gson.version}</version>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
<version>${junit.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.squareup.okhttp3</groupId>
<artifactId>mockwebserver</artifactId>
<version>${mockwebserver.version}</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.13.0</version>
<configuration>
<release>17</release>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>3.2.5</version>
</plugin>
</plugins>
</build>
</project>
Loading
Loading