Refactor provider registration and implement multiple services

CHANGELOG.md

@@ -8,6 +8,27 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](http://keepachangelog.com/)
 and this project adheres to [Semantic Versioning](http://semver.org/).
 
+## [6.0.0] - unreleased
+
+### Breaking changes
+
+- Providers are now registered per selected model instead of once per task type, so their IDs and names changed. Existing per-task-type provider preferences in the AI admin settings have to be set again.
+- The single service configuration was replaced by a list of connected services. The existing configuration, including the per-modality URL overrides, is migrated to services automatically.
+- The optional `model` input was removed from the providers: a provider always uses the model it was registered for.
+- The deprecated `ITranslationProvider` implementation was removed. The TextToTextTranslate providers cover translation.
+
+### Added
+
+- Connect any number of OpenAI-compatible services, each with its own URL, credentials, request behaviour and quotas
+- Select per service and per modality which models are exposed, including models the service does not list
+- Users can provide their own credentials for each connected service, which lifts that service's quotas
+
+### Changed
+
+- Quota amounts, quota rules and usage are tracked per service. Existing quota rules applied to every service, so each of them is copied to every connected service on upgrade
+- The measured processing time behind the expected runtime of a provider is now recorded per service
+- Model lists are fetched when the admin asks for them instead of being cached, so the daily model refresh job is gone
+
 ## [5.0.0] - 2026-07-27
 
 ### Breaking changes

README.md

@@ -9,12 +9,18 @@
 :warning: The smart pickers have been removed from this app
 as they are now included in the [Assistant app](https://apps.nextcloud.com/apps/assistant).
 
-This app implements:
+This app lets you connect any number of OpenAI-compatible services and pick, per service, which of their
+models you want to expose. Every selected model is registered as its own set of providers, named after the
+model, so you can offer several models side by side and choose between them in the AI admin settings.
 
-* Text generation providers: Free prompt, Summarize, Headline, Context Write, Chat, and Reformulate (using any available large language model)
-* A Translation provider (using any available language model)
-* A SpeechToText provider (using Whisper)
-* An image generation provider
+Per modality, the following providers are registered for each selected model:
+
+* Text models: Free prompt, Chat, Chat with tools, Summarize, Headline, Topics, Context Write, Reformulate,
+  Improve, Emoji, Change tone, Proofread, Reformat paragraphs and Translate (plus OCR, image analysis and
+  audio chat when the service accepts the matching attachments)
+* Image models: image generation, also with an LLM-improved prompt
+* Transcription models: transcription, subtitles and transcription with paragraph reformatting
+* Speech models: text to speech
 
 :warning: Context Write, Summarize, Headline and Reformulate have mainly been tested with OpenAI.
 They might work when connecting to other services, without any guarantee.
@@ -107,12 +113,13 @@ Learn more about the Nextcloud Ethical AI Rating [in our blog](https://nextcloud
 ### Admin settings
 
 There is an "Artificial intelligence" section in the **admin** settings where you can:
-* Choose whether you use OpenAI, a LocalAI instance or another remote service
-* Set a global API key (or basic auth credentials) for the Nextcloud instance
-* Configure default models and quota settings
+* Connect any number of services: OpenAI, LocalAI instances or any other remote service with an OpenAI-compatible API
+* Set the API key (or basic auth credentials) of each service
+* Select, per service and per modality, which models are exposed as providers
+* Configure the request behaviour, the usage quotas and the quota rules of each service, as well as the instance-wide quota period
 
 ### Personal settings
 
-There is an "Artificial intelligence" section in the **personal** settings where users can set their personal API key or basic auth credentials,
-as well as view their usage quota.
+There is an "Artificial intelligence" section in the **personal** settings where users can set their personal API key or basic auth credentials
+for each connected service, as well as view their usage quota per service. Using your own credentials for a service lifts that service's quotas.
 Users can also choose to disable the Nextcloud Assistant even if the Assistant app is installed.

appinfo/info.xml

@@ -11,12 +11,18 @@
 ⚠️ The smart pickers have been removed from this app
 as they are now included in the [Assistant app](https://apps.nextcloud.com/apps/assistant).
 
-This app implements:
+This app lets you connect any number of OpenAI-compatible services and pick, per service, which of their
+models you want to expose. Every selected model is registered as its own set of providers, named after the
+model, so you can offer several models side by side and choose between them in the AI admin settings.
 
-* Text generation providers: Free prompt, Summarize, Headline, Context Write, Chat, and Reformulate (using any available large language model)
-* A Translation provider (using any available language model)
-* A SpeechToText provider (using Whisper)
-* An image generation provider
+Per modality, the following providers are registered for each selected model:
+
+* Text models: Free prompt, Chat, Chat with tools, Summarize, Headline, Topics, Context Write, Reformulate,
+  Improve, Emoji, Change tone, Proofread, Reformat paragraphs and Translate (plus OCR, image analysis and
+  audio chat when the service accepts the matching attachments)
+* Image models: image generation, also with an LLM-improved prompt
+* Transcription models: transcription, subtitles and transcription with paragraph reformatting
+* Speech models: text to speech
 
 ⚠️ Context Write, Summarize, Headline and Reformulate have mainly been tested with OpenAI.
 They might work when connecting to other services, without any guarantee.
@@ -101,7 +107,7 @@ Negative:
 
 Learn more about the Nextcloud Ethical AI Rating [in our blog](https://nextcloud.com/blog/nextcloud-ethical-ai-rating/).
 ]]>	</description>
-	<version>5.0.0</version>
+	<version>6.0.0</version>
 	<licence>agpl</licence>
 	<author>Julien Veyssier</author>
 	<namespace>OpenAi</namespace>
@@ -121,7 +127,6 @@ Learn more about the Nextcloud Ethical AI Rating [in our blog](https://nextcloud
 	</dependencies>
 	<background-jobs>
 		<job>OCA\OpenAi\Cron\CleanupQuotaDb</job>
-		<job>OCA\OpenAi\Cron\RefreshModels</job>
 	</background-jobs>
 	<settings>
 		<admin>OCA\OpenAi\Settings\Admin</admin>

appinfo/routes.php

@@ -8,12 +8,18 @@
 return [
 	'routes' => [
 		['name' => 'config#setUserConfig', 'url' => '/config', 'verb' => 'PUT'],
-		['name' => 'config#setSensitiveUserConfig', 'url' => '/config/sensitive', 'verb' => 'PUT'],
 		['name' => 'config#setAdminConfig', 'url' => '/admin-config', 'verb' => 'PUT'],
-		['name' => 'config#setSensitiveAdminConfig', 'url' => '/admin-config/sensitive', 'verb' => 'PUT'],
-		['name' => 'config#autoDetectFeatures', 'url' => '/admin-config/auto-detect-features', 'verb' => 'POST'],
 
-		['name' => 'openAiAPI#getModels', 'url' => '/models', 'verb' => 'GET'],
+		['name' => 'service#index', 'url' => '/services', 'verb' => 'GET'],
+		['name' => 'service#create', 'url' => '/services', 'verb' => 'POST'],
+		['name' => 'service#update', 'url' => '/services/{id}', 'verb' => 'PUT'],
+		['name' => 'service#updateSensitive', 'url' => '/services/{id}/sensitive', 'verb' => 'PUT'],
+		['name' => 'service#destroy', 'url' => '/services/{id}', 'verb' => 'DELETE'],
+		['name' => 'service#models', 'url' => '/services/{id}/models', 'verb' => 'GET'],
+		['name' => 'service#autoDetectModalities', 'url' => '/services/{id}/auto-detect-modalities', 'verb' => 'POST'],
+		['name' => 'service#userCredentials', 'url' => '/services/user-credentials', 'verb' => 'GET'],
+		['name' => 'service#setUserCredentials', 'url' => '/services/{id}/user-credentials', 'verb' => 'PUT'],
+
 		['name' => 'openAiAPI#getUserQuotaInfo', 'url' => '/quota-info', 'verb' => 'GET'],
 		['name' => 'openAiAPI#getAdminQuotaInfo', 'url' => '/admin-quota-info', 'verb' => 'GET'],
 

lib/AppInfo/Application.php

@@ -8,32 +8,13 @@
 namespace OCA\OpenAi\AppInfo;
 
 use OCA\OpenAi\Capabilities;
+use OCA\OpenAi\Listener\TaskProcessingProviderListener;
 use OCA\OpenAi\Notification\Notifier;
-use OCA\OpenAi\OldProcessing\Translation\TranslationProvider as OldTranslationProvider;
-use OCA\OpenAi\TaskProcessing\AudioToAudioChatProvider;
-use OCA\OpenAi\TaskProcessing\AudioToAudioTranslateProvider;
-use OCA\OpenAi\TaskProcessing\AudioToTextEnhancedProvider;
-use OCA\OpenAi\TaskProcessing\AudioToTextProvider;
-use OCA\OpenAi\TaskProcessing\AudioToTextSubtitlesProvider;
-use OCA\OpenAi\TaskProcessing\ChangeToneProvider;
-use OCA\OpenAi\TaskProcessing\ContextWriteProvider;
-use OCA\OpenAi\TaskProcessing\EmojiProvider;
-use OCA\OpenAi\TaskProcessing\HeadlineProvider;
-use OCA\OpenAi\TaskProcessing\ReformulateProvider;
-use OCA\OpenAi\TaskProcessing\SummaryProvider;
-use OCA\OpenAi\TaskProcessing\TextToImageImprovedPromptProvider;
-use OCA\OpenAi\TaskProcessing\TextToImageProvider;
-use OCA\OpenAi\TaskProcessing\TextToSpeechProvider;
-use OCA\OpenAi\TaskProcessing\TextToTextChatProvider;
-use OCA\OpenAi\TaskProcessing\TextToTextImproveProvider;
-use OCA\OpenAi\TaskProcessing\TextToTextProvider;
-use OCA\OpenAi\TaskProcessing\TopicsProvider;
-use OCA\OpenAi\TaskProcessing\TranslateProvider;
 use OCP\AppFramework\App;
 use OCP\AppFramework\Bootstrap\IBootContext;
 use OCP\AppFramework\Bootstrap\IBootstrap;
 use OCP\AppFramework\Bootstrap\IRegistrationContext;
-use OCP\IAppConfig;
+use OCP\TaskProcessing\Events\GetTaskProcessingProvidersEvent;
 
 class Application extends App implements IBootstrap {
 	public const APP_ID = 'integration_openai';
@@ -72,6 +53,15 @@ class Application extends App implements IBootstrap {
 	public const DEFAULT_LOCALAI_IMAGE_GENERATION_TIME = 90; // seconds
 	public const EXPECTED_RUNTIME_LOWPASS_FACTOR = 0.1;
 
+	/**
+	 * Prefixes of the app config keys holding the measured processing time of
+	 * a service. The ID of the service is appended to them, so that a slow
+	 * service does not skew the runtime estimate of a fast one.
+	 */
+	public const TEXT_PROCESSING_TIME_KEY = 'text_generation_time';
+	public const IMAGE_PROCESSING_TIME_KEY = 'image_generation_time';
+	public const PROCESSING_TIME_KEYS = [self::TEXT_PROCESSING_TIME_KEY, self::IMAGE_PROCESSING_TIME_KEY];
+
 	public const QUOTA_TYPE_TEXT = 0;
 	public const QUOTA_TYPE_IMAGE = 1;
 	public const QUOTA_TYPE_TRANSCRIPTION = 2;
@@ -84,96 +74,34 @@ class Application extends App implements IBootstrap {
 		self::QUOTA_TYPE_SPEECH => 0, // 0 = unlimited
 	];
 
-	public const MODELS_CACHE_KEY = 'models';
 	public const QUOTA_RULES_CACHE_PREFIX = 'quota_rules';
-	public const MODELS_CACHE_TTL = 60 * 30;
 
 	public const LANGUAGE_CODES_AND_ENDONYMS = [['en', 'English'], ['zh', '中文'], ['de', 'Deutsch'], ['es', 'Español'], ['ru', 'Русский'], ['ko', '한국어'], ['fr', 'Français'], ['ja', '日本語'], ['pt', 'Português'], ['tr', 'Türkçe'], ['pl', 'Polski'], ['ca', 'Català'], ['nl', 'Nederlands'], ['ar', 'العربية'], ['sv', 'Svenska'], ['it', 'Italiano'], ['id', 'Bahasa Indonesia'], ['hi', 'हिन्दी'], ['fi', 'Suomi'], ['vi', 'Tiếng Việt'], ['he', 'עברית'], ['uk', 'Українська'], ['el', 'Ελληνικά'], ['ms', 'Bahasa Melayu'], ['cs', 'Česky'], ['ro', 'Română'], ['da', 'Dansk'], ['hu', 'Magyar'], ['ta', 'தமிழ்'], ['no', 'Norsk (bokmål / riksmål)'], ['th', 'ไทย / Phasa Thai'], ['ur', 'اردو'], ['hr', 'Hrvatski'], ['bg', 'Български'], ['lt', 'Lietuvių'], ['la', 'Latina'], ['mi', 'Māori'], ['ml', 'മലയാളം'], ['cy', 'Cymraeg'], ['sk', 'Slovenčina'], ['te', 'తెలుగు'], ['fa', 'فارسی'], ['lv', 'Latviešu'], ['bn', 'বাংলা'], ['sr', 'Српски'], ['az', 'Azərbaycanca / آذربايجان'], ['sl', 'Slovenščina'], ['kn', 'ಕನ್ನಡ'], ['et', 'Eesti'], ['mk', 'Македонски'], ['br', 'Brezhoneg'], ['eu', 'Euskara'], ['is', 'Íslenska'], ['hy', 'Հայերեն'], ['ne', 'नेपाली'], ['mn', 'Монгол'], ['bs', 'Bosanski'], ['kk', 'Қазақша'], ['sq', 'Shqip'], ['sw', 'Kiswahili'], ['gl', 'Galego'], ['mr', 'मराठी'], ['pa', 'ਪੰਜਾਬੀ / पंजाबी / پنجابي'], ['si', 'සිංහල'], ['km', 'ភាសាខ្មែរ'], ['sn', 'chiShona'], ['yo', 'Yorùbá'], ['so', 'Soomaaliga'], ['af', 'Afrikaans'], ['oc', 'Occitan'], ['ka', 'ქართული'], ['be', 'Беларуская'], ['tg', 'Тоҷикӣ'], ['sd', 'सिनधि'], ['gu', 'ગુજરાતી'], ['am', 'አማርኛ'], ['yi', 'ייִדיש'], ['lo', 'ລາວ / Pha xa lao'], ['uz', 'Ўзбек'], ['fo', 'Føroyskt'], ['ht', 'Krèyol ayisyen'], ['ps', 'پښتو'], ['tk', 'Туркмен / تركمن'], ['nn', 'Norsk (nynorsk)'], ['mt', 'bil-Malti'], ['sa', 'संस्कृतम्'], ['lb', 'Lëtzebuergesch'], ['my', 'Myanmasa'], ['bo', 'བོད་ཡིག / Bod skad'], ['tl', 'Tagalog'], ['mg', 'Malagasy'], ['as', 'অসমীয়া'], ['tt', 'Tatarça'], ['haw', 'ʻŌlelo Hawaiʻi'], ['ln', 'Lingála'], ['ha', 'هَوُسَ'], ['ba', 'Башҡорт'], ['jw', 'ꦧꦱꦗꦮ'], ['su', 'Basa Sunda'], ['yue', '粤语']];
 
-	public const SERVICE_TYPE_IMAGE = 'image';
-	public const SERVICE_TYPE_STT = 'stt';
-	public const SERVICE_TYPE_TTS = 'tts';
-
-	private IAppConfig $appConfig;
+	/**
+	 * The modalities the admin can select models for. Each selected model of a
+	 * modality is exposed as one task processing provider per task type of
+	 * that modality.
+	 */
+	public const MODALITY_TEXT = 'text';
+	public const MODALITY_IMAGE = 'image';
+	public const MODALITY_STT = 'stt';
+	public const MODALITY_TTS = 'tts';
+	/** App config key holding the JSON list of connected services */
+	public const SERVICES_CONFIG_KEY = 'services';
+
+	/** Sent to and accepted from the frontend in place of a stored secret */
+	public const SECRET_PLACEHOLDER = '**********';
 
 	public function __construct(array $urlParams = []) {
 		parent::__construct(self::APP_ID, $urlParams);
-
-		$container = $this->getContainer();
-		$this->appConfig = $container->get(IAppConfig::class);
 	}
 
 	public function register(IRegistrationContext $context): void {
-		// deprecated APIs
-		if ($this->appConfig->getValueString(Application::APP_ID, 'translation_provider_enabled', '1') === '1') {
-			$context->registerTranslationProvider(OldTranslationProvider::class);
-		}
-
-		$translationProviderEnabled = $this->appConfig->getValueString(Application::APP_ID, 'translation_provider_enabled', '1') === '1';
-		$sttProviderEnabled = $this->appConfig->getValueString(Application::APP_ID, 'stt_provider_enabled', '1') === '1';
-		$ttsProviderEnabled = $this->appConfig->getValueString(Application::APP_ID, 'tts_provider_enabled', '1') === '1';
-
-		// Task processing
-		if ($translationProviderEnabled) {
-			$context->registerTaskProcessingProvider(TranslateProvider::class);
-		}
-		if ($translationProviderEnabled && $sttProviderEnabled && $ttsProviderEnabled) {
-			$context->registerTaskProcessingProvider(AudioToAudioTranslateProvider::class);
-		}
-		if ($sttProviderEnabled) {
-			$context->registerTaskProcessingProvider(AudioToTextProvider::class);
-			if (class_exists('OCP\\TaskProcessing\\TaskTypes\\AudioToTextSubtitles')) {
-				$context->registerTaskProcessingProvider(AudioToTextSubtitlesProvider::class);
-			}
-			if (class_exists('OCP\\TaskProcessing\\TaskTypes\\TextToTextReformatParagraphs')) {
-				$context->registerTaskProcessingProvider(AudioToTextEnhancedProvider::class);
-			}
-		}
-
-		$serviceUrl = $this->appConfig->getValueString(Application::APP_ID, 'url');
-		$isUsingOpenAI = $serviceUrl === '' || $serviceUrl === Application::OPENAI_API_BASE_URL;
-
-		if ($this->appConfig->getValueString(Application::APP_ID, 'llm_provider_enabled', '1') === '1') {
-			$context->registerTaskProcessingProvider(TextToTextProvider::class);
-			$context->registerTaskProcessingProvider(TextToTextChatProvider::class);
-			$context->registerTaskProcessingProvider(SummaryProvider::class);
-			$context->registerTaskProcessingProvider(HeadlineProvider::class);
-			$context->registerTaskProcessingProvider(TopicsProvider::class);
-			$context->registerTaskProcessingProvider(ContextWriteProvider::class);
-			$context->registerTaskProcessingProvider(ReformulateProvider::class);
-			$context->registerTaskProcessingProvider(TextToTextImproveProvider::class);
-			$context->registerTaskProcessingProvider(EmojiProvider::class);
-			$context->registerTaskProcessingProvider(ChangeToneProvider::class);
-			$context->registerTaskProcessingProvider(\OCA\OpenAi\TaskProcessing\TextToTextChatWithToolsProvider::class);
-			$context->registerTaskProcessingProvider(\OCA\OpenAi\TaskProcessing\MultimodalChatWithToolsProvider::class);
-			$context->registerTaskProcessingProvider(\OCA\OpenAi\TaskProcessing\ProofreadProvider::class);
-			if (class_exists('OCP\\TaskProcessing\\TaskTypes\\TextToTextReformatParagraphs')) {
-				$context->registerTaskProcessingProvider(\OCA\OpenAi\TaskProcessing\ReformatParagraphsProvider::class);
-			}
-			if ($this->appConfig->getValueString(Application::APP_ID, 'multimodal_image_enabled', '1') === '1') {
-				$context->registerTaskProcessingProvider(\OCA\OpenAi\TaskProcessing\ImageToTextOcrProvider::class);
-				$context->registerTaskProcessingProvider(\OCA\OpenAi\TaskProcessing\AnalyzeImagesProvider::class);
-			}
-		}
-		$context->registerTaskProcessingProvider(TextToSpeechProvider::class);
-		if ($this->appConfig->getValueString(Application::APP_ID, 't2i_provider_enabled', '1') === '1') {
-			$context->registerTaskProcessingProvider(TextToImageProvider::class);
-			$context->registerTaskProcessingProvider(TextToImageImprovedPromptProvider::class);
-		}
-
-		// only register audio chat stuff if we're using OpenAI or stt+llm+tts are enabled
-		if (
-			$isUsingOpenAI
-			|| (
-				$this->appConfig->getValueString(Application::APP_ID, 'stt_provider_enabled', '1') === '1'
-				&& $this->appConfig->getValueString(Application::APP_ID, 'llm_provider_enabled', '1') === '1'
-				&& $this->appConfig->getValueString(Application::APP_ID, 'tts_provider_enabled', '1') === '1'
-			)
-		) {
-			if (class_exists('OCP\\TaskProcessing\\TaskTypes\\AudioToAudioChat')) {
-				$context->registerTaskProcessingProvider(AudioToAudioChatProvider::class);
-			}
-		}
+		// The task processing providers of this app depend on the admin's
+		// service and model selection, so they cannot be registered as
+		// classes. They are built per (service, model, task type) instead.
+		$context->registerEventListener(GetTaskProcessingProvidersEvent::class, TaskProcessingProviderListener::class);
 
 		$context->registerCapability(Capabilities::class);
 		$context->registerNotifierService(Notifier::class);

lib/Capabilities.php

@@ -10,19 +10,19 @@
 namespace OCA\OpenAi;
 
 use OCA\OpenAi\AppInfo\Application;
-use OCA\OpenAi\Service\OpenAiAPIService;
+use OCA\OpenAi\Service\ServicesService;
 use OCP\Capabilities\IPublicCapability;
 
 class Capabilities implements IPublicCapability {
 	public function __construct(
-		private OpenAiAPIService $openAiAPIService,
+		private ServicesService $servicesService,
 	) {
 	}
 
 	public function getCapabilities(): array {
 		return [
 			Application::APP_ID => [
-				'uses_openai' => $this->openAiAPIService->isUsingOpenAi(),
+				'uses_openai' => $this->servicesService->hasOpenAiService(),
 			],
 		];
 	}