Use of ABS Api

[Vito0912]

DISCLAIMER: I used the latest available stable APK release.

Hello,

I am a contributor of ABS (but not maintainer!) and check third party clients for their API usage as in history there were many bugs with stats, playback, permissions etc. which caused loss of e.g. stats, and caused requests and issues on the ABS Discord and GitHub. I try to make this for every new (or app I add to my collection of apps) app I see and try to give them some recommendations of the API, as the API docs are not maintained at the moment, so often it's not even knowable what is right. Most of the apps I send this issue are new, so don't take this as bad critisim, but a hint.
I also started a "database" to collect all clients, list their features and rate their API usage. As I like ABS myself, built a client, SDK and OpenAPI spec myself, I want to give a detailed feedback on what an app can do better in terms of Usage with ABS. All of this ofc. is just a recommendation

This report only represents a short testing period on a production server that has some usage, so bugs may have occurred that did not exist on smaller or newer server setups.

Note

This does not mean you have to implement that this way, this is just note that it was intended that way so either these endpoints become deprecated in the future or problems can occur with using these methods. You can ofc also leave it the way it is currently.
This should be NOT a bad criticism. As said I try to give feedback in that matter to any app.

Important

This is the first watch app I am testing. Although I have an Android watch, I only tested it in the emulator. Unfortunately, I do not have a setup to monitor the payloads the apps send, and I do not want to dig into the code, so I can only give estimates about why things fail.

Old authentication method

You are still using the old authentication method. The old authentication method you are using now (or at least the token you use) will be removed soon.

Playing a file

With the newer ABS versions there is a more secure way of playing files that don't need the token to be transmitted to request the files. You can check the web client or offical apps to see how it's done. The old api endpoint likely will not be removed any time soon, but the new one is more secure due to not needing to transmit the token. This needs the use of sessions. See the Progress section

Progress

I never got it to sync progress while playing. If I switched to another book or changed chapters from the view I saw, the app made a request, but it did not when just listening, which could cause a lot of progress loss if the app closed incorrectly.
Even when doing a progress sync, the books progress never updated or had any progress attached, so the book never showed any updated progress.

Even if, the /progress endpoint is the wrong one for updating progress. /progress should only be used for a one-time, static update (such as marking a book as finished from the UI). It does not send progress updates to clients when used through that endpoint, leading to wrong behavior with the official apps and any app using websockets.

You also never use sessions (I know it's a watch app and much harder). Without sessions, the user will have no stats generated by your app. All stats (daily listening, Year in Review, etc.) are calculated from sessions. By not using sessions, this data does not get recorded.
Please use the /play endpoint when connected to the internet and ONLY use local sessions after being online and reconnecting, but only after checking that the last edited date of the session is correct so you do not overwrite progress. The last edited date should be the actual last edit, not the last time online.

Requests

You make many requests to /api/items/?expanded=1&include=progress, which results in a 404. I also cannot mark a book as finished, failing with the same problem, probably because you are fetching it too early.

Downloads

They did not work at all for me. It instantly seems to fail without a message (icon resets). But I see that a request is being made to the server. But please use the /download permission. Without that even users without download permission could download (altough it generally did not work for me)

Usage of items

You request the whole library without pagination. This could leads to high RAM usage for the watch app I would imagine, but also stress the server and slow down requests. Not to say that it creates high data useage for the end user.

I hope this helps you building a great and awesome ABS client!

[Andead2]

Hi @Vito0912!

Thank you for taking the time to look into the app. I am also just a contributor to this app and I might work on some of the issues.

I have a few clarifying questions regarding your report:

Old authentication method

Is therre a timeline for the deprecation of the current authentication scheme? If you have more information about what exactly needs to be implemented, that would be extremely helpful. Is there some documentation on the new authentication method?

Unrelated to the issue you're describing but this is a problem with authentication that we currently have: If the user has a sufficiently secure password, it's quite cumbersome to input it on the watch. Are there any alternative authentication methods that could be better suited? An example I have in mind:

• Some apps (e.g. Nextcloud, Github, Youtube, ...) have a "Device Code authentication flow". The app retrieves a code and a secret from the backend, supplies the code to the user using a QR code, clickable link or just a code you can enter. Once the user logs in using the browser and accepts the code, the app can fetch a token for further authentication.

Playing a file

The security implications of sending a token vs. using a session (how?) are not clear immediately. I believe this would also require some further information on how to implement. This currently seems like a less pressing issue.

Progress

The progress syncing was recently improved in #12, but this should already be included in the latest APK release. The app should sync progress every 30 seconds during playback and that seemed to work for me. How exactly are you testing this? Are there some debug logs on the server that could be viewed or do you use something like mitmproxy?

I will also look into using sessions, this sounds like it requires significant code changes. Is there any documentation on the feature or specific code in the official app that we should read?

Requests

I don't quite understand the issue. Is the app doing too many requests or bad requests?

Downloads

This is interesting. The feature works for me in the emulator and on my watch. Does the server send an error when requesting the download? Do you see any errors in the logcat tab of the emulator (assuming you use Android Studio)? Is there a reference server that can be provided to further test this?

Usage of items

Right, this is definitely an issue and probably makes the app unusable for users with large libraries. We should work on this.

To summarise, these are the issues I understood:

1. Migrate off deprecated authentication

• Implement the new ABS authentication scheme (confirm endpoint/flow with maintainers).
• If supported by ABS, add device-code / QR login for better watch UX.

2. Adopt session-based playback

• Use /play to create/update sessions

3. Fix progress syncing

• Stop using /progress for live updates.
• Persist lastEdited and reconcile sessions safely when reconnecting.
• Make sure that session updates indeed occur every ~30s or on playback state changes (pause/stop/seek).
• Verify progress persistence and correct syncing with server/official app.

4. Eliminate 404s and bad requests

• Fix invalid calls such as GET /api/items/?expanded=1&include=progress.
• Correct "mark as finished" behavior.
• Surface errors in UI.

5. Pagination & incremental library sync

• Avoid fetching the entire library.
• Implement paged requests

Please let me know if I missed something.

[Vito0912]

Is there some documentation on the new authentication method?

Yes, there is a documentation thread on GitHub and every release has a note at the top

Is therre a timeline for the deprecation of the current authentication scheme?

I am just a contributor to ABS, so idk. But I assume not very long anymore as it was deprecated for months now.

Unrelated to the issue you're describing but this is a problem with authentication that we currently have: If the user has a sufficiently secure password, it's quite cumbersome to input it on the watch. Are there any alternative authentication methods that could be better suited? An example I have in mind:

No, there maybe will be in the future trough OIDC providers as these have a proper implementation.

The security implications of sending a token vs. using a session (how?) are not clear immediately. I believe this would also require some further information on how to implement. This currently seems like a less pressing issue.

I agree it's not that important, but some clients and media player (idk if you do) (require to) send the token as the query parameter which is in logs then. Thus using sessions is more secure as you can create one with the otken in the header before).
But if you use sessions this should be a little change. To use sessions on the other hand should be a very high priority imho.

The progress syncing was recently improved in #12, but this should already be included in the latest APK release. The app should sync progress every 30 seconds during playback and that seemed to work for me. How exactly are you testing this? Are there some debug logs on the server that could be viewed or do you use something like mitmproxy?

I just looked again and I tested the latest apk marked as stable latest. I did not test the prerelease. Maybe it was fixed there. I do normally use a proxy, but I couldn't get it work with the watch emulator as described above. I thus can just say that the app did not contact my server to sync (but I cannot say if it tries. But there is nothing that should block and other requests also did not fail)

I don't quite understand the issue. Is the app doing too many requests or bad requests?

No, you are doing requests that are made without an id. This results in a 404. Many use fail2ban which could lead to bans for users.

This is interesting. The feature works for me in the emulator and on my watch. Does the server send an error when requesting the download? Do you see any errors in the logcat tab of the emulator (assuming you use Android Studio)? Is there a reference server that can be provided to further test this?

I did use the demo server of abs to test iirc. I did not capture any logs (I test multiple apps a day if I test, so sadly I do not log everything). I can try to reproduce this later and then answer the results (but see comment about used apk). Maybe its fixed.
The app tried to download the file (I saw the logs) so I assume some internal crash.

[Andead2]

Thank you for clarification. Yes indeed, v1.12 has the sync improvements. So hopefully at least that is fixed already.

I just found the demo server you provide (https://audiobooks.dev/) and a quick test showed that the download feature works for me:

I also found the extensive documentation on the new authentication: advplyr/audiobookshelf#4460 so I guess we can start working on it.

The old authentication system will continue to function and be removed from the server no earlier than September 30, 2025 to allow 3rd party apps time to update

I do hope we have a few more weeks though!