Skip to content

Enhance timeout check to avoid false positives during processing [of video download to IIAB] - #170

Merged
holta merged 8 commits into
iiab:masterfrom
deldesir:deldesir-patch-34
Jun 4, 2024
Merged

Enhance timeout check to avoid false positives during processing [of video download to IIAB]#170
holta merged 8 commits into
iiab:masterfrom
deldesir:deldesir-patch-34

Conversation

@deldesir

@deldesir deldesir commented Jun 4, 2024

Copy link
Copy Markdown
Member

This PR fixes the very frustrating timeout issue which led to premature moves of video files. Now the download progress polling goes on par with xklb's lb dl execution. Upon [reaching] the timeout limit, the user will only be flashed a status message about the [abnormal] delay. The task will fail due to unavailable fragment only when xklb fails.

Tested rigorously and proudly on Ubuntu 24.10 (t4). Fix applied successfully on LRN2.

image

@deldesir
deldesir marked this pull request as draft June 4, 2024 13:58
@holta holta added the enhancement New feature or request label Jun 4, 2024
@deldesir deldesir changed the title Enhance timeout check to avoid false positives during post processing Enhance timeout check to avoid false positives during processing Jun 4, 2024
@holta holta added the bug Something isn't working label Jun 4, 2024
deldesir added 2 commits June 4, 2024 13:15
Also make code more concise and reliable
This goes on par with xklb execution. Fails only when xklb fails
@deldesir
deldesir requested a review from holta June 4, 2024 20:29
@deldesir deldesir self-assigned this Jun 4, 2024
@deldesir
deldesir marked this pull request as ready for review June 4, 2024 20:29
@holta

holta commented Jun 4, 2024

Copy link
Copy Markdown
Member

Tested rigorously and proudly on Ubuntu 24.10 (t4). Fix applied successfully on LRN2.

Awesome news:

Did a couple LRN2 tests (Ubuntu 24.04) confirm?

@deldesir

deldesir commented Jun 4, 2024

Copy link
Copy Markdown
Member Author

I made more than a couple tests on t4. I'll do playlist test right away on lrn2.

@deldesir

deldesir commented Jun 4, 2024

Copy link
Copy Markdown
Member Author

Did a couple LRN2 tests (Ubuntu 24.04) confirm?

Single videos and playlists now work.

Screenshot_20240604-172133

Comment thread cps/tasks/download.py Outdated
Comment on lines +9 to +11
from ..constants import XKLB_DB_FILE
from ..services.worker import CalibreTask, STAT_FINISH_SUCCESS, STAT_FAIL, STAT_STARTED, STAT_WAITING
from ..subproc_wrapper import process_open

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Absolute imports are usually preferred because relative imports (..) can lead to confusion in module resolution.

Does an absolute path starting from the package name work?

Suggested change
from ..constants import XKLB_DB_FILE
from ..services.worker import CalibreTask, STAT_FINISH_SUCCESS, STAT_FAIL, STAT_STARTED, STAT_WAITING
from ..subproc_wrapper import process_open
from calibreweb.cps.constants import XKLB_DB_FILE
from calibreweb.cps.services.worker import CalibreTask, STAT_FINISH_SUCCESS, STAT_FAIL, STAT_STARTED, STAT_WAITING
from calibreweb.cps.subproc_wrapper import process_open

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Absolute paths work. I was not comfortable seeing them flagged as unresolved in vscode. Thanks.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to remove the package name from absolute imports. It will work starting with cps.

Co-authored-by: Bernie Innocenti <bernie@codewiz.org>
Comment thread cps/tasks/download.py Outdated
log.info("Download task for %s completed successfully", self.media_url)
else:
self.end_time = datetime.now()
self.end_time = datetime.now()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If end_time needs to be set on both branches (lines 128 and 132), do it just once above the if.

By the way, this class sets end_time in multiple places (e.g. line 26 and 72). If the goal is to record the end time after the task completes, can these lines be eliminated?

Note: in case end_time ends up being unset, CalibreTask.runtime() will fail. The finally should catch all exit paths from the while loop, except when self.media_url is unset (which skips the entire thing).

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The goal is to record the run time dynamically per
https://github.com/iiab/calibre-web/blob/eb8302387b75714c9ab8addafac5df8af74e93c8/cps/services/worker.py#L232C5-L235C1
I'll remove the redundancy for 128 and 132, but if I remove 26 and 72, I will lose the dynamic run time messages. I'll move 72 to be more efficient.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in de4fdce

@holta

holta commented Jun 4, 2024

Copy link
Copy Markdown
Member

Safe to merge?

@deldesir

deldesir commented Jun 4, 2024

Copy link
Copy Markdown
Member Author

Safe to merge?

Yes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants