Arm64 vs aarch64 in pull request 123

[gebau00a]

Pull request 123 added an arch check for videophash.py
On arm64, it expects the arch type arm64, but distributions know arm64 as aarch64.
Running namer directly results in a Unsupport architecture error aarch64

Adding the dict link 'aarch64': 'arm64' runs, but videohashes-arm64-linux fails to produce output, thus an error is raised

Traceback (most recent call last):
  File "/usr/lib/python3.11/threading.py", line 995, in _bootstrap
    self._bootstrap_inner()
  File "/usr/lib/python3.11/threading.py", line 1038, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.11/threading.py", line 975, in run
    self._target(*self._args, **self._kwargs)
> File "/home/andy/.local/lib/python3.11/site-packages/namer/watchdog.py", line 137, in __processing_thread
    handle(command)
  File "/home/andy/.local/lib/python3.11/site-packages/namer/watchdog.py", line 54, in handle
    process_file(command)
  File "/home/andy/.local/lib/python3.11/site-packages/namer/namer.py", line 176, in process_file
    phash = VideoPerceptualHash().get_stash_phash(command.target_movie_file) if command.config.search_phash else None
  File "/home/andy/.local/lib/python3.11/site-packages/namer/videophash.py", line 91, in get_stash_phash
   return self._get_stash_phash(file, stat.st_size, stat.st_mtime)
  File "/home/andy/.local/lib/python3.11/site-packages/namer/videophash.py", line 96, in _get_stash_phash
    return self.__execute_stash_phash(file)
  File "/home/andy/.local/lib/python3.11/site-packages/namer/videophash.py", line 134, in __execute_stash_phash
    data = json.loads(stdout, object_hook=lambda d: SimpleNamespace(**d))
  File "/usr/lib/python3.11/json/__init__.py", line 359, in loads
    return cls(**kw).decode(s)
  File "/usr/lib/python3.11/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
 File "/usr/lib/python3.11/json/decoder.py", line 355, in raw_decode
   raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Running videohashes-arm64-linux manually with a link to a video file generates the error
ffmpeg/ffprobe executables not found

Scanning through the compiled code for videohashes-arm64-linux shows several matches for arm64, but none for aarch64.
I assume it's looking for an arm64 ffmpeg, and cannot find an aarch64 ffmpeg.

Not sure if this is an upstream error from https://github.com/peolic/videohashes or where the videohashes executable comes from

[4c0d3r]

Video hashes come from that repo, that they run locally is great (you do need to install ffmpeg though for it function). Adding aarch64 to this map: https://github.com/ThePornDatabase/namer/blob/main/namer/videophash.py#L38-L42. Adding it now.

[gebau00a]

I have ffmpeg installed, namer was working flawlessly before the videophash function was updated.
Nor sure why it cannot find ffmpeg.

[4c0d3r]

try latest release (just releaseed), should allow namer to call videohashes-arm64-linux. Verify ffmpeg is on the path, as videohashes-arm64-linux calls ffmpeg as well.

[4c0d3r]

Just verified, nothing special in the ffmpeg/fprobe look up, just need to be on path: https://github.com/stashapp/stash/blob/develop/pkg/ffmpeg/downloader.go#L21-L39

[4c0d3r]

@gebau00a please try with latest release and update here, I suspect it should work. Tempted to close this, but don't want to be perceived as rude, so please let me know what happens with the latest namer and ffmpeg/ffprobe verified on the path.

[gebau00a]

Journal log, namer run as systems service:

Dec 24 00:55:08 odroid python[113640]: 2022-12-24 00:55:08 | ℹ️ INFO     | ffmpeg version 4.4.3 found
Dec 24 00:55:08 odroid python[113640]: 2022-12-24 00:55:08 | ℹ️ INFO     | ffprobe version 4.4.3 found
Dec 24 00:55:08 odroid python[113640]: 2022-12-24 00:55:08 | ℹ️ INFO     | Start porndb scene watcher.... watching: /mnt/downloads/toTag/newFiles
Dec 24 00:55:08 odroid python[113640]: 2022-12-24 00:55:08 | ℹ️ INFO     | Starting server: http://0.0.0.0:6980/
Dec 24 00:55:25 odroid python[113640]: 2022-12-24 00:55:25 | ℹ️ INFO     | watchdog process called for xxx
Dec 24 00:55:25 odroid python[113640]: 2022-12-24 00:55:25 | ℹ️ INFO     | Moving /mnt/downloads/toTag/newFiles/xxx to /mnt/downloads/toTag/tag_work/xxx for processing
Dec 24 00:55:25 odroid python[113640]: 2022-12-24 00:55:25 | ℹ️ INFO     | Processing: xxx
Dec 24 00:55:25 odroid python[113640]: 2022-12-24 00:55:25 | ℹ️ INFO     | Calculating phash for file "xxx"
Dec 24 00:55:25 odroid python[113640]: 2022-12-24 00:55:25 | ❌ ERROR    | An error has been caught in function '__processing_thread', process 'MainProcess' (113640), thread 'Thread-2 (__processing_thread)' (281472612823456):
Dec 24 00:55:25 odroid python[113640]: Traceback (most recent call last):
Dec 24 00:55:25 odroid python[113640]:   File "/usr/lib/python3.11/threading.py", line 995, in _bootstrap
Dec 24 00:55:25 odroid python[113640]:     self._bootstrap_inner()
Dec 24 00:55:25 odroid python[113640]:   File "/usr/lib/python3.11/threading.py", line 1038, in _bootstrap_inner
Dec 24 00:55:25 odroid python[113640]:     self.run()
Dec 24 00:55:25 odroid python[113640]:   File "/usr/lib/python3.11/threading.py", line 975, in run
Dec 24 00:55:25 odroid python[113640]:     self._target(*self._args, **self._kwargs)
Dec 24 00:55:25 odroid python[113640]: > File "/home/andy/.local/lib/python3.11/site-packages/namer/watchdog.py", line 137, in __processing_thread
Dec 24 00:55:25 odroid python[113640]:     handle(command)
Dec 24 00:55:25 odroid python[113640]:   File "/home/andy/.local/lib/python3.11/site-packages/namer/watchdog.py", line 54, in handle
Dec 24 00:55:25 odroid python[113640]:     process_file(command)
Dec 24 00:55:25 odroid python[113640]:   File "/home/andy/.local/lib/python3.11/site-packages/namer/namer.py", line 176, in process_file
Dec 24 00:55:25 odroid python[113640]:     phash = VideoPerceptualHash().get_stash_phash(command.target_movie_file) if command.config.search_phash else None
Dec 24 00:55:25 odroid python[113640]:   File "/home/andy/.local/lib/python3.11/site-packages/namer/videophash.py", line 91, in get_stash_phash
Dec 24 00:55:25 odroid python[113640]:     return self._get_stash_phash(file, stat.st_size, stat.st_mtime)
Dec 24 00:55:25 odroid python[113640]:   File "/home/andy/.local/lib/python3.11/site-packages/namer/videophash.py", line 96, in _get_stash_phash
Dec 24 00:55:25 odroid python[113640]:     return self.__execute_stash_phash(file)
Dec 24 00:55:25 odroid python[113640]:   File "/home/andy/.local/lib/python3.11/site-packages/namer/videophash.py", line 134, in __execute_stash_phash
Dec 24 00:55:25 odroid python[113640]:     data = json.loads(stdout, object_hook=lambda d: SimpleNamespace(**d))
Dec 24 00:55:25 odroid python[113640]:   File "/usr/lib/python3.11/json/__init__.py", line 359, in loads
Dec 24 00:55:25 odroid python[113640]:     return cls(**kw).decode(s)
Dec 24 00:55:25 odroid python[113640]:   File "/usr/lib/python3.11/json/decoder.py", line 337, in decode
Dec 24 00:55:25 odroid python[113640]:     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
Dec 24 00:55:25 odroid python[113640]:   File "/usr/lib/python3.11/json/decoder.py", line 355, in raw_decode
Dec 24 00:55:25 odroid python[113640]:     raise JSONDecodeError("Expecting value", s, err.value) from None
Dec 24 00:55:25 odroid python[113640]: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

[gebau00a]

I think the problem is not in Namer itself, but in the videohash executable

[4c0d3r]

Is xxx a real video file?

[gebau00a]

Yes it is. I only replaced the name as the name has no influence on the bug.
None of the files is working

[4c0d3r]

well crap. You can turn off phashes in the config as a work around. see the https://github.com/ThePornDatabase/namer/blob/main/namer/namer.cfg.default for the layout.

I have no way to debug this, so unless you have some spare time and or know a bit of go the above might be your only option . Hopefully someone with the hardware has time to look at the arm64 video hasher.

[DirtyRacer1337]

Just need to print output and see that it says, errors not in json format even with -json flag

[4c0d3r]

According to gebau00a:

Running videohashes-arm64-linux manually with a link to a video file generates the error
ffmpeg/ffprobe executables not found

which are on the path according to namer's log

Improving error handling is a good idea, but it'll lead us back here.

[DirtyRacer1337]

Added error printing, #125

[4c0d3r]

Releasing that as 1.10.5, maybe it'll provide more info.

[DirtyRacer1337]

Well if videohashes can't find ffmpeg it's problem with stash code, https://github.com/stashapp/stash