feat: Implement new update system with git hash tracking

[drpaneas]

This commit introduces a new update system that uses git hashes to track changes in git-based ports and version numbers for non-git ports. Key changes:

• Add git hash tracking for git-based ports
• Replace version-check with new update target that handles both git and version-based ports
• Add update-all.sh utility script for managing all ports
• Update build-all.sh to use clean install instead of version-check
• Add update-with-deps target to handle dependencies as well if you want to
• Add *.hash to .gitignore to prevent tracking local state
• Update documentation to reflect new update system and remove version-check

The new system provides better tracking of git-based ports and clearer status reports during updates. It also maintains backward compatibility with version-based ports while adding more robust update detection for git repositories.

[drpaneas]

To be clear, this is not replacing the current system -- I didn't want to be too intrusive here. It just adds the possibility to use git commit hashes. It stores it locally and makes sure it's in the .gitignore so you don't push it by accident.

To test this properly, you need to pick a kos-port that has GIT_REPOSITORY in their Makefile, such as raylibc4dc.

drpaneas@m2:/opt/toolchains/dc/kos-ports (master)% cd raylib4dc

drpaneas@m2:/opt/toolchains/dc/kos-ports/raylib4dc (master)% make show-deps
Dependencies for raylib4dc:
  ✗ libGL (not installed)

It is currently not installed, not its dependency. btw I added this nice show-deps as well :)
So if you try to update something that is not installed, it fails obviously:

drpaneas@m2:/opt/toolchains/dc/kos-ports/raylib4dc (master)% make update

Checking raylib4dc for updates...
  ✗ raylib4dc is not currently installed.
  Nothing to update. To install this port, run: make clean install
make: *** [update] Error 1
drpaneas@m2:/opt/toolchains/d

So let's install it:

... omitted for brevity...

Installing...
Marking raylib4dc 5.5.0 as installed.
Current directory: /opt/toolchains/dc/kos-ports/raylib4dc/build/raylib4dc-5.5.0/src
Checking specified branch: raylib4Consoles
Stored git hash: 34b35e8fd83bad7f5c9b25e2478b7dd6a157778c

it stored Stored git hash: 34b35e8fd83bad7f5c9b25e2478b7dd6a157778c at

cat ../lib/.kos-ports/raylib4dc.hash

34b35e8fd83bad7f5c9b25e2478b7dd6a157778c

by the way we can see that make install also installed its dependency. This is not new feature, just nice thing to see :)

drpaneas@m2:/opt/toolchains/dc/kos-ports/raylib4dc (master)% make show-deps

Dependencies for raylib4dc:
  ✓ libGL (installed)

and also the same functionality for libGL which also has GIT_REPOSITORY into its Makefile:

drpaneas@m2:/opt/toolchains/dc/kos-ports/raylib4dc (master)% cat ../lib/.kos-ports/libGL.hash

dfeb32c609e0fcfd846dc0fb8eed2d17d9818396

If you try to update now, it will check if there is a new commit upstream and decide:

drpaneas@m2:/opt/toolchains/dc/kos-ports/raylib4dc (master)% make update

Checking raylib4dc for updates...
  Last stored hash: 34b35e8fd83bad7f5c9b25e2478b7dd6a157778c
  Checking specified branch: raylib4Consoles
  Current repository hash: 34b35e8fd83bad7f5c9b25e2478b7dd6a157778c
  ✓ raylib4dc is up to date with git repository. No changes detected.

Now I am going to be evil and pretend there is a new update, for libGL, by changing its local .hash value, so when it compares with it updstream it finds a difference. The system doesn't check timestamps, if there is diff in git hashes, it triggers an update.

# you should not do this, this is just for demo testing purposes
drpaneas@m2:/opt/toolchains/dc/kos-ports/raylib4dc (master)% echo "123456789abcdfe" > ../lib/.kos-ports/libGL.hash

so if you try now to update raylibdc with dependencies it will check also its depedencies (that is libGL) in this example:

drpaneas@m2:/opt/toolchains/dc/kos-ports/raylib4dc (master)% make update-with-deps

Checking dependencies for raylib4dc...
  Updating dependency: libGL
Checking libGL for updates...
  Last stored hash: 123456789abcdfe
  Using default branch: master
  Current repository hash: dfeb32c609e0fcfd846dc0fb8eed2d17d9818396
  ! libGL has new changes in repository. Rebuilding...

... omitted for brevity ...

Installing...
Marking libGL 1.1.0 as installed.
Current directory: /opt/toolchains/dc/kos-ports/libGL/build/libGL-1.1.0
Using default branch: master
Stored git hash: dfeb32c609e0fcfd846dc0fb8eed2d17d9818396
---
Checking raylib4dc for updates...
Checking raylib4dc for updates...
  Last stored hash: 34b35e8fd83bad7f5c9b25e2478b7dd6a157778c
  Checking specified branch: raylib4Consoles
  Current repository hash: 34b35e8fd83bad7f5c9b25e2478b7dd6a157778c
  ✓ raylib4dc is up to date with git repository. No changes detected.

Of course you could also go to libGL directory and issue a make update from there, it's the same thing. But I think having this update-with-deps it's more handy.

Ok, so what happens if yout kos-port does not use git, but versions and releases. Then simply enough, the make update is not going to work there. If it would, that be lying to you.

drpaneas@m2:/opt/toolchains/dc/kos-ports/SDL (master)% make update

Checking SDL for updates...
  ✗ SDL is not currently installed.
  Nothing to update. To install this port, run: make clean install
make: *** [update] Error 1

Indeed it tells you that there is no way to verify if there is a new version there. Based on the implementation of it, the Maintainer has to keep track of the PORTVERSION and make sure it's up to date. Unless there is some URL that points to a latest tag, the only way to make sure you get the changes there is to ... force update :) so no changes there.

Now, instead of updating doing one by one, you can run the script:

drpaneas@m2:/opt/toolchains/dc/kos-ports (master)% ./utils/update-all.sh

... omitted for brevity...

Status Summary:
✓ Up to date: 2 port(s)
- Skipped: 45 port(s) (not installed)
- Skipped: 1 port(s) (not using git)

The two kos-ports I am using (raylib4dc, and libGL) are already up to date. There 45 which I have not installed, so nothing to update there and there is another one, which I do have it installed, but it's not using git, meaning it's not using GIT_REPOSITORY, thus you cannot use the new update mechanism there.

I hope these clears things up in what this PR is doing :)

[QuzarDC]

Hi! First off, thank you very much for this contribution and your very clear explanation of the intent. And my apologies for the long delay, but I'm not super proficient with these build scripts and a big concern was to set up some of our other supported environments to validate that it worked on multiple platforms.

I found though that it didn't quite work right:

/opt/toolchains/dc/sh-elf/bin/sh-elf-gcc-ar rcs ../src/libraylib.a rcore.o rshapes.o rtextures.o rtext.o utils.o  rmodels.o raudio.o
raylib static library generated (libraylib.a) in ../src!
make[2]: Leaving directory '/opt/toolchains/dc/kos-ports/raylib4dc/build/raylib4dc-5.5.0/src'
touch build-stamp
Installing...
Marking raylib4dc 5.5.0 as installed.
make[1]: *** No rule to make target 'store-hash'.  Stop.
make[1]: Leaving directory '/opt/toolchains/dc/kos-ports/raylib4dc'
make: *** [/opt/toolchains/dc/kos/../kos-ports/scripts/build.mk:44: install] Error 2

What seemed to happen was that the script set the branch of kos-ports back to master when I was working off your update branch. So it started with the updated scripts, but by the time it was finishing build.mk it had been restored back to the current master branch of kos-ports.

Since I was setting up a brand new environment I have the good opportunity to exactly recreate how I arrived at the error:

   1 mkdir -p /opt/toolchains/dc
   2 chmod -R 755 /opt/toolchains/dc
   3 chown -R $(id -u):$(id -g) /opt/toolchains/dc
   4 git clone https://github.com/KallistiOS/KallistiOS.git /opt/toolchains/dc/kos
   5 cd /opt/toolchains/dc/kos/utils/dc-chain
   6 make
   7 cd ../..
   8 cp ./doc/environ.sh.sample ./environ.sh
   9 vi environ.sh
  10 source ./environ.sh
  11 make
  12 git clone --recursive https://github.com/KallistiOS/kos-ports /opt/toolchains/dc/kos-ports
  13 cd ../kos-ports/
  14 git remote add DrPaneas https://github.com/drpaneas/kos-ports.git
  15 git pull DrPaneas
  16 git switch update
  17 cd raylib4dc/
  18 make
  19 make install

[drpaneas]

Thanks @QuzarDC for your review! I have addressed your issues.

drpaneas@m2:/opt/toolchains/dc/kos (master)% git clone --recursive https://github.com/KallistiOS/kos-ports 
drpaneas@m2:/opt/toolchains/dc/kos (master)% cd ../kos-ports/
drpaneas@m2:/opt/toolchains/dc/kos-ports (master)% git remote add DrPaneas https://github.com/drpaneas/kos-ports.git
drpaneas@m2:/opt/toolchains/dc/kos-ports (master)% git pull DrPaneas
drpaneas@m2:/opt/toolchains/dc/kos-ports (master)% git switch update
branch 'update' set up to track 'DrPaneas/update'.
drpaneas@m2:/opt/toolchains/dc/kos-ports (update)%  cd raylib4dc/
drpaneas@m2:/opt/toolchains/dc/kos-ports/raylib4dc (update)% make
Please build your port with 'make install clean'.
drpaneas@m2:/opt/toolchains/dc/kos-ports/raylib4dc (update)% make install clean

this worked without any issues.