Feature/improve build time

[fekete965]

Description

Fixes #54

Type of Change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Style/UI improvements
• Code refactoring
• Performance improvements
• Test additions/updates

Additional Notes

Trialing out the following:

• Using bun instead of node to install the dependency
• Using bun instead of node to build with astro
• Use production build with astro
• Introduce caching in the GH build action

[lovestaco]

Hey @fekete965
I thought the incremental option was removed after the experimental version. Can you check once more?

Did you try the bun build? How long did it take?
What is the spec of your build machine?

[fekete965]

Hey @fekete965 I thought the incremental option was removed after the experimental version. Can you check once more?

Did you try the bun build? How long did it take? What is the spec of your build machine?

You are absolutely right! Last time when I used astro it was still a thing. I should have double-checked it in the documentation. I'll remove the related commit.

I'm still running build time comparison. I think the caching could speed up the build time but that only for the CI.

[fekete965]

I will try a different approach using Bun for it,
Since Bun doesn't allow us to tinker with the heap size, we can't use bun run astro build, it will kill the VM.
I will create a new GH action instead of updating the current one which can be ran manually. This will allow us to see how the build performs.
So this is my todo list for now:

• 1. Revert build GH action
• 2. Create a new runnable GH action dummy file which does nothing
• 3. Add some scripts to package.json using bun with astro to build different parts of the application (not really necessary anymore imo)
• 4. Create a separate jobs to build the follow sections: src/pages/png_icons, src/pages/svg_icons, src/pages/tools, src/pages/tldr in the new GH action (use a matrix)
• 5. Add caching between jobs if possible to speed up the CI
• 6. Make sure the built sections are in the same folder
• 7. Observe if there is any improvements and if there is anything else we could do
• 8. Once we are happy with this approach add a new section to the GH action where we deploy the app to a staging env if possible
• 9. Check staging against prod

[fekete965]

@lovestaco I can hopefully present something today. The very first PR will not completely address the issue. I do not want to risk taking down production which means I will need some help from you to setup a staging environment so we can safely test the new deployment flow. I'll update my previous comment (#79 (comment)) to reflect how far we are from completing my proposal.
Unfortunately, to test the new GH actions, we will need to merge the PR as we won't able to run GH actions locally. (There is a way but I found it unreliable annoyingly)

[lovestaco]

Hey @fekete965, What I’d suggest is to first try running npm run build on your machine with the old Astro setup and note how long it takes. Then do the same using Bun so we can get a clear comparison.

Before starting the build, make sure to clean everything for accurate results:

rm -rf node_modules/ yarn.lock package-lock.json dist .astro/cache

Since we don’t have a staging setup right now, I don’t think that’s an option at the moment.

[fekete965]

Hey @fekete965, What I’d suggest is to first try running npm run build on your machine with the old Astro setup and note how long it takes. Then do the same using Bun so we can get a clear comparison.

Before starting the build, make sure to clean everything for accurate results:

rm -rf node_modules/ yarn.lock package-lock.json dist .astro/cache

Since we don’t have a staging setup right now, I don’t think that’s an option at the moment.

Yes, that's what I was doing. I just had to fix a couple of things in my bash scripts.

[fekete965]

@lovestaco here is the summary of my trial/research:

🚀 Astro Build Benchmark Summary (Bun vs Node)

This document summarizes the benchmark results comparing Astro builds using Bun (both sequentially and in parallel via CI) against a single Node build.

---

🧪 Test Overview

• Environment: Local machine (Bun 1.x, Astro production mode)
• Benchmarked with: hyperfine
• Setup: 10 partial builds (prepare_tmp_folders.sh for each section)
• Goal: Measure total build time, CPU usage, and potential CI speedup.

---

📊 Sequential Builds (Bun, Local)

Build Name	Real (s)	Real (h:mm:ss)	User (s)	User (h:mm:ss)	System (s)	System (h:mm:ss)
c	131.76	0:02:11	285.49	0:04:45	42.98	0:00:43
emojis	162.75	0:02:43	371.21	0:06:11	51.12	0:00:51
mcp	438.88	0:07:19	1032.63	0:17:12	66.65	0:01:07
mcp-pages	139.34	0:02:19	275.79	0:04:36	45.69	0:00:46
png_icons	2017.13	0:33:37	3941.66	1:05:41	125.27	0:02:05
png_icons_pages	136.60	0:02:17	273.98	0:04:34	45.95	0:00:46
svg_icons	1872.26	0:31:12	3590.14	0:59:50	141.17	0:02:21
svg_icons_pages	128.35	0:02:08	261.91	0:04:21	40.82	0:00:41
t	127.70	0:02:07	255.87	0:04:15	42.06	0:00:42
tldr	726.33	0:12:06	789.52	0:13:09	56.59	0:00:57
🟰 Total	6870.41 s	1h 54m 30s	13077.27 s	3h 37m 57s	658.27 s	10m 58s

💡 Running all builds sequentially on one machine took ~1 hour 54 minutes total.

---

⚙️ Parallel Builds (Bun, in CI)

When these 10 builds run in parallel (e.g., in GitHub Actions matrix jobs), the slowest job determines the total wall time.

Build Name	Real (s)	Real (h:mm:ss)
c	131.76	0:02:11
emojis	162.75	0:02:43
mcp	438.88	0:07:19
mcp-pages	139.34	0:02:19
png_icons (slowest)	2017.13	0:33:37 ⏱️
png_icons_pages	136.60	0:02:17
svg_icons	1872.26	0:31:12
svg_icons_pages	128.35	0:02:08
t	127.70	0:02:07
tldr	726.33	0:12:06

🧮 Estimated total wall-clock time in CI ≈ ~34–36 minutes
(dominated by the png_icons build, plus ~2 minutes setup overhead)

---

⚖️ Bun (Parallel) vs Node (Full Build)

Build Type	Wall Time (s)	Wall Time (h:mm:ss)	User (s)	User (h:mm:ss)	System (s)	System (h:mm:ss)
Bun (parallel builds, CI)	≈ 2160 s	0h 36m 00s	~13077 s (aggregate)	3h 37m 57s	~658 s	0h 10m 58s
Node (single full build)	4318 s	1h 11m 58s	10074 s	2h 47m 54s	533 s	0h 08m 53s
Δ Difference (Bun − Node)	−2158 s	≈ 36 m faster	+3003 s	+50 m more CPU time	+125 s	+2 m more system time

---

🧠 Insights

• 🟩 Parallel Bun builds in CI can reduce total build wall time by ~50% compared to a full Node build.
• 🧱 The slowest segment (png_icons) dominates the CI wall time — optimizing or caching it will/could improve results further.
• ⚙️ Although CPU usage is higher, CI runners handle these workloads in parallel, making it a good trade-off.
• 🧹 Sequential local runs are slower (1h 54m), but great for testing consistency and stability.

---

📌 Summary

Scenario	Approx Wall Time	Notes
Node full build	1h 12m	Baseline, single process
Bun partial builds (sequential)	1h 54m	42 min slower locally
Bun partial builds (parallel CI)	~36m 🏆	~2× faster overall

---

✅ Conclusion

Using Bun + Astro with parallelized partial builds is significantly faster in CI environments, cutting total wall time nearly in half compared to a single Node-based build.

[lovestaco]

@fekete965, the report looks good, and I can see the improvement. Can you pls push the script used for testing in the scripts folder?

Also, you have one merge conflict in package.json.

Also how does parallel building work? Does bun by default does parallel build? cuz I dont see any changes in any files regarding the parallel.

[fekete965]

@fekete965, the report looks good, and I can see the improvement. Can you pls push the script used for testing in the scripts folder?

Also, you have one merge conflict in package.json.

Also how does parallel building work? Does bun by default does parallel build? cuz I dont see any changes in any files regarding the parallel.

I'll push the changes very soon. I'll explain the changes and the idea behind them. Bun supports some kind of parallelizm but the main problem is the amount of data that needs handling. The best we can do it to build each section of the application on a separate thread using VM's. Locally we would still need to use node to build the project because bun does not support heap size configuration (yet) so calling build with bun will very quickly eat up your memory.
Fortunately, we build the project in a CI, so we can summon multiple jobs (VMs) in a matrix. Each of these instances will run a build in parallel and save the result as an artifact so we can retrieve it safely in the (very near) future once every build has finished. We could still use node for this purpose but bun seems to be ~10-15% faster based on other people's research.

So this is what's changed:

• I've created a bash script called prepare_tmp_folders. It takes in 1 argument which is the target folder we try to build. This essentially just adds an _ (underscore) to all the other folder expect the target folder (first arg of the script) which makes astro ignore those folders. I've found this approach much faster and cleaner than moving the folders into a temp folder and back. The script keeps track of the renamed folders by saving them into a separate file.
• I've created another bash script called restore_tmp_folders which restores the folder names using the temp file created by the previous script.
• I made a new GH action which allows us to perform targeted builds, it uses the above mentioned bash scripts to build a single section/part of the application. If something goes side ways, we restore the renamed folders. To be honest this is not really necessary as each job will be a separate VM so any changes made to them will be discarded once the job finished but just for piece of mind I made sure we restore the folder names
• I made another GH action which can be triggered manually. It takes in an environment so we can target production, staging, uat etc... if needed. This action will summon separate jobs using a matrix to build the project then it will gather each build and merge them together. In the end it doesn't do anything but logs out the final project's dir structure, no deployment is performed. This is purely just to see how much we could improve the build time and if the merged build is actually correct.

[lovestaco]

@fekete965
Okay, I’ll check more on the Bun setup.

Regarding your approach with the tmp folders and separate actions — that’s probably unnecessary because:

1. It increases the GitHub Action runner minutes.
2. We don’t need to rebuild everything — changes are only in svg_icons. We already have that setup here: https://github.com/HexmosTech/FreeDevTools/blob/feat/svg_icons-sqlite3/.github/workflows/buid_deploy.yml
3. So instead of focusing on targeted builds, let’s just focus on improving the build time itself.

You can remove those tmp scripts, just make sure the buid_deploy.yml uses Bun, and push those commits.
I’ll merge your code and test it in the runner tomorrow.

Edit: was reading about bun here are some pointers
https://dev.to/lovestaco/bun-vs-node-for-astro-static-site-builds-4okk

[fekete965]

@fekete965 Okay, I’ll check more on the Bun setup.

Regarding your approach with the tmp folders and separate actions — that’s probably unnecessary because:

1. It increases the GitHub Action runner minutes.
2. We don’t need to rebuild everything — changes are only in svg_icons. We already have that setup here: https://github.com/HexmosTech/FreeDevTools/blob/feat/svg_icons-sqlite3/.github/workflows/buid_deploy.yml
3. So instead of focusing on targeted builds, let’s just focus on improving the build time itself.

You can remove those tmp scripts, just make sure the buid_deploy.yml uses Bun, and push those commits. I’ll merge your code and test it in the runner tomorrow.

Edit: was reading about bun here are some pointers https://dev.to/lovestaco/bun-vs-node-for-astro-static-site-builds-4okk

I read through your current deployment flow and I think it was performing a full build every time regardless what has changed but I haven't looked at it that closely.

We pretty much cut down the build time in half + since we are using bun the build time is faster as well. Why would this increase the overall runner time? 🤔 You wouldn't even to use a custom runner for the CI which should reduce cost.
This would be the first step towards a more controlled build flow, it is pretty much the preparation for something which will let you prepare for a partial builds.
I have a code snippet which filters out which part of the application changed.
It would help us with to conditionally build sections of the application.
Since we would use GH artifacts, we would only need to add 1 extra step to the GH action and tinker with the matrix.
The other part of the deployment wouldn't need to change as we will always need to grab all the artifacts to merge them into a final build.

I'm happy to remove the scripts and use bun only, it's up to you.
I didn't want to do that as it killed my my Macbook Pro because of the heap size it requires for building the app and currently we can't control the heap size for Bun the same way we do for Node. I didn't want to create a PR which breaks the app just to get a Hactoberfest point.

[lovestaco]

I read through your current deployment flow and I think it was performing a full build every time regardless what has changed but I haven't looked at it that closely.

It's not doing a full rebuild every time.

https://github.com/HexmosTech/FreeDevTools/actions/runs/18942069147/job/54083205984

I'm happy to remove the scripts and use bun only, it's up to you.

Just check this action,
what it does is it builds the svg_icons only and once static files are generated, we rsync them to hosting server.

I didn't want to do that as it killed my my Macbook Pro because of the heap size it requires for building the app and currently we can't control the heap size for Bun the same way we do for Node

Agreed, there is one command in packege.json npm run build:mcp which builds only mcp part. Usually we dont use this in GH Actions. Just for local testing I had built that.

Guess we can have only bun switching in this PR?

[fekete965]

I read through your current deployment flow and I think it was performing a full build every time regardless what has changed but I haven't looked at it that closely.

It's not doing a full rebuild every time.

https://github.com/HexmosTech/FreeDevTools/actions/runs/18942069147/job/54083205984

> I'm happy to remove the scripts and use bun only, it's up to you.

Just check this action, what it does is it builds the svg_icons only and once static files are generated, we rsync them to hosting server.

I didn't want to do that as it killed my my Macbook Pro because of the heap size it requires for building the app and currently we can't control the heap size for Bun the same way we do for Node

Agreed, there is one command in packege.json npm run build:mcp which builds only mcp part. Usually we dont use this in GH Actions. Just for local testing I had built that.

Guess we can have only bun switching in this PR?

Nice!! Thanks for clarifying that!

I'll make a new PR instead of bloating this up and will only change the CI action to use bun stead of node.