Porting over few more improvements from #777 that can be applied to HQX despite the older Rust version:
- Removed Cargo.lock from .gitignore (the file itself was added in the original PR, but is still ignored and wouldn't get committed on changes).
- Removed couple of stray .DS_Store accidentally added in that PR.
- Added a `--locked` to `wasm-pack` build to make sure we rebuild HQX with the same versions from Cargo.lock.
- Removed separate `wasm-strip` and `wasm-opt -Os` steps from build.sh in HQX because they're already included in wasm-pack, and running twice only makes build slower.
Use a shared base image with fixed Emscripten version, autotools and optimisation flags for all C++ codecs.
Additionally, move build commands for codecs themselves to Makefile - they're already platform-specific, and Make allows for better caching and parallelisation that custom ad-hoc scripts.
This is essentially same as #777 but for C++.
It's not possible to share them across threads, so in case we decide to use multithreading in the future, it's best to mark them as thread_local right away, even if it's a no-op right now.
This consolidates Rust build process for various codecs into a single top-level image that is built once and reused.
This ensures that we use same version of tools across codecs (now controlled from a single place), simplifies build configs and commands, speeds up common builds and reduces disk space taken by Docker images by reusing same one.
Additionally, this PR renames all codecs to squoosh-* to work around the https://github.com/rustwasm/wasm-pack/issues/829 (which has been already fixed on master of wasm-pack but not released in a while), as well as adds `publish = false` to Cargo.toml to avoid accidental publishing for now.
I'm planning to do similar for Emscripten in a separate PR, although abilities to share configs there are much more limited due to lack of package manager in C++.
- Store Emscripten cache inside node_modules/.em_cache. Docker image ships without LTO libs, so Emscripten has to rebuild stdlibs on every build otherwise.
- Merge webp_enc + webp_dec build scripts. Core libwebp library is same in both cases, so there's no point in storing and building two copies of it.
* Add LTO for C++ builds
This didn't have much effect on fastcomp builds, but provides further size savings with new LLVM backend we switched to in #750 (and fixes the MozJPEG size regression from the same PR).
In the future we won't need to pass `--llvm-lto 1` explicitly, but latest Emscripten Docker image doesn't contain the Emscripten version with the necessary fixes for this.
* Delete build.log
Co-authored-by: Jake Archibald <jaffathecake@gmail.com>
We've been running each Make command in a single thread, resulting in fairly slow builds for C++ codecs.
This change instead runs all `make` invocations with `-j` defaulting to number of cores (retrieved via `nproc`).
On my machine Docker uses a VM configured to 4 cores out of 8 available. This change brings total build time for C++ codecs down from 10m28s to 7m5s (~3.5 minutes difference).
Note (1): I've converted imagequant builds to use built-in `make` as well to leverage this parallelisation and future-proof build script.
Note (2): we don't need to do the same for Rust, since Cargo parallelises builds by default.
Looks like we've been stuck on Emscripten fastcomp backend, misssing out on all new optimisations between LLVM 6 and LLVM 11 and going via slow asm.js pipeline when building.
This changes Docker images for Emscripten projects to point to emscripten-upstream instead and commits the updated artifacts.
This was a no-op in Wasm anyway. Now that I've added ability to control logging upstream, let's use it to disable it from compiled unit altogether for a slight win in size and perf.
This makes building simpler and allows us to potentially use multithreading version in the future.
For now points to a custom fork of OxiPNG that enables WebAssembly support, as PR is still pending review.
Benefits:
- newer versions of the libraries
- zlib: 1.2.8 -> 1.2.11
- libpng: 1.6.18beta04 -> 1.6.34
- much fewer dependencies to install (as libs are already in optipng archive and we don't need napa)
- much smaller build thanks to customised versions of zlib and libpng containing only APIs necessary for optipng itself: 238950 -> 177359 bytes
- much faster build thanks to preconfigured libpng and stripped APIs: 2m15s -> 40s
- much simpler build script: 77 -> 46 lines
This delegates loading of Wasm modules to Webpack itself, making wrapper code simpler.
Emscripten-generated modules are still using custom loading glue as they're not compatible with Webpack.
Nightly is no longer necessary for wee_alloc, so we don't need to rely on unstable versions of Rust anymore.
As a bonus, this reduces size of each Rust-related Docker image by >1 GB:
- `squoosh-rotate`: 2.94GB -> 1.87GB
- `squoosh-resize`: 3.09GB -> 2.02GB
- `squoosh-hqx`: 3.13GB -> 2.06GB
