Commit Graph
59 Commits
Author SHA1 Message Date
Tom Foster 4f23d566ed docs(docker): Restructure deployment guide and add env var reference
Add Quick Run section with complete getting-started workflow including
admin user creation via --execute flag. Consolidate Docker Compose to
treat reverse proxy as essential with Traefik/Caddy/nginx examples.

Move detailed image building to development guide, keeping deployment
docs focused on using pre-built images.

Create environment variables reference with practical examples and
context. Clarify built-in TLS is for testing only; production should
use reverse proxies.
2026-03-07 18:28:47 +00:00
Tom Foster 70ef6e4211 docs: Document maxperf Docker image variants from #1017
Add documentation for the new performance-optimised Docker images with
"-maxperf" suffix. These use the release-max-perf build profile with LTO
and target haswell CPU architecture on amd64 for optimal performance.

Also restructure the static prebuilt binary section in generic deployment
docs for better clarity and fix various UK English spelling issues.
2026-01-06 00:45:12 +00:00
Tom Foster 48ebf86335 feat(ci): Add Fedora RPM package build workflow
Build and publish RPM packages for Fedora using rpkg and official
rust-packaging macros. Packages are automatically signed by Forgejo's
built-in package registry (introduced in v9.0).

Publishes packages to organised groups:
- continuwuity (binary): base group (stable/dev/branch-name)
- continuwuity-debuginfo: GROUP-debug
- continuwuity (source RPM): GROUP-src

Workflow triggers on pushes to relevant paths and version tags (v*).
Tagged releases use clean version numbers (v1.2.3 becomes 1.2.3-1)
while branch builds use sanitised branch name versioning.

Uses dnf builddep to install build dependencies directly from the
generated SRPM, ensuring consistency between CI and spec file without
duplication. This also prevents hiding packaging issues that could
occur with --nodeps fallbacks.
2025-09-23 22:15:00 +01:00
Tom Foster 8959ac06ac ci: Split Rust build cache into dependencies and incremental caches
Replace single large build cache with separate dependencies and incremental
caches. Dependencies cache survives source code changes and uses tiered
restore keys. Removes build directory from caching to improve CI performance
while maintaining effective compilation caching with sccache.
2025-09-23 04:30:35 +01:00
Tom Foster 47f7ebfd68 fix: Use node_version in npm cache key for wrangler installation
Replace hashFiles('**/package-lock.json') with node_version since wrangler
is installed via npm without a lockfile to hash. Removes trailing dash from
cache keys and ensures npm dependencies are regenerated when Node.js version
changes.
2025-09-23 04:30:35 +01:00
Tom Foster 7d91f218b1 ci: Migrate to detect-versions with namespaced cache keys
Replace local detect-runner-os action with external detect-versions@v1 to
reduce custom action maintenance. Add architecture detection for future
cross-platform support and namespace all cache keys with "continuwuity-"
prefix to prevent collisions with other projects on shared runners.

Updates cache mount IDs in Dockerfiles to match the new namespacing
convention, ensuring consistent cache isolation across CI and Docker builds.
2025-09-23 04:30:22 +01:00
Tom Foster a120a4fa95 fix: Handle runner cargo bin path migration in timelord action
Runner images have migrated from /usr/share/rust/.cargo/bin to standard
~/.cargo/bin location. Action now checks old location first and migrates
binaries if found, maintaining compatibility with both paths.

Bump cache key to v3 to ensure fresh binary cache after path changes.
2025-09-15 16:17:32 +01:00
Tom Foster 81b6b3547c fix: Resolve Forgejo runner v11 matrix job execution failure
Matrix jobs stopped starting after upgrading from runner v9 to v11 due to
changes in job dependency resolution. Remove redundant define-variables job
that computed static image paths and replace with IMAGE_PATH environment
variable.

Also fix timelord action binary caching for compatibility between different
runner images that install cargo binaries in different locations.
2025-09-13 17:12:09 +01:00
Tom Foster 3d5355dfc3 chore(renovate): Add auto-merge for renovatebot and reorganise package rules
Enable automatic merging of ghcr.io/renovatebot/renovate docker image updates
to reduce manual maintenance overhead.

Reorganise package rules by manager type (cargo, github-actions, docker) and
add missing description for cargo concurrency limit rule to improve config
maintainability.
2025-09-12 17:50:08 +01:00
Tom Foster 542dff50bd ci: Split Docker builds into sequential release and max-perf stages
Separate fast release builds from slow max-perf builds to optimise runner
utilisation and provide quicker feedback. Release builds complete first with
standard optimisations, followed by Haswell-optimised dragrace builds once
the safe builds pass successfully.

Extract build logic into focused composite actions for better log visibility
in Forgejo UI. Split monolithic build action into prepare-docker-build,
inline docker build step, and upload-docker-artifacts to ensure each phase
completes independently and shows logs immediately.

Creates separate manifests at each stage to avoid waiting for all builds
before publishing.
2025-09-12 12:43:19 +01:00
Tom Foster 9c147b182f ci: Fix BuildKit cache invalidation and add Haswell-optimised builds
The workflow was rebuilding dependencies unnecessarily despite timelord
restoring timestamps because TARGET_CPU and RUST_PROFILE weren't passed
to Docker, creating inconsistent cache keys. Now passes both arguments
for proper cache reuse.

Adds Haswell-optimised builds alongside baseline builds using -march=haswell
for PCLMUL instruction support. Recent build improvements reducing compile
times from 15-20 minutes to ~5 minutes make this additional CPU variant
feasible. Users can pull optimised images with -haswell suffix.
2025-09-11 13:59:43 +01:00
Tom Foster 5126cb4554 fix: Use forgejo/upload-artifact@v4 for artifact consistency
Follow-on to correct #1009. The previous fix downgraded upload-artifact
to v3 but kept download-artifact@v4, creating incompatible storage
formats that prevented artifact pattern filtering from working.

Update all upload-artifact actions to v4 and adjust renovate
configuration to disable automatic updates for forgejo artifact
actions to maintain version consistency.
2025-09-11 11:57:04 +01:00
Tom Foster 0673ac1a6c fix: Fix artifact action compatibility and add digest debugging
Resolve upload-artifact v4 GHES compatibility errors by downgrading to v3.
Switch to standard forgejo/download-artifact@v4 for pattern filtering support.
Update renovate configuration to prevent future incompatible upgrades.

Add diagnostic output to digest export step to troubleshoot zero-byte
artifact uploads preventing manifest creation. Include CI triggers for
Element workflow to test changes in pull requests.
2025-09-11 10:44:11 +01:00
Tom Foster 7950e2cc7f ci: Refactor timelord action to use git-warp-time fallback
Updates the timelord action to fall back to git-warp-time when the cache
is completely empty, enabling timestamp restoration even on fresh builds.
When git-warp-time is used, performs an unshallow fetch to get full history,
while subsequent runs use normal fetches. Simplifies the interface by making
inputs optional with sensible defaults.

Adds binary caching for timelord-cli and git-warp-time tools to avoid
repeated installations, and updates paths to use /usr/share/rust/.cargo/bin/
for the catthehacker runner image used by the dind profile (may need updating
if/when switching to standard image).

The main timelord restore now happens inside the Dockerfile itself, as Docker
intentionally wipes all file mtimes on COPY/ADD operations.
2025-09-08 08:34:29 +00:00
Tom Foster 6cf3c839e4 ci(release-image): Skip digest upload when not pushing images
After #992, builds without registry credentials skip Docker image output
but still extract binary artifacts. However, we were still trying to
upload digests for images that weren't created. Add conditional check
to only upload digests when actually pushing to registry.
2025-09-07 21:27:56 +01:00
Tom Foster 4a1091dd06 ci(release-image): Unify binary extraction using BuildKit local output
Fork PRs currently fail binary extraction with 'invalid reference format'
and 'must specify at least one container source' errors. This replaces the
registry-specific docker create/copy method with BuildKit's local output
feature for all builds.

Uses multiple outputs in single build: image export plus local binary
extraction from /sbin. Speeds up extracting binary artifacts and saves a
couple of extra workflow steps in the process.
2025-09-07 20:46:11 +01:00
Tom Foster 1e9701f379 ci(release-image): Skip setup steps when using persistent BuildKit
When BUILDKIT_ENDPOINT is set, builds run on a persistent BuildKit instance,
making runner setup steps unnecessary. Skip Rust toolchain installation,
QEMU setup, caching steps, and timelord to eliminate ~7 operations per job.

Also adds output to git SHA and timestamp steps for visibility.

Cuts at least a minute off average build time through fewer installs,
cache restores, and cache saves.
2025-09-07 18:59:05 +01:00
Tom Foster 2cedf0d2e1 fix(ci): Use image output instead of docker for fork PRs
Docker exporter doesn't support manifest lists (multi-platform builds).
For fork PRs without registry credentials, use 'type=image,push=false'
instead of 'type=docker' to build multi-platform images locally without pushing.
2025-09-07 18:32:38 +01:00
Tom Foster 84fdcd326a fix(ci): Resolve registry push failures for fork PRs
Fork PRs now fail during Docker image build with 'tag is needed when
pushing to registry' because BUILTIN_REGISTRY_ENABLED evaluates to false
without proper credentials, leaving the images list empty. This appears
to be due to recent Forgejo permission changes affecting fork access to
repository secrets.

Add fallback to official registry when credentials unavailable, skip
registry login and push operations for forks, and make merge job
conditional since no digests exist without push. This allows forks to
test Docker builds whilst avoiding authentication failures.
2025-09-07 17:39:18 +01:00
Tom Foster d640853f9d ci(docs): Optimise build performance with caching and conditional Node.js
Skip installing Node.js entirely if v20+ is already available, otherwise
install v22. Add npm dependency caching with OS-specific cache keys using
the custom detect-runner-os action for proper cache isolation between
runners. Dependencies normally take just under 10s, so this should more
than halve the doc build time to free up runner slots.
2025-09-07 14:51:10 +01:00
Tom Foster fff9629b0f fix(docker): Resolve liburing.so.2 loading error for non-root users
Container failed to start when running as non-root (user 1000:1000) because
copied directories had restrictive 770 permissions, likely due to different
umask in persistent BuildKit. Non-root users couldn't access /usr/lib to
load required dynamic libraries.

Introduces prepper stage using Ubuntu to organize files into layered structure
with explicit 755 directory permissions before copying to scratch image.
Also fixes workflow syntax error and removes docker/** from paths-ignore to
ensure Docker changes trigger CI builds.
2025-09-07 14:13:14 +01:00
Tom Foster 1a3107c20a fix(ci): Replace Mozilla sccache action with token-free alternative
Replace mozilla-actions/sccache-action with a custom Forgejo-specific
implementation that eliminates GitHub token dependencies and rate limiting
issues for all contributors regardless of repository permissions.

The new action mirrors sccache binaries to the Forgejo package registry
and queries that instead of GitHub releases, maintaining identical functionality
including hostedtoolcache support.
2025-09-07 09:29:32 +01:00
Tom Foster 2516e783ba ci: Support optional persistent BuildKit endpoints in Docker builds
Allows us to use runners with persistent BuildKit containers for improved
caching and faster build times. Falls back to standard docker-container
driver when BUILDKIT_ENDPOINT environment variable is not set.
2025-09-06 16:05:51 +01:00
Tom Foster 609e239436 fix(fedora): Correct linting issues in RPM spec file
The Fedora RPM packaging files added in PR #950 weren't passing pre-commit
checks, causing CI failures for any branches rebased after that merge. This
applies prek linting fixes (typo correction, trailing whitespace removal,
and EOF newline) to ensure CI passes for all contributors.
2025-08-30 16:10:41 +01:00
Tom Foster b5a2e49ae4 fix: Resolve Clippy CI failures from elided lifetime warnings
The latest Rust nightly compiler (2025-08-27) introduced the
elided-named-lifetimes lint which causes Clippy CI checks to fail
when an elided lifetime ('_) resolves to a named lifetime that's
already in scope.

This commit fixes the Clippy warnings by:
- Making lifetime relationships explicit where 'a is already in scope
- Keeping elided lifetimes ('_) in functions without explicit
  lifetime parameters
- Ensuring proper lifetime handling in the database pool module

Affected files (17 total):
- Database map modules: Handle, Key, and KeyVal references in get,
  qry, keys, and stream operations
- Database pool module: into_recv_seek function

This change resolves the CI build failures without changing any
functionality, ensuring the codebase remains compatible with the
latest nightly Clippy checks.
2025-08-28 21:13:19 +01:00
Tom Foster dd22325ea2 refactor(ci): Consolidate Rust checks with optimised toolchain setup
Merge rust-checks.yml into prek-checks.yml for a unified workflow that
runs formatting and clippy/test checks in parallel jobs.

Add reusable composite actions:
- setup-rust: Smart Rust toolchain management with caching
  * Uses cargo-binstall for pre-built binary downloads
  * Integrates Mozilla sccache-action for compilation caching
  * Workspace-relative paths for better cache control
  * GitHub token support for improved rate limits
- setup-llvm-with-apt: LLVM installation with native dependencies
- detect-runner-os: Consistent OS detection for cache keys

Key improvements:
- Install prek via cargo-binstall --git (crates.io outdated at v0.0.1)
- Download timelord-cli from cargo-quickinstall
- Set BINSTALL_MAXIMUM_RESOLUTION_TIMEOUT=10 to avoid rate limit delays
- Default Rust version 1.87.0 with override support
- Remove redundant sccache stats (handled by Mozilla action)

Significantly reduces CI runtime through binary downloads instead of
compilation while maintaining all existing quality checks.
2025-08-28 19:20:14 +01:00
Tom Foster aacaf5a2a0 fix(ci): Downgrade setup-uv action from v6 to v5
The setup-uv@v6 action has deprecated Node 18 support mid-version by
using the File API, causing workflow failures. Temporarily downgrading
to v5 until we migrate to a better runner image with Node 20+ support.
2025-08-21 21:10:15 +01:00
Tom Foster 14a4b24fc5 fix(ci): Configure Renovate for Forgejo platform
- Set platform to 'forgejo' with proper API endpoint
- Use environment variables for all Renovate configuration
- Add git timeout and disable GitHub token warnings
- Move PR limit configuration to workflow
2025-08-17 17:37:24 +01:00
Tom Foster 731761f0fc Merge branch 'main' into tom/prek-was-prefligit 2025-08-17 15:08:44 +00:00
Tom Foster 4524a00fc6 chore(ci): Remove obsolete prefligit action
Now using prek directly via uvx, this custom action is no longer needed.
2025-08-17 16:00:42 +01:00
Tom Foster 9db750e97c fix(ci): Add full GitHub URL to renovate action
Forgejo's runner doesn't automatically assume actions are on github.com,
so we need to specify the full URL.
2025-08-17 15:51:29 +01:00
Tom Foster b14a4d470b Merge branch 'main' into tom/prek-was-prefligit 2025-08-17 14:16:35 +00:00
Tom Foster 5d1f141882 ci: Rename prefligit-checks.yml to prek-checks.yml
Rename workflow file to match the updated tool name.
2025-08-17 15:13:02 +01:00
Tom Foster b447cfff56 ci: Update prefligit to prek
The prefligit project has been renamed to prek due to typosquatting
concerns. This updates our CI to use the new name and recommended
installation method via uv, which significantly reduces setup time
compared to cargo install and includes automatic caching.

- Replace outdated static prefligit action with direct prek invocation
- Use uv as recommended by upstream: https://github.com/j178/prek
- Update check-byte-order-marker to fix-byte-order-marker (deprecated)
- Simplify workflow by removing unused ref calculations

The same .pre-commit-config.yaml works unchanged. Developers can
install locally with 'uvx prek install' or other methods from the repo.
2025-08-17 15:11:38 +01:00
Tom Foster 283888e788 Merge branch 'main' into tom/renovate 2025-08-17 13:27:33 +00:00
Tom Foster f54e59a068 ci: Add Renovate for automated dependency management
Configures Renovate bot to create PRs for outdated dependencies.
Runs daily at 5am UTC with manual trigger via workflow_dispatch.

Configuration:
- Ignores custom forks (jemalloc, telemetry packages)
- Groups: GHA minor/patch, Rust toolchain, lockfile, Rust patches
- Limits: 3 concurrent PRs, 2 PRs per hour
- Supports: Cargo, GitHub Actions, Nix
2025-08-17 14:20:20 +01:00
Tom Foster 2a183cc5a4 fix(build): Remove hardened_malloc from full feature set
The hardened_malloc feature conflicts with jemalloc, preventing successful
builds with the --features full flag. Commenting out hardened_malloc allows
the full profile to build correctly while maintaining all other features.
2025-08-17 13:44:32 +01:00
Tom Foster 583cb924f1 refactor: address code review feedback for auth and pagination improvements
- Extract duplicated thread/message pagination functions to shared utils module
- Refactor pagination token parsing to use Option combinators instead of defaults
- Split access token generation from assignment for clearer error handling
- Add appservice token collision detection at startup and registration
- Allow appservice re-registration with same token (for config updates)
- Simplify thread relation chunk building using iterator chaining
- Fix saturating_inc edge case in relation queries with explicit filtering
- Add concise comments explaining non-obvious behaviour choices
2025-08-12 05:29:41 +01:00
Tom Foster 9286838d23 fix(relations): improve thread pagination and include root event
Replace unreliable PduCount pagination tokens with ShortEventId throughout
the relations and messages endpoints. ShortEventId provides stable, unique
identifiers that persist across server restarts and database operations.

Key improvements:
- Add token parsing helpers that try ShortEventId first, fall back to
  PduCount for backwards compatibility
- Include thread root event when paginating backwards to thread start
- Fix off-by-one error in get_relations that was returning the starting
  event in results
- Only return next_batch/prev_batch tokens when more events are available,
  preventing clients from making unnecessary requests at thread boundaries
- Ensure consistent token format between /relations, /messages, and /sync
  endpoints for interoperability

This fixes duplicate events when scrolling at thread boundaries and ensures
the thread root message is visible when viewing a thread, matching expected
client behaviour.
2025-08-10 19:12:56 +01:00
Tom Foster d1ebcfaf0b fix(auth): prevent token collisions and optimise lookups
Ensures access tokens are unique across both user and appservice tables to
prevent authentication ambiguity and potential security issues.

Changes:
- On startup, automatically logout any user devices using tokens that
  conflict with appservice tokens (resolves in favour of appservices)
  and log a warning with affected user/device details
- When creating new user tokens, check for conflicts with appservice tokens
  and generate a new token if a collision would occur
- When registering new appservices, reject registration if the token is
  already in use by a user device
- Use futures::select_ok to race token lookups concurrently for better
  performance (adapted from tuwunel commit 066097a8)

This fix-forward approach resolves existing token collisions on startup
whilst preventing new ones from being created, without breaking existing
valid authentications.

The find_token optimisation is adapted from tuwunel (matrix-construct/tuwunel)
commit 066097a8: "Optimize user and appservice token queries" by Jason Volk.
2025-08-10 17:10:06 +01:00
Tom Foster e820551f62 fix(appservice): create sender_localpart user during appservice startup
Fixes #813: Application services were unable to work because their sender_localpart
user was never created in the database, preventing authentication.

This fix ensures the appservice user account is created when:
- The server starts up and loads existing appservices from the database
- A new appservice is registered via the admin command

Additionally, if an appservice user has been accidentally deactivated, it will be
automatically reactivated when the appservice starts.

The solution centralises all appservice startup logic into a single `start_appservice`
helper method, eliminating code duplication between the registration and startup paths.
2025-08-10 17:10:06 +01:00
Tom Foster 7e406445d4 Element Web build fixes 2025-07-03 22:26:02 +01:00
Tom Foster 60caa448b0 Tidy up publishing restriction check 2025-04-27 18:13:10 +00:00
Tom Foster eb886b6760 Element Web client build 2025-04-27 18:13:10 +00:00
Tom Foster 73c991edd0 Ignore all markdown for auto image builds 2025-04-27 18:13:10 +00:00
Tom Foster c698d65a92 Make Cloudflare Pages optional in CI 2025-04-27 18:13:10 +00:00
Tom Foster 4c8dfc4c2c Suggested community guidelines changes 2025-04-24 00:41:16 +01:00
Tom Foster b91af70e0b Add Forgejo CI workflow for Cloudflare Pages 2025-04-16 15:49:46 +01:00
Tom Foster 538347204f Add Matrix .well-known files 2025-04-16 15:49:46 +01:00
Tom Foster 90880e2689 Update mdBook config for continuwuity 2025-04-16 15:49:46 +01:00
Tom Foster 536efe2cd7 CI working with job summaries
All images should be generating correctly with parallelism and Docker manifests, and should output the end of the CI testing errors in a job summary box when the test fails.

When the test succeeds you get a big ✅ then at the end of the Docker publish it should include the `docker pull` commands for both Docker Hub and GHCR registries to make those pesky Docker users lives easier!
2024-04-26 02:03:40 -04:00
Tom Foster 6fbf4b5679 Simplify docker manifest CI stage 2024-04-26 02:03:40 -04:00
Tom Foster d57110e2f3 Improved CI artifact filename handling 2024-04-26 02:03:40 -04:00
Tom Foster e17f8d5b24 Multi-threaded CI to accelerate builds 2024-04-26 02:03:40 -04:00
Tom Foster 77d73583f6 Separate CI job for publishing docker manifest 2024-04-26 02:03:40 -04:00
Tom Foster 820cf3b9af ci: extract OCI images before loading and before login
Signed-off-by: strawberry <strawberry@puppygock.gay>
2024-04-26 02:03:40 -04:00
Tom Foster a36b37ee3d Simplify to publish combined jemalloc image for all architectures 2024-04-26 02:03:40 -04:00
Tom Foster 8525dda468 Simplify publish to Dockerhub 2024-04-26 02:03:40 -04:00
Tom Foster 37b6c1e7f7 Add extra example options 2024-03-08 12:51:21 -05:00