Compare commits

...
50 Commits
Author SHA1 Message Date
Jason Volk ab6da09c02 Bump 1.1.0
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-19 09:03:17 +00:00
Jason Volk 18b9d7bc1f Add org.matrix.login.jwt support.
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-18 11:53:25 +00:00
Jason Volk b5dc933880 Additional epoch time convenience utils.
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-18 09:50:22 +00:00
Jason Volk f68038a826 Restrict password login to accounts of type 'password' or legacy untyped.
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-18 09:50:22 +00:00
Jason Volk b3a47566ff Move and rename 'token' to 'access_token' where applicable.
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-18 01:02:22 +00:00
Jason Volk 61cbd38284 Fix unrecognized prof_active jemalloc_conf on aarch64.
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-18 01:02:22 +00:00
Jason Volk 6cc251796e ci: Remove unnecessary/unused condition.
docker: Fix readme typo; link.

Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-17 22:05:07 +00:00
Jason Volk 7e8c4c8883 Fix password reset regression. (78a02edbbf) (fixes #70)
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-17 03:41:27 +00:00
Jason Volk 73c179b20a docker: Adjust rpm and deb packaging for multi-platform.
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-17 03:41:27 +00:00
Jason Volk cd8648dce5 Auth related cleanups.
Cleanup; additional error macros.

Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-17 03:41:27 +00:00
Jason Volk 83ceda808c ci: Add multi-runner/multi-arch to matrices. (closes #55)
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-17 03:41:27 +00:00
Jason Volk fc3fcf4526 docker: Determine nproc for darwin. (closes #52)
docker: Further separate apt and rust cache mounts.

Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-17 00:22:50 +00:00
Jason Volk f4e6807a3c Fix hard-coded well-known IP address.
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-16 22:43:47 +00:00
Jason Volk 13e0113587 Fix nightly coerce-container-to-any.
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-16 22:43:47 +00:00
Jason Volk 51778b9354 ci: Remove the deps workflow.
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-16 04:25:58 +00:00
Jason Volk 8fef40a37e docker: Add assertion for dynamic or static executable upon install.
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-16 04:25:58 +00:00
Jason Volk ee6524b803 docker: Fix erroneous sys_ variable names. (#52)
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-16 04:25:53 +00:00
Jason Volk 1645518859 Add link to nixpkgs request. [ci skip]
docker: Add apt command for getting-started section. [ci skip]
2025-06-15 02:36:07 +00:00
Jason Volk 6088d0e295 Update nix flake. (closes #64)
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-14 23:28:54 +00:00
Jason Volk 5a717266b8 Add new sentry endpoint. (closes #66)
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-14 22:48:49 +00:00
Wojciech Kordalski f2c5314296 Fix building RocksDB on Nix 2025-06-14 22:48:49 +00:00
Jason Volk 814d3b903e Update NixOS related. [ci skip]
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-14 22:48:49 +00:00
Jason Volk a114ab3ab6 Fix use of inherent_str_constructors (#64 #65)
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-14 22:18:21 +00:00
Jason Volk b01c5d5ef1 docker: Use specific msrv for stable toolchain.
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-14 22:10:00 +00:00
Jason Volk 21c0e4176b Add security disclosure policy. [ci skip]
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-14 09:52:30 +00:00
Jason Volk f1f31eff17 Detailed version string. (closes #62)
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-13 01:27:28 +00:00
Jason Volk b0b441d534 Add macro to get rustc version. (#62)
Add macro to query git-describe. (#62)

Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-13 01:27:28 +00:00
Jason Volk 5799059196 Abstract get_crate_name() into macro utils. 2025-06-12 21:31:44 +00:00
Jason Volk 026d756ef4 Adjust SmallVec buffer sizes to account for overhead.
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-12 11:08:29 +00:00
Jason Volk 37afdb0e90 docker: Fix typo. [ci skip]
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-12 02:39:02 +00:00
Jason Volk 2a2486182a Split login case bodies into handlers.
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-11 01:21:11 +00:00
Jason Volk 3056b60159 Bundle systemd service and post-install script for RPM package. (#59 #60)
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-11 01:21:11 +00:00
Jason Volk 000b876dcb Updates for Arch package related.
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-11 00:50:49 +00:00
Jason Volk fcd51ef30e docker: Simplify target names for distro packages.
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-11 00:50:49 +00:00
Jason Volk 7edbe11eb8 docker: Dedup cargo and chef dockerfiles.
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-10 20:32:58 +00:00
Jason Volk 0f256d7191 Split api::client::session into units.
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-08 20:02:53 +00:00
Jason Volk 72c231c613 Bump various additional dependencies.
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-08 19:34:21 +00:00
Jason Volk 66b36563b6 docker: Reformat cargo build rustflags.
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-08 09:22:43 +00:00
Jason Volk e6d782c701 Bump rocksdb.
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-08 06:03:23 +00:00
Jason Volk f63cc84c98 Bump opentelemetry.
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-08 03:39:24 +00:00
Jason Volk 83fe14c522 Bump core_affinity.
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-08 03:08:57 +00:00
Jason Volk 46ce426b87 Bump rebased rustyline-async.
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-08 03:08:57 +00:00
9a975c0304 chore: Migrate to axum 0.8
Co-authored-by: dasha_uwu
Co-authored-by: Jason Volk <jason@zemos.net>
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-08 03:08:57 +00:00
Jason Volk dc8bab9bcd Disable tokio-console for Axum dependency conflicts.
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-08 03:08:57 +00:00
Jason Volk ceb0f8f933 Bump trivial dependencies; rebase hyper-util.
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-08 03:08:57 +00:00
Jason Volk 53f2ea8e58 ci: Rename NEWS.md to RELEASE.md.
ci: Build fewer packages in non-main branch.

ci: Add lint phase for rust-check.

Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-08 03:08:57 +00:00
Jason Volk 8a83c23537 Split tuwunel_service::users into units.
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-06 22:41:01 +00:00
Jason Volk b0048fec6d Add target platform to artifact names.
Use better gzip input name which is preserved by metadata.

Add chmod for input file and use for executables.

Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-06 19:27:18 +00:00
Jason Volk 9bad2fa1eb ci: Use fatter layer caching; make script output consistent.
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-06 19:27:18 +00:00
Jason Volk 06f37e9e8a Use all-features for special-tag preview and latest images.
Signed-off-by: Jason Volk <jason@zemos.net>
2025-06-06 17:03:23 +00:00
85 changed files with 3655 additions and 3019 deletions

No files matched your search

+14 -6
View File
@@ -30,6 +30,10 @@ on:
machines:
type: string
description: Hardware platforms
runner:
type: string
description: Runner mask
default: 'self-hosted'
excludes:
type: string
default: '[]'
@@ -86,7 +90,7 @@ jobs:
${{matrix.sys_version}}
${{matrix.sys_target}}
runs-on: ${{matrix.machine}}
runs-on: ["${{matrix.machine}}", "${{inputs.runner}}"]
strategy:
fail-fast: ${{inputs.fail_fast}}
max-parallel: ${{inputs.max-parallel}}
@@ -141,12 +145,14 @@ jobs:
iid: ${{matrix.bake_target}}--${{matrix.cargo_profile}}--${{matrix.rust_toolchain}}--${{matrix.rust_target}}--${{matrix.feat_set}}--${{matrix.sys_name}}--${{matrix.sys_version}}--${{matrix.sys_target}}
dst: ${{ fromJSON(inputs.artifact)[matrix.bake_target].dst }}
src: ${{ fromJSON(inputs.artifact)[matrix.bake_target].src || fromJSON(inputs.artifact)[matrix.bake_target].dst }}
mod: ${{ fromJSON(inputs.artifact)[matrix.bake_target].chmod || 'u+r' }}
run: |
cid=$(docker create "$iid" /)
rm -rf _artifact
mkdir -p "_artifact"
docker cp "$cid:$src" "_artifact/$dst"
chmod "$mod" "_artifact/$dst"
# Optionally extract the image itself as an artifact.
- name: Extract Image Artifact
@@ -174,11 +180,13 @@ jobs:
env:
dst: ${{ fromJSON(inputs.artifact)[matrix.bake_target].dst }}
src: ${{ fromJSON(inputs.artifact)[matrix.bake_target].src || fromJSON(inputs.artifact)[matrix.bake_target].dst }}
mod: ${{ fromJSON(inputs.artifact)[matrix.bake_target].chmod || 'u+r' }}
run: |
rm -rf _artifact
mkdir -p "_artifact"
mv "$src" "_artifact/$dst"
chmod "$mod" "_artifact/$dst"
# Optionally compress extracted artifact
- name: Compress Artifact
@@ -194,9 +202,9 @@ jobs:
run: |
if test -n "$gzlev"; then
mv "_artifact/$dst" "_artifact/foo"
gzip "-$gzlev" "_artifact/foo"
mv "_artifact/foo.gz" "_artifact/$dst"
mv "_artifact/$dst" "_artifact/$dst.artifact"
gzip "-$gzlev" "_artifact/$dst.artifact"
mv "_artifact/$dst.artifact.gz" "_artifact/$dst"
elif test -n "$zstlev"; then
zstd "-$zstlev" "_artifact/$dst"
mv "_artifact/$dst.zst" "_artifact/$dst"
@@ -211,7 +219,7 @@ jobs:
uses: actions/upload-artifact@v4
with:
path: _artifact/*
name: ${{matrix.cargo_profile}}-${{matrix.feat_set}}-${{fromJSON(inputs.artifact)[matrix.bake_target].dst}}
name: ${{matrix.cargo_profile}}-${{matrix.feat_set}}-${{matrix.sys_target}}-${{fromJSON(inputs.artifact)[matrix.bake_target].dst}}
# Optionally upload artifact as pages-artifact
- name: Upload Pages Artifact
@@ -240,5 +248,5 @@ jobs:
with:
upload_url: ${{inputs.release_url}}
asset_path: _artifact/${{fromJSON(inputs.artifact)[matrix.bake_target].dst}}
asset_name: ${{github.ref_name}}-${{matrix.cargo_profile}}-${{matrix.feat_set}}-${{fromJSON(inputs.artifact)[matrix.bake_target].dst}}
asset_name: ${{github.ref_name}}-${{matrix.cargo_profile}}-${{matrix.feat_set}}-${{matrix.sys_target}}-${{fromJSON(inputs.artifact)[matrix.bake_target].dst}}
asset_content_type: ${{fromJSON(inputs.artifact)[matrix.bake_target].mime || 'application/octet-stream'}}
-163
View File
@@ -1,163 +0,0 @@
name: Dependencies
on:
workflow_call:
inputs:
cargo_profiles:
type: string
description: Cargo profiles
feat_sets:
type: string
description: Cargo feature groups
rust_toolchains:
type: string
description: Rust toolchains
sys_names:
type: string
description: System names
sys_versions:
type: string
description: System versions
rust_targets:
type: string
description: Rust targets
sys_targets:
type: string
description: System targets
machines:
type: string
description: Hardware platform vector
excludes:
type: string
default: '[]'
description: Matrix exclusions
includes:
type: string
default: '[]'
description: Matrix inclusions
show_systems:
type: boolean
default: true
description: Pre-build systems dependency targets
show_sources:
type: boolean
default: true
description: Pre-build sources dependency targets
show_rocksdb:
type: boolean
default: true
description: Pre-build rocksdb dependency targets
show_cargo:
type: boolean
default: true
description: Pre-build cargo dependency targets
jobs:
systems:
if: >
!failure() && !cancelled()
&& inputs.show_systems
name: System
uses: ./.github/workflows/bake.yml
with:
bake_targets: '["systems"]'
cargo_profiles: ${{inputs.cargo_profiles}}
feat_sets: ${{inputs.feat_sets}}
rust_toolchains: ${{inputs.rust_toolchains}}
sys_names: ${{inputs.sys_names}}
sys_versions: ${{inputs.sys_versions}}
rust_targets: ${{inputs.rust_targets}}
sys_targets: ${{inputs.sys_targets}}
machines: ${{inputs.machines}}
excludes: ${{inputs.excludes}}
includes: ${{inputs.includes}}
buildsys:
if: >
!failure() && !cancelled()
&& inputs.show_systems
name: Builder
needs: [systems]
uses: ./.github/workflows/bake.yml
with:
bake_targets: '["rust"]'
cargo_profiles: ${{inputs.cargo_profiles}}
feat_sets: ${{inputs.feat_sets}}
rust_toolchains: ${{inputs.rust_toolchains}}
sys_names: ${{inputs.sys_names}}
sys_versions: ${{inputs.sys_versions}}
rust_targets: ${{inputs.rust_targets}}
sys_targets: ${{inputs.sys_targets}}
machines: ${{inputs.machines}}
excludes: ${{inputs.excludes}}
includes: ${{inputs.includes}}
sources:
if: >
!failure() && !cancelled()
&& inputs.show_sources
name: Acquire
needs: [buildsys]
uses: ./.github/workflows/bake.yml
with:
bake_targets: '["sources"]'
cargo_profiles: ${{inputs.cargo_profiles}}
feat_sets: ${{inputs.feat_sets}}
rust_toolchains: ${{inputs.rust_toolchains}}
sys_names: ${{inputs.sys_names}}
sys_versions: ${{inputs.sys_versions}}
rust_targets: ${{inputs.rust_targets}}
sys_targets: ${{inputs.sys_targets}}
machines: ${{inputs.machines}}
excludes: ${{inputs.excludes}}
includes: ${{inputs.includes}}
rocksdb:
if: >
!failure() && !cancelled()
&& inputs.show_rocksdb
name: RocksDB
needs: [sources]
uses: ./.github/workflows/bake.yml
with:
bake_targets: '["rocksdb"]'
cargo_profiles: ${{inputs.cargo_profiles}}
feat_sets: ${{inputs.feat_sets}}
rust_toolchains: ${{inputs.rust_toolchains}}
sys_names: ${{inputs.sys_names}}
sys_versions: ${{inputs.sys_versions}}
rust_targets: ${{inputs.rust_targets}}
sys_targets: ${{inputs.sys_targets}}
machines: ${{inputs.machines}}
excludes: ${{inputs.excludes}}
includes: ${{inputs.includes}}
deps:
if: >
!failure() && !cancelled()
&& inputs.show_cargo
name: Build
needs: [rocksdb]
uses: ./.github/workflows/bake.yml
with:
bake_targets: >
[
"deps-clippy",
"deps-build-tests",
"deps-build-bins"
]
cargo_profiles: ${{inputs.cargo_profiles}}
feat_sets: ${{inputs.feat_sets}}
rust_toolchains: ${{inputs.rust_toolchains}}
sys_names: ${{inputs.sys_names}}
sys_versions: ${{inputs.sys_versions}}
rust_targets: ${{inputs.rust_targets}}
sys_targets: ${{inputs.sys_targets}}
machines: ${{inputs.machines}}
excludes: ${{inputs.excludes}}
includes: ${{inputs.includes}}
+31 -4
View File
@@ -43,6 +43,7 @@ jobs:
&& contains(fromJSON(inputs.cargo_profiles), fromJSON('["test"]')[0])
&& contains(fromJSON(inputs.feat_sets), fromJSON('["all"]')[0])
&& contains(fromJSON(inputs.rust_toolchains), fromJSON('["nightly"]')[0])
&& contains(fromJSON(inputs.machines), fromJSON('["X64"]')[0])
name: Format
uses: ./.github/workflows/bake.yml
@@ -55,7 +56,7 @@ jobs:
sys_names: ${{inputs.sys_names}}
sys_targets: ${{inputs.sys_targets}}
sys_versions: ${{inputs.sys_versions}}
machines: ${{inputs.machines}}
machines: '["X64"]'
excludes: ${{inputs.excludes}}
includes: ${{inputs.includes}}
@@ -65,6 +66,7 @@ jobs:
&& contains(fromJSON(inputs.cargo_profiles), fromJSON('["test"]')[0])
&& contains(fromJSON(inputs.feat_sets), fromJSON('["all"]')[0])
&& contains(fromJSON(inputs.rust_toolchains), fromJSON('["nightly"]')[0])
&& contains(fromJSON(inputs.machines), fromJSON('["X64"]')[0])
name: Typos
uses: ./.github/workflows/bake.yml
@@ -77,7 +79,7 @@ jobs:
sys_versions: ${{inputs.sys_versions}}
rust_targets: ${{inputs.rust_targets}}
sys_targets: ${{inputs.sys_targets}}
machines: ${{inputs.machines}}
machines: '["X64"]'
excludes: ${{inputs.excludes}}
includes: ${{inputs.includes}}
@@ -87,6 +89,7 @@ jobs:
&& contains(fromJSON(inputs.cargo_profiles), fromJSON('["test"]')[0])
&& contains(fromJSON(inputs.feat_sets), fromJSON('["all"]')[0])
&& contains(fromJSON(inputs.rust_toolchains), fromJSON('["nightly"]')[0])
&& contains(fromJSON(inputs.machines), fromJSON('["X64"]')[0])
name: Audit
uses: ./.github/workflows/bake.yml
@@ -99,7 +102,7 @@ jobs:
sys_versions: ${{inputs.sys_versions}}
rust_targets: ${{inputs.rust_targets}}
sys_targets: ${{inputs.sys_targets}}
machines: ${{inputs.machines}}
machines: '["X64"]'
excludes: ${{inputs.excludes}}
includes: ${{inputs.includes}}
@@ -109,6 +112,7 @@ jobs:
&& contains(fromJSON(inputs.cargo_profiles), fromJSON('["test"]')[0])
&& contains(fromJSON(inputs.feat_sets), fromJSON('["all"]')[0])
&& contains(fromJSON(inputs.rust_toolchains), fromJSON('["nightly"]')[0])
&& contains(fromJSON(inputs.machines), fromJSON('["X64"]')[0])
name: Lychee
uses: ./.github/workflows/bake.yml
@@ -121,7 +125,30 @@ jobs:
sys_versions: ${{inputs.sys_versions}}
rust_targets: ${{inputs.rust_targets}}
sys_targets: ${{inputs.sys_targets}}
machines: ${{inputs.machines}}
machines: '["X64"]'
excludes: ${{inputs.excludes}}
includes: ${{inputs.includes}}
check:
if: >
!failure() && !cancelled()
&& contains(fromJSON(inputs.cargo_profiles), fromJSON('["test"]')[0])
&& contains(fromJSON(inputs.feat_sets), fromJSON('["all"]')[0])
&& contains(fromJSON(inputs.rust_toolchains), fromJSON('["nightly"]')[0])
&& contains(fromJSON(inputs.machines), fromJSON('["X64"]')[0])
name: Check
uses: ./.github/workflows/bake.yml
with:
bake_targets: '["check"]'
cargo_profiles: '["test"]'
feat_sets: '["all"]'
rust_toolchains: '["nightly"]'
sys_names: ${{inputs.sys_names}}
sys_versions: ${{inputs.sys_versions}}
rust_targets: ${{inputs.rust_targets}}
sys_targets: ${{inputs.sys_targets}}
machines: '["X64"]'
excludes: ${{inputs.excludes}}
includes: ${{inputs.includes}}
+61 -38
View File
@@ -18,7 +18,7 @@ concurrency:
jobs:
init:
name: Init
runs-on: self-hosted
runs-on: ${{matrix.runner}}
env:
default_cargo_profiles: '["test", "release"]'
default_feat_sets: '["none", "default", "all"]'
@@ -27,7 +27,7 @@ jobs:
default_sys_versions: '["testing-slim"]'
default_rust_targets: '["x86_64-unknown-linux-gnu"]'
default_sys_targets: '["x86_64-linux-gnu"]'
default_machines: '["x86_64"]'
default_machines: '["X64"]'
outputs:
cargo_profiles: ${{vars.CARGO_PROFILES || env.default_cargo_profiles}}
@@ -40,8 +40,19 @@ jobs:
machines: ${{vars.MACHINES || env.default_machines}}
release_url: ${{steps.release.outputs.upload_url}}
strategy:
fail-fast: true
matrix:
runner: ${{fromJSON(vars.RUNNERS)}}
machine: ${{fromJSON(vars.MACHINES)}}
exclude: ${{fromJSON(vars.RUNNERS_EXCLUDES)}}
steps:
- name: Initialize Builder
env:
runner: ${{matrix.runner}}
reserved_space: '{"het": "256GB", "aws": "48GB", "gcp": "160GB"}'
max_used_space: '{"het": "448GB", "aws": "64GB", "gcp": "192GB"}'
run: |
set +e
docker buildx inspect "${GITHUB_ACTOR}"
@@ -49,7 +60,9 @@ jobs:
exit 0
fi
set -e
set -eux
reserved_space=$(echo -n "$reserved_space" | jq -r ".$runner")
max_used_space=$(echo -n "$max_used_space" | jq -r ".$runner")
cat <<EOF > ./buildkitd.toml
[system]
platformsCacheMaxAge = "504h"
@@ -57,11 +70,11 @@ jobs:
enabled = true
rootless = false
gc = true
reservedSpace = "256GB"
maxUsedSpace = "576GB"
reservedSpace = "${reserved_space}"
maxUsedSpace = "${max_used_space}"
[[worker.oci.gcpolicy]]
reservedSpace = "256GB"
maxUsedSpace = "576GB"
reservedSpace = "${reserved_space}"
maxUsedSpace = "${max_used_space}"
filters = ["label!=cache==pin"]
all = true
EOF
@@ -76,6 +89,7 @@ jobs:
- if: >
!failure() && !cancelled()
&& contains(github.ref, 'refs/tags/v')
&& matrix.runner == 'het'
id: release
name: Create Release
@@ -85,32 +99,10 @@ jobs:
with:
release_name: Release ${{github.ref_name}}
tag_name: ${{github.ref}}
body_path: NEWS.md
body_path: RELEASE.md
draft: ${{ contains(github.ref, '-draft') }}
prerelease: ${{ contains(github.ref, '-rc') }}
## commented due to https://github.com/orgs/community/discussions/18001
#deps:
# if: ${{fromJSON(vars.CI_VERBOSE_DEPS || 'false')}}
# name: Deps
# needs: [init]
# uses: ./.github/workflows/deps.yml
# with:
# cargo_profiles: ${{needs.init.outputs.cargo_profiles}}
# feat_sets: ${{needs.init.outputs.feat_sets}}
# rust_toolchains: ${{needs.init.outputs.rust_toolchains}}
# rust_targets: ${{needs.init.outputs.rust_targets}}
# sys_names: ${{needs.init.outputs.sys_names}}
# sys_targets: ${{needs.init.outputs.sys_targets}}
# sys_versions: ${{needs.init.outputs.sys_versions}}
# machines: ${{needs.init.outputs.machines}}
# excludes: >
# [
# {"cargo_profile": "release-debuginfo", "rust_toolchain": "nightly"},
# {"cargo_profile": "release-native", "rust_toolchain": "stable"},
# {"cargo_profile": "release-native", "feat_set": "none"}
# ]
lint:
if: >
!failure() && !cancelled()
@@ -130,13 +122,22 @@ jobs:
machines: ${{needs.init.outputs.machines}}
excludes: >
[
{"cargo_profile": "test", "rust_toolchain": "stable", "feat_set": "none"},
{"cargo_profile": "test", "rust_target": "aarch64-unknown-linux-gnu"},
{"cargo_profile": "release", "rust_toolchain": "nightly", "feat_set": "none"},
{"cargo_profile": "release", "rust_toolchain": "nightly", "feat_set": "default"},
{"cargo_profile": "release-debuginfo"},
{"cargo_profile": "release-native", "rust_toolchain": "stable"},
{"cargo_profile": "release-native", "feat_set": "none"},
{"cargo_profile": "release-native", "feat_set": "default"},
{"cargo_profile": "test", "rust_toolchain": "stable", "feat_set": "none"},
{"cargo_profile": "release", "rust_toolchain": "nightly", "feat_set": "none"},
{"cargo_profile": "release", "rust_toolchain": "nightly", "feat_set": "default"}
{"cargo_profile": "release-native", "rust_target": "aarch64-unknown-linux-gnu"},
{"rust_target": "aarch64-unknown-linux-gnu", "feat_set": "none"},
{"rust_target": "aarch64-unknown-linux-gnu", "feat_set": "default"},
{"rust_target": "aarch64-unknown-linux-gnu", "rust_toolchain": "nightly"},
{"rust_target": "aarch64-unknown-linux-gnu", "sys_target": "x86_64-linux-gnu"},
{"rust_target": "x86_64-unknown-linux-gnu", "sys_target": "aarch64-linux-gnu"},
{"sys_target": "x86_64-linux-gnu", "machine": "ARM64"},
{"sys_target": "aarch64-linux-gnu", "machine": "X64"},
]
test:
@@ -157,13 +158,23 @@ jobs:
sys_targets: ${{needs.init.outputs.sys_targets}}
sys_versions: ${{needs.init.outputs.sys_versions}}
machines: ${{needs.init.outputs.machines}}
complement_runner: 'het'
excludes: >
[
{"cargo_profile": "test", "rust_toolchain": "stable", "feat_set": "none"},
{"cargo_profile": "test", "rust_target": "aarch64-unknown-linux-gnu"},
{"cargo_profile": "release-debuginfo"},
{"cargo_profile": "release-native", "rust_toolchain": "stable"},
{"cargo_profile": "release-native", "feat_set": "default"},
{"cargo_profile": "release-native", "feat_set": "none"},
{"cargo_profile": "release-debuginfo"}
{"cargo_profile": "release-native", "rust_target": "aarch64-unknown-linux-gnu"},
{"rust_target": "aarch64-unknown-linux-gnu", "feat_set": "none"},
{"rust_target": "aarch64-unknown-linux-gnu", "feat_set": "default"},
{"rust_target": "aarch64-unknown-linux-gnu", "rust_toolchain": "nightly"},
{"rust_target": "aarch64-unknown-linux-gnu", "sys_target": "x86_64-linux-gnu"},
{"rust_target": "x86_64-unknown-linux-gnu", "sys_target": "aarch64-linux-gnu"},
{"sys_target": "x86_64-linux-gnu", "machine": "ARM64"},
{"sys_target": "aarch64-linux-gnu", "machine": "X64"},
]
package:
@@ -190,12 +201,19 @@ jobs:
{"cargo_profile": "release-debuginfo", "feat_set": "default"},
{"cargo_profile": "release-debuginfo", "rust_toolchain": "nightly"},
{"cargo_profile": "release-debuginfo", "bake_target": "oci"},
{"cargo_profile": "release-debuginfo", "rust_target": "aarch64-unknown-linux-gnu"},
{"cargo_profile": "release-native", "feat_set": "default"},
{"cargo_profile": "release-native", "rust_toolchain": "stable"},
{"cargo_profile": "release-native", "feat_set": "default"},
{"cargo_profile": "release-native", "bake_target": "docker"},
{"cargo_profile": "release-native", "bake_target": "oci"},
{"cargo_profile": "release", "rust_toolchain": "nightly"}
{"cargo_profile": "release-native", "rust_target": "aarch64-unknown-linux-gnu"},
{"cargo_profile": "release", "rust_toolchain": "nightly"},
{"rust_target": "aarch64-unknown-linux-gnu", "feat_set": "default"},
{"rust_target": "aarch64-unknown-linux-gnu", "sys_target": "x86_64-linux-gnu"},
{"rust_target": "x86_64-unknown-linux-gnu", "sys_target": "aarch64-linux-gnu"},
{"sys_target": "x86_64-linux-gnu", "machine": "ARM64"},
{"sys_target": "aarch64-linux-gnu", "machine": "X64"},
]
publish:
@@ -203,7 +221,7 @@ jobs:
!failure() && !cancelled()
name: Publish
needs: [init, package, test]
needs: [init, test, package]
uses: ./.github/workflows/publish.yml
with:
docker_repo: ${{vars.DOCKER_REPO}}
@@ -220,9 +238,14 @@ jobs:
[
{"feat_set": "none"},
{"cargo_profile": "test"},
{"rust_toolchain": "nightly", "cargo_profile": "release"},
{"cargo_profile": "release-debuginfo"},
{"cargo_profile": "release-native"}
{"cargo_profile": "release-native"},
{"cargo_profile": "release", "rust_toolchain": "nightly"},
{"rust_target": "aarch64-unknown-linux-gnu", "feat_set": "default"},
{"rust_target": "aarch64-unknown-linux-gnu", "sys_target": "x86_64-linux-gnu"},
{"rust_target": "x86_64-unknown-linux-gnu", "sys_target": "aarch64-linux-gnu"},
{"sys_target": "x86_64-linux-gnu", "machine": "ARM64"},
{"sys_target": "aarch64-linux-gnu", "machine": "X64"},
]
secrets:
+11 -9
View File
@@ -46,6 +46,7 @@ jobs:
&& contains(fromJSON(inputs.cargo_profiles), fromJSON('["release"]')[0])
&& contains(fromJSON(inputs.feat_sets), fromJSON('["default"]')[0])
&& contains(fromJSON(inputs.rust_toolchains), fromJSON('["stable"]')[0])
&& contains(fromJSON(inputs.machines), fromJSON('["X64"]')[0])
name: Book
uses: ./.github/workflows/bake.yml
@@ -58,7 +59,8 @@ jobs:
sys_names: ${{inputs.sys_names}}
sys_targets: ${{inputs.sys_targets}}
sys_versions: ${{inputs.sys_versions}}
machines: ${{inputs.machines}}
machines: '["X64"]'
runner: 'het'
excludes: ${{inputs.excludes}}
includes: ${{inputs.includes}}
release_url: ${{inputs.release_url}}
@@ -96,7 +98,8 @@ jobs:
"dst": "tuwunel.zst",
"src": "/usr/bin/tuwunel",
"mime": "application/zstd",
"zstd": 11
"zstd": 11,
"chmod": "a+x"
}
}
@@ -141,11 +144,12 @@ jobs:
&& contains(fromJSON(inputs.feat_sets), fromJSON('["all"]')[0])
&& contains(fromJSON(inputs.cargo_profiles), fromJSON('["release"]')[0])
&& contains(fromJSON(inputs.rust_toolchains), fromJSON('["stable"]')[0])
&& github.ref == 'refs/heads/main'
name: Distro Packages
uses: ./.github/workflows/bake.yml
with:
bake_targets: '["pkg-deb", "pkg-rpm"]'
bake_targets: '["deb", "rpm"]'
cargo_profiles: '["release"]'
feat_sets: '["all"]'
rust_toolchains: '["stable"]'
@@ -159,14 +163,12 @@ jobs:
release_url: ${{inputs.release_url}}
artifact: >
{
"pkg-deb": {
"deb": {
"dst": "tuwunel.deb",
"src": "tuwunel_1.0.0-1_amd64.deb",
"mime": "application/vnd.debian.binary-package"
},
"pkg-rpm": {
"rpm": {
"dst": "tuwunel.rpm",
"src": "tuwunel-1.0.0-1.x86_64.rpm",
"mime": "application/x-rpm"
}
}
@@ -177,13 +179,13 @@ jobs:
&& contains(fromJSON(inputs.feat_sets), fromJSON('["all"]')[0])
&& contains(fromJSON(inputs.cargo_profiles), fromJSON('["release"]')[0])
&& contains(fromJSON(inputs.rust_toolchains), fromJSON('["stable"]')[0])
&& false
&& github.ref == 'refs/heads/main'
name: Check Packages
needs: [pkgs]
uses: ./.github/workflows/bake.yml
with:
bake_targets: '["pkg-deb-install", "pkg-rpm-install"]'
bake_targets: '["deb-install", "rpm-install"]'
cargo_profiles: '["release"]'
feat_sets: '["all"]'
rust_toolchains: '["stable"]'
+4 -4
View File
@@ -49,12 +49,12 @@ jobs:
documents:
if: >
!failure() && !cancelled()
&& inputs.machines
&& contains(fromJSON(inputs.machines), fromJSON('["X64"]')[0])
&& (github.ref == 'refs/heads/main' || contains(github.ref, 'refs/tags/v'))
&& !contains(github.ref, '-draft')
name: Documents
runs-on: self-hosted
runs-on: ['X64', 'het']
permissions:
pages: write
id-token: write
@@ -120,8 +120,8 @@ jobs:
acct: ${{github.actor}}
repo: ${{github.repository}}
docker_repo: ${{inputs.docker_repo}}
docker_tag_preview: ${{ contains(github.ref, 'refs/tags/v') }}
docker_tag_latest: ${{ contains(github.ref, 'refs/tags/v') && !contains(github.ref, '-rc') }}
docker_tag_preview: ${{ matrix.cargo_profile == 'release' && matrix.feat_set == 'all' && matrix.sys_target == 'x86_64-linux-gnu' && contains(github.ref, 'refs/tags/v') }}
docker_tag_latest: ${{ matrix.cargo_profile == 'release' && matrix.feat_set == 'all' && matrix.sys_target == 'x86_64-linux-gnu' && contains(github.ref, 'refs/tags/v') && !contains(github.ref, '-rc') }}
run: |
docker/bake.sh "${{matrix.bake_target}}"
+32 -10
View File
@@ -41,6 +41,8 @@ on:
complement_feat_sets:
type: string
default: '["all"]'
complement_runner:
type: string
jobs:
unit:
@@ -48,6 +50,7 @@ jobs:
contains(fromJSON(inputs.cargo_profiles), fromJSON('["test"]')[0])
&& contains(fromJSON(inputs.feat_sets), fromJSON('["all"]')[0])
&& contains(fromJSON(inputs.rust_toolchains), fromJSON('["nightly"]')[0])
&& contains(fromJSON(inputs.machines), fromJSON('["X64"]')[0])
name: Unit
uses: ./.github/workflows/bake.yml
@@ -60,12 +63,16 @@ jobs:
sys_versions: ${{inputs.sys_versions}}
rust_targets: ${{inputs.rust_targets}}
sys_targets: ${{inputs.sys_targets}}
machines: ${{inputs.machines}}
machines: '["X64"]'
excludes: ${{inputs.excludes}}
includes: ${{inputs.includes}}
smoke:
if: inputs.cargo_profiles
if: >
!failure() && !cancelled()
&& inputs.cargo_profiles
&& inputs.machines
name: Smoke
uses: ./.github/workflows/bake.yml
with:
@@ -82,26 +89,37 @@ jobs:
excludes: >
[
{"cargo_profile": "test", "feat_set": "default"},
{"cargo_profile": "test", "rust_toolchain": "stable"},
{"cargo_profile": "test", "feat_set": "none", "bake_target": "smoke-valgrind"},
{"cargo_profile": "test", "rust_toolchain": "stable"},
{"cargo_profile": "test", "rust_target": "aarch64-unknown-linux-gnu"},
{"cargo_profile": "release", "rust_toolchain": "nightly"},
{"cargo_profile": "release", "rust_toolchain": "stable", "feat_set": "none"},
{"cargo_profile": "release", "bake_target": "smoke-valgrind"},
{"cargo_profile": "release-debuginfo", "rust_toolchain": "nightly"},
{"cargo_profile": "release-debuginfo", "rust_toolchain": "stable", "feat_set": "none"},
{"cargo_profile": "release-debuginfo", "bake_target": "smoke"},
{"cargo_profile": "release-debuginfo", "rust_target": "aarch64-unknown-linux-gnu"},
{"cargo_profile": "release-native", "rust_toolchain": "stable"},
{"cargo_profile": "release-native", "feat_set": "none"},
{"cargo_profile": "release-native", "feat_set": "default"},
{"cargo_profile": "release-native", "bake_target": "smoke-valgrind"}
{"cargo_profile": "release-native", "bake_target": "smoke-valgrind"},
{"cargo_profile": "release-native", "rust_target": "aarch64-unknown-linux-gnu"},
{"rust_target": "aarch64-unknown-linux-gnu", "feat_set": "none"},
{"rust_target": "aarch64-unknown-linux-gnu", "feat_set": "default"},
{"rust_target": "aarch64-unknown-linux-gnu", "sys_target": "x86_64-linux-gnu"},
{"rust_target": "x86_64-unknown-linux-gnu", "sys_target": "aarch64-linux-gnu"},
{"sys_target": "x86_64-linux-gnu", "machine": "ARM64"},
{"sys_target": "aarch64-linux-gnu", "machine": "X64"},
]
complement:
if: >
inputs.complement
!failure() && !cancelled()
&& inputs.complement
&& contains(fromJSON(inputs.cargo_profiles), fromJSON('["test"]')[0])
&& contains(fromJSON(inputs.complement_feat_sets), fromJSON('["all"]')[0])
&& contains(fromJSON(inputs.rust_toolchains), fromJSON('["nightly"]')[0])
&& contains(fromJSON(inputs.machines), fromJSON('["X64"]')[0])
name: Complement
uses: ./.github/workflows/bake.yml
@@ -114,20 +132,24 @@ jobs:
sys_versions: ${{inputs.sys_versions}}
rust_targets: ${{inputs.rust_targets}}
sys_targets: ${{inputs.sys_targets}}
machines: ${{inputs.machines}}
machines: '["X64"]'
runner: ${{inputs.complement_runner}}
excludes: ${{inputs.excludes}}
includes: ${{inputs.includes}}
compliance:
if: >
inputs.complement
!failure() && !cancelled()
&& inputs.complement
&& inputs.machines
&& contains(fromJSON(inputs.cargo_profiles), fromJSON('["test"]')[0])
&& contains(fromJSON(inputs.feat_sets), fromJSON('["all"]')[0])
&& contains(fromJSON(inputs.rust_toolchains), fromJSON('["nightly"]')[0])
&& contains(fromJSON(inputs.machines), fromJSON('["X64"]')[0])
name: Matrix Compliance
needs: [complement]
runs-on: ${{matrix.machine}}
runs-on: ["${{matrix.machine}}", "${{inputs.complement_runner}}"]
concurrency:
group: complement-cant-walk-and-chew-bubblegum
cancel-in-progress: false
@@ -142,7 +164,7 @@ jobs:
sys_version: ${{fromJSON(inputs.sys_versions)}}
rust_target: ${{fromJSON(inputs.rust_targets)}}
sys_target: ${{fromJSON(inputs.sys_targets)}}
machine: ${{fromJSON(inputs.machines)}}
machine: ${{fromJSON('["X64"]')}}
exclude: ${{fromJSON(inputs.excludes)}}
include: ${{fromJSON(inputs.includes)}}
@@ -177,7 +199,7 @@ jobs:
id: upload
uses: actions/upload-artifact@v4
with:
name: test_results.${{matrix.feat_set}}.${{matrix.sys_name}}.${{matrix.sys_target}}.jsonl
name: test_results-${{matrix.feat_set}}-${{matrix.sys_name}}-${{matrix.sys_target}}.jsonl
path: ./tests/test_results/complement/test_results.jsonl
- name: Accept
Generated
+575 -714
View File
File diff suppressed because it is too large. Load diff
+60 -52
View File
@@ -29,7 +29,7 @@ license = "Apache-2.0"
readme = "README.md"
repository = "https://github.com/matrix-construct/tuwunel"
rust-version = "1.86.0"
version = "1.0.0"
version = "1.1.0"
[workspace.metadata.crane]
name = "tuwunel"
@@ -57,7 +57,7 @@ version = "2.3.1"
version = "0.1.88"
[workspace.dependencies.axum]
version = "0.7.9"
version = "0.8.4"
default-features = false
features = [
"form",
@@ -70,10 +70,10 @@ features = [
]
[workspace.dependencies.axum-client-ip]
version = "0.6.1"
version = "0.7"
[workspace.dependencies.axum-extra]
version = "0.9.6"
version = "0.10.1"
default-features = false
features = [
"typed-header",
@@ -86,6 +86,7 @@ default-features = false
[workspace.dependencies.axum-server-dual-protocol]
version = "0.7"
default-features = false
[workspace.dependencies.base64]
version = "0.22.1"
@@ -103,10 +104,10 @@ features = [
version = "1.10.1"
[workspace.dependencies.bytesize]
version = "2.0"
version = "2.0.1"
[workspace.dependencies.cargo_toml]
version = "0.21"
version = "0.22.1"
default-features = false
features = ["features"]
@@ -114,12 +115,12 @@ features = ["features"]
version = "0.1"
[workspace.dependencies.chrono]
version = "0.4.38"
version = "0.4.41"
features = ["alloc", "std"]
default-features = false
[workspace.dependencies.clap]
version = "4.5.35"
version = "4.5.39"
default-features = false
features = [
"derive",
@@ -131,17 +132,18 @@ features = [
"usage",
]
[workspace.dependencies.console-subscriber]
version = "0.4"
# Disabled until Axum updated. Remember to remove cfgs for tuwunel_disable.
#[workspace.dependencies.console-subscriber]
#version = "0.4"
[workspace.dependencies.core_affinity]
version = "0.8.1"
version = "0.8.3"
[workspace.dependencies.const-str]
version = "0.6.2"
[workspace.dependencies.ctor]
version = "0.2.9"
version = "0.4.2"
[workspace.dependencies.cyborgtime]
version = "2.1.1"
@@ -171,7 +173,7 @@ features = [
]
[workspace.dependencies.hickory-resolver]
version = "0.25.1"
version = "0.25.2"
default-features = false
features = [
"serde",
@@ -199,7 +201,7 @@ features = [
]
[workspace.dependencies.hyper-util]
version = "0.1.11"
version = "0.1.14"
default-features = false
features = [
"server-auto",
@@ -208,7 +210,7 @@ features = [
]
[workspace.dependencies.image]
version = "0.25.5"
version = "0.25.6"
default-features = false
features = [
"jpeg",
@@ -223,6 +225,11 @@ version = "0.1.3"
[workspace.dependencies.itertools]
version = "0.14.0"
[workspace.dependencies.jsonwebtoken]
version = "9.3.1"
default-features = false
features = ["use_pem"]
[workspace.dependencies.ldap3]
git = "https://github.com/matrix-construct/ldap3"
rev = "7d423314b9dbc66347284e38fc2b78c3d8f3d494"
@@ -230,17 +237,17 @@ default-features = false
features = ["sync", "tls-rustls"]
[workspace.dependencies.libc]
version = "0.2"
version = "0.2.172"
[workspace.dependencies.libloading]
version = "0.8.6"
version = "0.8.8"
[workspace.dependencies.log]
version = "0.4.27"
default-features = false
[workspace.dependencies.loole]
version = "0.4.0"
version = "0.4.1"
[workspace.dependencies.lru-cache]
version = "0.1.2"
@@ -249,37 +256,38 @@ version = "0.1.2"
version = "1.0.2"
[workspace.dependencies.minicbor]
version = "0.26.3"
version = "1.0.0"
features = ["std"]
[workspace.dependencies.minicbor-serde]
version = "0.4.1"
version = "0.5.0"
features = ["std"]
[workspace.dependencies.nix]
version = "0.29.0"
version = "0.30.1"
default-features = false
features = ["resource"]
[workspace.dependencies.num-traits]
version = "0.2"
version = "0.2.19"
[workspace.dependencies.opentelemetry]
version = "0.21.0"
version = "0.30.0"
[workspace.dependencies.opentelemetry-jaeger]
version = "0.20.0"
features = ["rt-tokio"]
# Disabled until they move to opentelemetry 0.30
#[workspace.dependencies.opentelemetry-jaeger]
#version = "0.22.0"
#features = ["rt-tokio"]
[workspace.dependencies.opentelemetry_sdk]
version = "0.21.2"
version = "0.30.0"
features = ["rt-tokio"]
[workspace.dependencies.proc-macro2]
version = "1.0"
version = "1.0.95"
[workspace.dependencies.quote]
version = "1.0"
version = "1.0.40"
[workspace.dependencies.rand]
version = "0.8.5"
@@ -288,7 +296,7 @@ version = "0.8.5"
version = "1.11.1"
[workspace.dependencies.reqwest]
version = "0.12.15"
version = "0.12.19"
default-features = false
features = [
"rustls-tls-native-roots",
@@ -303,7 +311,7 @@ default-features = false
[workspace.dependencies.ruma]
git = "https://github.com/matrix-construct/ruma"
rev = "920148dca1076454ca0ca5d43b5ce1aa708381d4"
rev = "0155c2b33233bec9dece79d5134a9574b347f4c1"
features = [
"compat",
"rand",
@@ -341,17 +349,17 @@ features = [
]
[workspace.dependencies.rustls]
version = "0.23.25"
version = "0.23.27"
default-features = false
features = ["aws_lc_rs"]
[workspace.dependencies.rustyline-async]
version = "0.4.3"
version = "0.4.6"
default-features = false
[workspace.dependencies.rust-rocksdb]
git = "https://github.com/matrix-construct/rust-rocksdb"
rev = "cca241c5ca48f7c7f5c1469294f11dc156dd987a"
rev = "225a42519276e502205bdc845cebdb22d70ee245"
default-features = false
features = [
"multi-threaded-cf",
@@ -369,7 +377,7 @@ version = "0.4.5"
default-features = false
[workspace.dependencies.sentry]
version = "0.37.0"
version = "0.38.1"
default-features = false
features = [
"backtrace",
@@ -385,10 +393,10 @@ features = [
]
[workspace.dependencies.sentry-tower]
version = "0.37.0"
version = "0.38.1"
[workspace.dependencies.sentry-tracing]
version = "0.37.0"
version = "0.38.1"
[workspace.dependencies.serde]
version = "1.0.219"
@@ -414,7 +422,7 @@ version = "0.10.6"
default-features = false
[workspace.dependencies.sha2]
version = "0.10.8"
version = "0.10.9"
default-features = false
[workspace.dependencies.smallstr]
@@ -426,7 +434,7 @@ features = [
]
[workspace.dependencies.smallvec]
version = "1.14.0"
version = "1.15.1"
features = [
"const_generics",
"const_new",
@@ -436,7 +444,7 @@ features = [
]
[workspace.dependencies.syn]
version = "2.0"
version = "2.0.101"
default-features = false
features = [
"extra-traits",
@@ -447,7 +455,7 @@ features = [
]
[workspace.dependencies.termimad]
version = "0.31.2"
version = "0.33.0"
default-features = false
[workspace.dependencies.thiserror]
@@ -456,7 +464,7 @@ default-features = false
[workspace.dependencies.tikv-jemallocator]
git = "https://github.com/matrix-construct/jemallocator"
rev = "82af58d6a13ddd5dcdc7d4e91eae3b63292995b8"
rev = "03bed96afbbc898bef4d4f7d335c0519e3d1afad"
default-features = false
features = [
"background_threads_runtime_support",
@@ -465,13 +473,13 @@ features = [
[workspace.dependencies.tikv-jemalloc-ctl]
git = "https://github.com/matrix-construct/jemallocator"
rev = "82af58d6a13ddd5dcdc7d4e91eae3b63292995b8"
rev = "03bed96afbbc898bef4d4f7d335c0519e3d1afad"
default-features = false
features = ["use_std"]
[workspace.dependencies.tikv-jemalloc-sys]
git = "https://github.com/matrix-construct/jemallocator"
rev = "82af58d6a13ddd5dcdc7d4e91eae3b63292995b8"
rev = "03bed96afbbc898bef4d4f7d335c0519e3d1afad"
default-features = false
features = [
"background_threads_runtime_support",
@@ -479,7 +487,7 @@ features = [
]
[workspace.dependencies.tokio]
version = "1.44.2"
version = "1.45.1"
default-features = false
features = [
"fs",
@@ -494,10 +502,10 @@ features = [
]
[workspace.dependencies.tokio-metrics]
version = "0.4.0"
version = "0.4.2"
[workspace.dependencies.toml]
version = "0.8.14"
version = "0.8.23"
default-features = false
features = ["parse"]
@@ -507,7 +515,7 @@ default-features = false
features = ["util"]
[workspace.dependencies.tower-http]
version = "0.6.2"
version = "0.6.6"
default-features = false
features = [
"add-extension",
@@ -532,7 +540,7 @@ default-features = false
version = "0.2.0"
[workspace.dependencies.tracing-opentelemetry]
version = "0.22.0"
version = "0.31.0"
[workspace.dependencies.tracing-subscriber]
version = "0.3.19"
@@ -582,7 +590,7 @@ rev = "1e64095a8051a1adf0d1faa307f9f030889ec2aa"
# https://github.com/girlbossceo/rustyline-async/commit/67d8c49aeac03a5ef4e818f663eaa94dd7bf339b
[patch.crates-io.rustyline-async]
git = "https://github.com/matrix-construct/rustyline-async"
rev = "deaeb0694e2083f53d363b648da06e10fc13900c"
rev = "ba743cae940659e95d7b8604bbd6fdd755171c8c"
# adds LIFO queue scheduling; this should be updated with PR progress.
[patch.crates-io.event-listener]
@@ -595,12 +603,12 @@ rev = "92e5e74063bf2a3b10414bcc8a0d68b235644280"
# adds affinity masks for selecting more than one core at a time
[patch.crates-io.core_affinity]
git = "https://github.com/matrix-construct/core_affinity_rs"
rev = "9c8e51510c35077df888ee72a36b4b05637147da"
rev = "ff148c2985cbaff3e6e953d45cda4d7696f38d75"
# reverts hyperium#148 conflicting with our delicate federation resolver hooks
[patch.crates-io.hyper-util]
git = "https://github.com/matrix-construct/hyper-util"
rev = "e4ae7628fe4fcdacef9788c4c8415317a4489941"
rev = "c4fd9b4cd3a59fcfd28ca3802d68a8ea8988d9a3"
# allows no-aaaa option in resolv.conf
# bumps rust edition and toolchain to 1.86.0 and 2024
-17
View File
@@ -1,17 +0,0 @@
# One Tuwunel
June 6, 2025
### New Features
- LDAP authentication. Thanks @RatCornu!
- Podman systemd. Thanks @everypizza1!
### Bug Fixes
- Several panics fixed by Jade Ellis (@JadedBlueEyes) et al. Thank you 🙏
### Honorable Mentions
- Special thanks to Peter Gervai (@grinapo) for hosting the fanclub room; also
for documentation and wiki contributions.
+7 -6
View File
@@ -35,9 +35,10 @@ development. It is now maintained by full-time staff.
- [Sourcecode](https://github.com/matrix-construct/tuwunel/) `git clone https://github.com/matrix-construct/tuwunel.git`
- [DockerHub](https://hub.docker.com/r/jevolk/tuwunel) or `docker pull jevolk/tuwunel:latest`
- [GHCR](https://github.com/matrix-construct/tuwunel/pkgs/container/tuwunel) or `docker pull ghcr.io/matrix-construct/tuwunel:latest`
- Static binaries available as [releases](https://github.com/matrix-construct/tuwunel/releases) or [build artifacts](https://github.com/matrix-construct/tuwunel/actions?query=branch%3Amain).
- Deb and RPM packages available as [releases](https://github.com/matrix-construct/tuwunel/releases) or [build artifacts](https://github.com/matrix-construct/tuwunel/actions?query=branch%3Amain).
- Static binaries also available as [releases](https://github.com/matrix-construct/tuwunel/releases) or [build artifacts](https://github.com/matrix-construct/tuwunel/actions?query=branch%3Amain).
- Arch package is expected shortly. Nix package still needs some love.
- Arch package available as [tuwunel](https://aur.archlinux.org/packages/tuwunel) or [tuwunel-git](https://aur.archlinux.org/packages/tuwunel-git).
- Nix package still [needs some love](https://github.com/NixOS/nixpkgs/issues/415469).
**1.** [Configure](https://matrix-construct.github.io/tuwunel/configuration.html) by
copying and editing the `tuwunel-example.toml`. The `server_name` and `database_path` must be
@@ -145,10 +146,10 @@ the first release; we will have updates to this section. This is a fully moderat
the wellbeing of our users and create a non-toxic work environment for staff. If you require assistance
with anything that is not remedied by the documentation, don't hesitate to open an issue on github.
If you are opposed to using github, or private discussion is required, or for any other reason,
I would be happy to receive your DM at [@jason:tuwunel.me](https://matrix.to/#/@jason:tuwunel.me),
you will not be bothering me and it would be my pleasure to help you anytime. As an emergency contact
you can send an email to jasonzemos@gmail.com.
If you are opposed to using github, or if private discussion is required such as for security
disclosures, or for any other reason, I would be happy to receive your DM at
[@jason:tuwunel.me](https://matrix.to/#/@jason:tuwunel.me). This will not be bothering me as it would
be my pleasure to help you anytime. As an emergency contact you can send an email to jasonzemos@gmail.com.
##### Tuwunel Fanclub
+37
View File
@@ -0,0 +1,37 @@
# Tuwunel 1.1.0
June 19, 2025
All dependencies have been fully upgraded for the first time since the
conduwuit transition. RocksDB is now synchronized to 10.2.1-tuwunel for all
builders. The Nix build itself has now been fully migrated from conduwuit;
special thanks to @wkordalski for making this happen. Thanks to @Askhalion
for opening a NixOS package request which you can [vote for here](https://github.com/NixOS/nixpkgs/issues/415469).
An [Arch package](https://aur.archlinux.org/packages/tuwunel) has also been
created in the AUR courtesy of @drrossum in addition to the transitional
package setup by @Kimiblock which we failed to acknowledge during the first
release. The RPM package now has systemd and proper installation added thanks
to a report by @alythemonk.
ARMv8 builds are now supported and bundled with this release. Thanks to
@zaninime and @clement-escolano for reminding us.
JSON Web Token logins are now supported. This feature was commissioned and
made public by an enterprise sponsor. The type `org.matrix.login.jwt` is now
recognized.
### New Features
- JWT login support.
### Follow-up Features
- aarch64 build and packages.
- NixOS build support. (thanks @wkordalski and @coolGi69)
- Dependency upgrades, including Axum 0.8. (thanks @dasha_uwu)
- RPM package systemd and proper installation scripts.
### Bug Fixes
- Changing passwords for pre-migration users was precluded by an error.
Special thanks to @teidesu for making a superb report about this.
+1 -1
View File
@@ -59,7 +59,7 @@ StateDirectory=tuwunel
RuntimeDirectory=tuwunel
RuntimeDirectoryMode=0750
Environment="CONDUWUIT_CONFIG=/etc/tuwunel/tuwunel.toml"
Environment="TUWUNEL_CONFIG=/etc/tuwunel/tuwunel.toml"
BindPaths=/var/lib/private/tuwunel:/var/lib/conduwuit
BindPaths=/var/lib/private/tuwunel:/var/lib/private/conduwuit
BindPaths=/var/lib/private/tuwunel:/var/lib/matrix-conduit
+50 -9
View File
@@ -4,6 +4,7 @@ FROM input AS cargo
ARG sys_target
ARG rust_target
ARG rust_toolchain
ARG RUST_HOME
ARG RUSTUP_HOME
ARG CARGO_HOME
ARG CARGO_TARGET
@@ -11,25 +12,56 @@ ARG CARGO_TARGET_DIR
ARG cargo_target_profile
ARG cargo_target_artifact
ARG cargo_target_share
ARG CARGO_TERM_VERBOSE=0
ARG RUST_BACKTRACE
ARG JEMALLOC_OVERRIDE
ARG ROCKSDB_LIB_DIR
ARG CARGO_BUILD_RUSTFLAGS
ARG CARGO_PROFILE_TEST_DEBUG
ARG CARGO_PROFILE_TEST_INCREMENTAL
ARG CARGO_PROFILE_BENCH_DEBUG
ARG CARGO_PROFILE_BENCH_LTO
ARG CARGO_PROFILE_RELEASE_LTO
ARG CARGO_PROFILE_RELEASE_DEBUGINFO_DEBUG
ARG CARGO_PROFILE_RELEASE_DEBUGINFO_LTO
ARG cargo_profile
ARG cargo_features
ARG cargo_spec_features
ARG cargo_cmd
ARG color_args="--color always"
ARG recipe_args=""
ARG cargo_args=""
ARG git_checkout
WORKDIR /
COPY --link --from=input . .
COPY --link --from=rust ${RUST_HOME} ${RUST_HOME}
COPY --link --from=deps /usr/src/tuwunel /usr/src/tuwunel
WORKDIR /usr/lib/${sys_target}
COPY --link --from=rocksdb . .
WORKDIR /usr/src/tuwunel
ENV PATH="${CARGO_HOME}/bin:$PATH"
ENV CARGO_TERM_VERBOSE="${CARGO_TERM_VERBOSE}"
ENV RUST_BACKTRACE="${RUST_BACKTRACE}"
ENV JEMALLOC_OVERRIDE="${JEMALLOC_OVERRIDE}"
ENV ROCKSDB_LIB_DIR="${ROCKSDB_LIB_DIR}"
ENV CARGO_PROFILE_TEST_DEBUG="${CARGO_PROFILE_TEST_DEBUG}"
ENV CARGO_PROFILE_TEST_INCREMENTAL="${CARGO_PROFILE_TEST_INCREMENTAL}"
ENV CARGO_PROFILE_BENCH_DEBUG="${CARGO_PROFILE_BENCH_DEBUG}"
ENV CARGO_PROFILE_BENCH_LTO="${CARGO_PROFILE_BENCH_LTO}"
ENV CARGO_PROFILE_RELEASE_LTO="${CARGO_PROFILE_RELEASE_LTO}"
ENV CARGO_PROFILE_RELEASE_DEBUGINFO_DEBUG="${CARGO_PROFILE_RELEASE_DEBUGINFO_DEBUG}"
ENV CARGO_PROFILE_RELEASE_DEBUGINFO_LTO="${CARGO_PROFILE_RELEASE_DEBUGINFO_LTO}"
ENV CARGO_BUILD_RUSTFLAGS="${CARGO_BUILD_RUSTFLAGS}"
ENV CARGO_TARGET_DIR="${CARGO_TARGET_DIR}"
ENV targ_dir="${CARGO_TARGET_DIR}/${cargo_target_profile}"
ENV targ_targ_dir="${CARGO_TARGET_DIR}/${rust_target}/${cargo_target_profile}"
RUN \
--mount=type=cache,dst=${RUSTUP_HOME}/downloads,sharing=shared \
--mount=type=cache,dst=${CARGO_HOME}/registry,sharing=shared \
--mount=type=cache,dst=${CARGO_HOME}/git,sharing=shared \
--mount=type=cache,dst=${RUSTUP_HOME}/downloads,sharing=shared,ro \
--mount=type=cache,dst=${CARGO_HOME}/registry,sharing=shared,ro \
--mount=type=cache,dst=${CARGO_HOME}/git,sharing=shared,ro \
--mount=type=cache,dst=${targ_dir}/deps,id=${cargo_target_share}/deps,sharing=locked \
--mount=type=cache,dst=${targ_dir}/build,id=${cargo_target_share}/build,sharing=locked \
--mount=type=cache,dst=${targ_dir}/examples,id=${cargo_target_share}/examples,sharing=locked \
@@ -48,11 +80,20 @@ RUN \
--locked \
--workspace \
"${cargo_spec_features}" \
--features "${cargo_features}" \
--profile "${cargo_profile}" \
--target "${CARGO_TARGET}" \
--target-dir "${CARGO_TARGET_DIR}" \
--manifest-path Cargo.toml \
--color always \
"--features=${cargo_features}" \
"--profile=${cargo_profile}" \
"--target=${CARGO_TARGET}" \
"--target-dir=${CARGO_TARGET_DIR}" \
"--manifest-path=Cargo.toml" \
${recipe_args} \
${color_args} \
${cargo_args}
# If this image is further reused with other cargo commands, all
# modifications made by cargo chef cook outside of target-dir have to be
# cleared. If not, resulting build artifacts will link incorrectly, even
# without error. For example, a target executable may be produced which
# does nothing except exit(0). If you have observed a smoketest failing in
# such a manner, investigate this as a cause of the issue.
git restore -W -S --source=${git_checkout} .
EOF
+11 -10
View File
@@ -1,6 +1,6 @@
# syntax = docker/dockerfile:1.11-labs
FROM input AS deb
FROM input AS build-deb
ARG sys_target
ARG rust_target
ARG rust_toolchain
@@ -55,28 +55,29 @@ RUN \
--output "${pkg_dir}" \
${deb_args}
dpkg --info ${pkg_dir}/*
dpkg --contents ${pkg_dir}/*
mv ${pkg_dir}/* "${pkg_dir}/tuwunel.deb"
dpkg --info "${pkg_dir}/tuwunel.deb"
dpkg --contents "${pkg_dir}/tuwunel.deb"
EOF
FROM scratch AS pkg-deb
FROM scratch AS deb
ARG pkg_dir
COPY --from=input ${pkg_dir}/* .
COPY --from=input ${pkg_dir}/tuwunel.deb .
FROM input AS pkg-deb-install
FROM input AS deb-install
ARG pkg_dir
WORKDIR /
COPY --link --from=input . .
WORKDIR ${pkg_dir}
COPY --from=pkg-deb . .
COPY --from=deb . .
RUN <<EOF
set -eux
dpkg --info *
dpkg --contents *
dpkg --install *
dpkg --info tuwunel.deb
dpkg --contents tuwunel.deb
dpkg --install tuwunel.deb
EOF
-84
View File
@@ -1,84 +0,0 @@
# syntax = docker/dockerfile:1.11-labs
FROM input AS deps
ARG sys_target
ARG rust_target
ARG rust_toolchain
ARG RUSTUP_HOME
ARG CARGO_HOME
ARG CARGO_TARGET
ARG CARGO_TARGET_DIR
ARG cargo_target_profile
ARG cargo_target_artifact
ARG cargo_target_share
ARG CARGO_BUILD_RUSTFLAGS
ARG CARGO_PROFILE_TEST_DEBUG
ARG CARGO_PROFILE_TEST_INCREMENTAL
ARG CARGO_PROFILE_BENCH_DEBUG
ARG CARGO_PROFILE_BENCH_LTO
ARG CARGO_PROFILE_RELEASE_LTO
ARG CARGO_PROFILE_RELEASE_DEBUGINFO_DEBUG
ARG CARGO_PROFILE_RELEASE_DEBUGINFO_LTO
ARG cargo_profile
ARG cargo_features
ARG cargo_spec_features
ARG cook_args
ARG git_checkout
WORKDIR /
COPY --link --from=input . .
WORKDIR /usr/lib/${sys_target}
COPY --link --from=rocksdb . .
WORKDIR /usr/src/tuwunel
COPY --link --from=recipe recipe.json .
ENV CARGO_PROFILE_TEST_DEBUG="${CARGO_PROFILE_TEST_DEBUG}"
ENV CARGO_PROFILE_TEST_INCREMENTAL="${CARGO_PROFILE_TEST_INCREMENTAL}"
ENV CARGO_PROFILE_BENCH_DEBUG="${CARGO_PROFILE_BENCH_DEBUG}"
ENV CARGO_PROFILE_BENCH_LTO="${CARGO_PROFILE_BENCH_LTO}"
ENV CARGO_PROFILE_RELEASE_LTO="${CARGO_PROFILE_RELEASE_LTO}"
ENV CARGO_PROFILE_RELEASE_DEBUGINFO_DEBUG="${CARGO_PROFILE_RELEASE_DEBUGINFO_DEBUG}"
ENV CARGO_PROFILE_RELEASE_DEBUGINFO_LTO="${CARGO_PROFILE_RELEASE_DEBUGINFO_LTO}"
ENV CARGO_BUILD_RUSTFLAGS="${CARGO_BUILD_RUSTFLAGS}"
ENV CARGO_TARGET_DIR="${CARGO_TARGET_DIR}"
ENV targ_dir="${CARGO_TARGET_DIR}/${cargo_target_profile}"
ENV targ_targ_dir="${CARGO_TARGET_DIR}/${rust_target}/${cargo_target_profile}"
RUN \
--mount=type=cache,dst=${RUSTUP_HOME}/downloads,sharing=shared \
--mount=type=cache,dst=${CARGO_HOME}/registry,sharing=shared \
--mount=type=cache,dst=${CARGO_HOME}/git,sharing=shared \
--mount=type=cache,dst=${targ_dir}/deps,id=${cargo_target_share}/deps,sharing=locked \
--mount=type=cache,dst=${targ_dir}/build,id=${cargo_target_share}/build,sharing=locked \
--mount=type=cache,dst=${targ_dir}/examples,id=${cargo_target_share}/examples,sharing=locked \
--mount=type=cache,dst=${targ_dir}/incremental,id=${cargo_target_share}/incremental,sharing=locked \
--mount=type=cache,dst=${targ_dir}/.fingerprint,id=${cargo_target_share}/fingerprint,sharing=locked \
--mount=type=cache,dst=${targ_targ_dir}/deps,id=${cargo_target_share}/deps,sharing=locked \
--mount=type=cache,dst=${targ_targ_dir}/build,id=${cargo_target_share}/build,sharing=locked \
--mount=type=cache,dst=${targ_targ_dir}/examples,id=${cargo_target_share}/examples,sharing=locked \
--mount=type=cache,dst=${targ_targ_dir}/incremental,id=${cargo_target_share}/incremental,sharing=locked \
--mount=type=cache,dst=${targ_targ_dir}/.fingerprint,id=${cargo_target_share}/fingerprint,sharing=locked \
<<EOF
set -eux
rustup run ${rust_toolchain} \
cargo chef cook ${cook_args} \
--verbose \
--locked \
--workspace \
"${cargo_spec_features}" \
--features "${cargo_features}" \
--profile "${cargo_profile}" \
--target "${CARGO_TARGET}" \
--target-dir "${CARGO_TARGET_DIR}" \
--manifest-path Cargo.toml \
--recipe-path recipe.json
# If this image is further reused with other cargo commands, all
# modifications made by cargo chef cook outside of target-dir have to be
# cleared. If not, resulting build artifacts will link incorrectly, even
# without error. For example, a target executable may be produced which
# does nothing except exit(0). If you have observed a smoketest failing in
# such a manner, investigate this as a cause of the issue.
git restore -W -S --source=${git_checkout} .
EOF
+9 -7
View File
@@ -1,6 +1,6 @@
# syntax = docker/dockerfile:1.11-labs
FROM input AS rpm
FROM input AS build-rpm
ARG sys_target
ARG rust_toolchain
ARG RUSTUP_HOME
@@ -47,25 +47,27 @@ RUN \
--payload-compress zstd \
--output "${pkg_dir}" \
${gen_rpm_args}
mv ${pkg_dir}/* "${pkg_dir}/tuwunel.rpm"
EOF
FROM scratch AS pkg-rpm
FROM scratch AS rpm
ARG pkg_dir
COPY --link --from=input ${pkg_dir}/* .
COPY --from=input ${pkg_dir}/tuwunel.rpm .
FROM input AS pkg-rpm-install
FROM input AS rpm-install
ARG pkg_dir
WORKDIR /
COPY --link --from=input . .
WORKDIR ${pkg_dir}
COPY --link --from=pkg-rpm . .
COPY --link --from=rpm . .
RUN <<EOF
set -eux
rpm -i --test *
rpm -i *
rpm -i --test tuwunel.rpm
rpm -i tuwunel.rpm
EOF
+4 -4
View File
@@ -8,8 +8,8 @@ WORKDIR /
COPY --link --from=input . .
RUN \
--mount=type=cache,dst=${var_cache},sharing=locked \
--mount=type=cache,dst=${var_lib_apt},sharing=locked \
--mount=type=cache,dst=/var/cache,id=${var_cache},sharing=locked \
--mount=type=cache,dst=/var/lib/apt,id=${var_lib_apt},sharing=locked \
<<EOF
set -eux
apt-get -y -U install --no-install-recommends openssl gawk
@@ -156,8 +156,8 @@ ARG complement_run=".*"
WORKDIR /
RUN \
--mount=type=cache,dst=${var_cache},sharing=locked \
--mount=type=cache,dst=${var_lib_apt},sharing=locked \
--mount=type=cache,dst=/var/cache,id=${var_cache},sharing=locked \
--mount=type=cache,dst=/var/lib/apt,id=${var_lib_apt},sharing=locked \
--mount=type=cache,dst=/go/pkg/mod/cache,sharing=locked \
<<EOF
set -eux
+8
View File
@@ -5,6 +5,7 @@ ARG rust_target
ARG CARGO_TARGET_DIR
ARG cargo_target_profile
ARG install_prefix
ARG assert_linkage=""
WORKDIR /
COPY --link --from=input . .
@@ -14,6 +15,13 @@ ENV src_path="${CARGO_TARGET_DIR}/${rust_target}/${cargo_target_profile}/tuwunel
ENV dst_path="${install_prefix}/bin/tuwunel"
COPY --from=bins $src_path $dst_path
RUN <<EOF
ret=$(ldd "${dst_path}")
if [ "$ret" = "0" ] && [ "$assert_linkage" = "static" ]; then
exit 1
elif [ "$ret" != "0" ] && [ "$assert_linkage" = "dynamic" ]; then
exit 1
fi
set -eux
ldd -v ${dst_path} || true
du -h ${dst_path}
+10 -6
View File
@@ -1,6 +1,9 @@
# syntax = docker/dockerfile:1.11-labs
FROM input AS rocksdb-fetch
ARG rocksdb_branch="tuwunel-changes"
ARG rust_rocksdb_branch="conduwuit-changes"
ARG librocksdb_submodule="librocksdb-sys/rocksdb"
WORKDIR /
COPY --link --from=input . .
@@ -10,12 +13,13 @@ COPY --link --from=recipe rocksdb.url .
RUN <<EOF
set -eux
submodule="librocksdb-sys/rocksdb"
url="$(cat rocksdb.url)"
rocksdb_url="$(cat rocksdb.url)"
git clone \
--depth 1 \
--recurse-submodules="${submodule}" \
"${url}" \
--single-branch \
--branch "${rust_rocksdb_branch}" \
"--recurse-submodules=${librocksdb_submodule}" \
"${rocksdb_url}" \
/usr/src/rocksdb
EOF
@@ -25,7 +29,7 @@ ARG rocksdb_shared=0
ARG rocksdb_portable=1
ARG rocksdb_opt_level="3"
ARG rocksdb_build_type="Release"
ARG rocksdb_cxx_flags="-ftls-model=initial-exec"
ARG rocksdb_cxx_flags="-ftls-model=initial-exec -mno-outline-atomics"
ARG rocksdb_make_verbose="ON"
ARG rocksdb_make_rule_messages="OFF"
ARG rocksdb_jemalloc=1
@@ -81,7 +85,7 @@ RUN <<EOF
cmake \
--build build \
--target install \
--parallel ${nprocs}
--parallel "${nprocs}"
EOF
+3 -2
View File
@@ -3,7 +3,7 @@
FROM input AS rustup
ARG RUST_HOME
ARG rust_target
ARG rustup_version="1.28.1"
ARG rustup_version="1.28.2"
ARG rustup_profile="minimal"
WORKDIR /
@@ -13,7 +13,8 @@ WORKDIR ${RUST_HOME}
RUN <<EOF
set -eux
url="https://static.rust-lang.org/rustup/archive/${rustup_version}/${rust_target}/rustup-init"
var="${rustup_version}/${rust_target}"
url="https://static.rust-lang.org/rustup/archive/$var/rustup-init"
curl -S -O -s "$url"
chmod o+x rustup-init
EOF
+2 -6
View File
@@ -28,23 +28,18 @@ ARG CARGO_TARGET
ARG CARGO_TERM_VERBOSE=0
ARG RUST_BACKTRACE
ARG JEMALLOC_OVERRIDE
#ARG ZSTD_SYS_USE_PKG_CONFIG=0
ARG ROCKSDB_LIB_DIR
WORKDIR /
COPY --link --from=input . .
WORKDIR /
COPY --link --from=rust ${RUST_HOME} ${RUST_HOME}
COPY --link --from=source /usr/src/tuwunel /usr/src/tuwunel
WORKDIR /usr/src/tuwunel
COPY --link --from=source /usr/src/tuwunel .
ENV PATH="${CARGO_HOME}/bin:$PATH"
ENV CARGO_TERM_VERBOSE="${CARGO_TERM_VERBOSE}"
ENV RUST_BACKTRACE="${RUST_BACKTRACE}"
ENV JEMALLOC_OVERRIDE="${JEMALLOC_OVERRIDE}"
#ENV ZSTD_SYS_USE_PKG_CONFIG="${ZSTD_SYS_USE_PKG_CONFIG}"
ENV ROCKSDB_LIB_DIR="${ROCKSDB_LIB_DIR}"
RUN \
--mount=type=cache,dst=${RUSTUP_HOME}/downloads,sharing=locked \
@@ -91,6 +86,7 @@ RUN \
package="rust-librocksdb-sys"
cmd="cargo tree --manifest-path ${manifest} -f {r} -p ${package}"
url="$(rustup run ${rust_toolchain} ${cmd} | head -n 1)"
url="https://github.com/matrix-construct/rust-rocksdb" #TODO fixme
echo "$url" > rocksdb.url
sha1sum recipe.json rocksdb.url
+2 -2
View File
@@ -17,8 +17,8 @@ COPY --link --from=input . .
ENV DEBIAN_FRONTEND="${DEBIAN_FRONTEND}"
RUN \
--mount=type=cache,dst=${var_cache},sharing=locked \
--mount=type=cache,dst=${var_lib_apt},sharing=locked \
--mount=type=cache,dst=/var/cache,id=${var_cache},sharing=locked \
--mount=type=cache,dst=/var/lib/apt,id=${var_lib_apt},sharing=locked \
<<EOF
echo $(uname -a) $0 $-
set -eux
+5 -5
View File
@@ -14,7 +14,7 @@ variations while only paying the cost of rebuilding the last layer for each one.
This directory is made up of three types of files:
- Shell scripts are the user interface. Use this system through one of the shell scripts. The
bake files can still be docker'ed directly but it's recomended to run the script.
bake files can still be docker'ed directly but it's recommended to run the script.
- The `.hcl` files specify the targets of the tree. This is all standard
[docker bake](https://docs.docker.com/build/bake/reference/). All targets are ordered where each
@@ -30,8 +30,8 @@ many targets to create many variations using the same Dockerfile.
### Getting started
1. You will need to install docker buildx/buildkit and maybe a couple other related things.
I would appreciate if you could contribute exact commands for your platform when you perform
this step.
`apt-get install docker-buildx`
2. You will need to create a builder. There are a few complications that must be explained here
so please be patient.
@@ -62,7 +62,7 @@ so please be patient.
all = true
EOF
BKD_FLAGS="--allow-insecure-entitlement netwok.host"
BKD_FLAGS="--allow-insecure-entitlement network.host"
docker buildx create \
--name owo \
--bootstrap \
@@ -84,7 +84,7 @@ cues from the primary user of this system, the [GitHub CI](https://github.com/ma
5. Defeat the final boss by building and running complement to completion. This will involve
building the targets for `complement-tester` and `complement-testee` using `bake.sh` and then
invoking `complement.sh`. You can take cues again from another user of this in the
[GitHub CI](https://github.com/matrix-construct/tuwunel/blob/main/.github/workflows/test.yml#L79).
[GitHub CI](https://github.com/matrix-construct/tuwunel/blob/7e8c4c8883ca17de91c3323d329541342366ee0c/.github/workflows/test.yml#L140).
## Notes
+193 -140
View File
@@ -38,21 +38,24 @@ cargo_feat_sets = {
default = "brotli_compression,element_hacks,gzip_compression,io_uring,jemalloc,jemalloc_conf,media_thumbnail,release_max_log_level,systemd,url_preview,zstd_compression"
all = "blurhashing,brotli_compression,tuwunel_mods,console,default,direct_tls,element_hacks,gzip_compression,hardened_malloc,io_uring,jemalloc,jemalloc_conf,jemalloc_prof,jemalloc_stats,ldap,media_thumbnail,perf_measurements,release_max_log_level,sentry_telemetry,systemd,tokio_console,url_preview,zstd_compression"
}
variable "cargo_features_always" {
default = "direct_tls"
}
variable "feat_sets" {
default = "[\"none\", \"default\", \"all\"]"
}
variable "cargo_profiles" {
default = "[\"test\", \"release\"]"
}
variable "install_prefix" {
default = "/usr"
}
variable "rust_msrv" {
default = "stable"
}
variable "rust_toolchains" {
default = "[\"nightly\", \"stable\"]"
}
@@ -149,58 +152,68 @@ variable "git_checkout" {
default = "HEAD"
}
#
# Rustflags
#
rustflags = [
"-C link-arg=--verbose",
"-C link-arg=-Wl,--gc-sections",
]
static_rustflags = [
"-C relocation-model=static",
"-C target-feature=+crt-static",
]
static_libs = [
"-C link-arg=-l:libstdc++.a",
"-C link-arg=-l:libc.a",
"-C link-arg=-l:libm.a",
]
dynamic_rustflags = [
"-C relocation-model=pic",
"-C target-feature=-crt-static",
"-C link-arg=-Wl,--as-needed",
]
dynamic_libs = [
"-C link-arg=-lstdc++",
"-C link-arg=-lc",
"-C link-arg=-lm",
]
nightly_rustflags = [
"--cfg tokio_unstable",
"--cfg tuwunel_bench",
"--allow=unstable-features",
"-Zcrate-attr=feature(test)",
"-Zenforce-type-length-limit",
#"-Ztime-passes",
#"-Ztime-llvm-passes",
]
static_rustflags = [
"-Crelocation-model=static",
"-Ctarget-feature=+crt-static",
"-Clink-arg=--verbose",
"-Clink-arg=-Wl,--gc-sections",
"-Clink-arg=-L/usr/lib/gcc/x86_64-linux-gnu/14", # FIXME
"-Clink-arg=-l:libstdc++.a",
"-Clink-arg=-l:libc.a",
"-Clink-arg=-l:libm.a",
]
dynamic_rustflags = [
"-Crelocation-model=pic",
"-Ctarget-feature=-crt-static",
"-Clink-arg=--verbose",
"-Clink-arg=-Wl,--gc-sections",
"-Clink-arg=-Wl,--as-needed",
"-Clink-arg=-lstdc++",
"-Clink-arg=-lc",
"-Clink-arg=-lm",
"-Z crate-attr=feature(test)",
"-Z enforce-type-length-limit",
#"-Z time-passes",
#"-Z time-llvm-passes",
]
static_nightly_rustflags = [
"-Ztls-model=local-exec",
"-Z tls-model=local-exec",
]
rmp_rustflags = [
"-Ctarget-cpu=native",
"-Ztune-cpu=native",
"-Zinline-mir=true",
"-Zmir-opt-level=3",
native_rustflags = [
"-C target-cpu=native",
"-Z tune-cpu=native",
"-Z inline-mir=true",
"-Z mir-opt-level=3",
]
override_rustflags = [
"-Crelocation-model=pic",
"-Ctarget-feature=-crt-static",
"-Clink-arg=-Wl,--no-gc-sections",
"-C relocation-model=pic",
"-C target-feature=-crt-static",
"-C link-arg=-Wl,--no-gc-sections",
]
macro_rustflags = [
"-Crelocation-model=pic",
"-Ctarget-feature=-crt-static",
"-C relocation-model=pic",
"-C target-feature=-crt-static",
]
#
@@ -579,7 +592,7 @@ target "static" {
tags = [
elem_tag("static", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target], "latest"),
]
output = ["type=docker,compression=zstd,mode=min,compression-level=${zstd_image_compress_level}"]
output = ["type=docker,compression=uncompressed,mode=min"]
matrix = cargo_rust_feat_sys
inherits = [
elem("install", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target]),
@@ -631,7 +644,7 @@ target "install" {
]
labels = install_labels
output = ["type=docker,compression=zstd,mode=min,compression-level=${zstd_image_compress_level}"]
cache_to = ["type=local,compression=zstd,mode=min,compression-level=${cache_compress_level}"]
cache_to = ["type=local,compression=zstd,mode=max,compression-level=${cache_compress_level}"]
dockerfile = "${docker_dir}/Dockerfile.install"
target = "install"
matrix = cargo_rust_feat_sys
@@ -655,43 +668,27 @@ target "install" {
group "pkg" {
targets = [
"pkg-deb",
"pkg-rpm",
"pkg-deb-install",
"pkg-rpm-install",
"deb",
"rpm",
"deb-install",
"rpm-install",
]
}
target "pkg-rpm-install" {
name = elem("pkg-rpm-install", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target])
target "rpm-install" {
name = elem("rpm-install", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target])
tags = [
elem_tag("pkg-rpm-install", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target], "latest"),
elem_tag("rpm-install", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target], "latest"),
]
target = "pkg-rpm-install"
target = "rpm-install"
output = ["type=cacheonly,compression=zstd,mode=min,compression-level=${cache_compress_level}"]
matrix = cargo_rust_feat_sys
inherits = [
elem("pkg-rpm", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target]),
]
contexts = {
input = "docker-image://redhat/ubi9"
pkg-rpm = elem("target:pkg-rpm", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target])
}
}
target "pkg-rpm" {
name = elem("pkg-rpm", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target])
tags = [
elem_tag("pkg-rpm", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target], "latest"),
]
target = "pkg-rpm"
output = ["type=docker,compression=zstd,mode=min"]
matrix = cargo_rust_feat_sys
inherits = [
elem("rpm", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target]),
]
contexts = {
input = elem("target:rpm", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target]),
input = "docker-image://redhat/ubi9"
rpm = elem("target:rpm", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target])
}
}
@@ -701,6 +698,22 @@ target "rpm" {
elem_tag("rpm", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target], "latest"),
]
target = "rpm"
output = ["type=docker,compression=zstd,mode=min"]
matrix = cargo_rust_feat_sys
inherits = [
elem("build-rpm", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target]),
]
contexts = {
input = elem("target:build-rpm", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target]),
}
}
target "build-rpm" {
name = elem("build-rpm", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target])
tags = [
elem_tag("build-rpm", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target], "latest"),
]
target = "build-rpm"
dockerfile = "${docker_dir}/Dockerfile.cargo.rpm"
matrix = cargo_rust_feat_sys
inherits = [
@@ -714,36 +727,20 @@ target "rpm" {
}
}
target "pkg-deb-install" {
name = elem("pkg-deb-install", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target])
target "deb-install" {
name = elem("deb-install", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target])
tags = [
elem_tag("pkg-deb-install", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target], "latest"),
elem_tag("deb-install", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target], "latest"),
]
target = "pkg-deb-install"
target = "deb-install"
output = ["type=cacheonly,compression=zstd,mode=min,compression-level=${cache_compress_level}"]
matrix = cargo_rust_feat_sys
inherits = [
elem("pkg-deb", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target]),
]
contexts = {
input = elem("target:runtime", [feat_set, sys_name, sys_version, sys_target])
pkg-deb = elem("target:pkg-deb", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target]),
}
}
target "pkg-deb" {
name = elem("pkg-deb", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target])
tags = [
elem_tag("pkg-deb", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target], "latest"),
]
target = "pkg-deb"
output = ["type=docker,compression=zstd,mode=min"]
matrix = cargo_rust_feat_sys
inherits = [
elem("deb", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target]),
]
contexts = {
input = elem("target:deb", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target]),
input = elem("target:runtime", [feat_set, sys_name, sys_version, sys_target])
deb = elem("target:deb", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target]),
}
}
@@ -753,6 +750,22 @@ target "deb" {
elem_tag("deb", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target], "latest"),
]
target = "deb"
output = ["type=docker,compression=zstd,mode=min"]
matrix = cargo_rust_feat_sys
inherits = [
elem("build-deb", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target]),
]
contexts = {
input = elem("target:build-deb", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target]),
}
}
target "build-deb" {
name = elem("build-deb", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target])
tags = [
elem_tag("build-deb", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target], "latest"),
]
target = "build-deb"
dockerfile = "${docker_dir}/Dockerfile.cargo.deb"
matrix = cargo_rust_feat_sys
inherits = [
@@ -828,7 +841,7 @@ target "docs" {
elem("build", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target]),
]
contexts = {
input = (use_chef == "true"?
deps = (use_chef == "true"?
elem("target:deps-build", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target]):
elem("target:ingredients", [rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target])
)
@@ -851,7 +864,7 @@ target "build-bins" {
elem("build", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target]),
]
contexts = {
input = (use_chef == "true"?
deps = (use_chef == "true"?
elem("target:deps-build-bins", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target]):
elem("target:build", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target])
)
@@ -873,7 +886,7 @@ target "build-tests" {
elem("build", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target]),
]
contexts = {
input = (use_chef == "true"?
deps = (use_chef == "true"?
elem("target:deps-build-tests", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target]):
elem("target:build", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target])
)
@@ -895,7 +908,7 @@ target "build" {
elem("cargo", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target]),
]
contexts = {
input = (use_chef == "true"?
deps = (use_chef == "true"?
elem("target:deps-build", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target]):
elem("target:ingredients", [rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target])
)
@@ -917,7 +930,7 @@ target "clippy" {
elem("cargo", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target]),
]
contexts = {
input = (use_chef == "true"?
deps = (use_chef == "true"?
elem("target:deps-clippy", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target]):
elem("target:ingredients", [rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target])
)
@@ -939,7 +952,7 @@ target "check" {
elem("cargo", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target]),
]
contexts = {
input = (use_chef == "true"?
deps = (use_chef == "true"?
elem("target:deps-check", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target]):
elem("target:ingredients", [rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target])
)
@@ -1023,14 +1036,20 @@ target "fmt" {
target "cargo" {
name = elem("cargo", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target])
target = "cargo"
output = ["type=cacheonly,compression=zstd,mode=min,compression-level=${cache_compress_level}"]
cache_to = ["type=local,compression=zstd,mode=min,compression-level=${cache_compress_level}"]
dockerfile = "${docker_dir}/Dockerfile.cargo"
cache_to = ["type=local,compression=zstd,mode=max,compression-level=${cache_compress_level}"]
matrix = cargo_rust_feat_sys
inherits = [
elem("deps-base", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target])
]
contexts = {
deps = elem("target:deps-base", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target])
}
args = {
recipe_args = ""
cargo_args = ""
color_args = "--color=always"
}
}
#
@@ -1057,7 +1076,7 @@ target "deps-build-bins" {
elem("deps-base", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target])
]
args = {
cook_args = "--bins"
cargo_cmd = "chef cook --bins"
}
}
@@ -1071,7 +1090,7 @@ target "deps-build-tests" {
elem("deps-base", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target])
]
args = {
cook_args = (cargo_profile == "bench"? "--benches": "--tests")
cargo_cmd = (cargo_profile == "bench"? "chef cook --benches": "chef cook --tests")
}
}
@@ -1085,7 +1104,7 @@ target "deps-build" {
elem("deps-base", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target])
]
args = {
cook_args = "--all-targets"
cargo_cmd = "chef cook --all-targets"
}
}
@@ -1099,7 +1118,7 @@ target "deps-clippy" {
elem("deps-base", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target])
]
args = {
cook_args = "--all-targets --clippy"
cargo_cmd = "chef cook --all-targets --clippy"
}
}
@@ -1113,7 +1132,7 @@ target "deps-check" {
elem("deps-base", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target])
]
args = {
cook_args = "--all-targets --check"
cargo_cmd = "chef cook --all-targets --check"
}
}
@@ -1126,22 +1145,26 @@ target "deps-base" {
tags = [
elem_tag("deps-base", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target], "latest")
]
target = "deps"
target = "cargo"
output = ["type=cacheonly,compression=zstd,mode=min,compression-level=${cache_compress_level}"]
cache_to = ["type=local,compression=zstd,mode=min,compression-level=${cache_compress_level}"]
dockerfile = "${docker_dir}/Dockerfile.cargo.deps"
cache_to = ["type=local,compression=zstd,mode=max,compression-level=${cache_compress_level}"]
dockerfile = "${docker_dir}/Dockerfile.cargo"
matrix = cargo_rust_feat_sys
inherits = [
elem("kitchen", [feat_set, sys_name, sys_version, sys_target]),
elem("rust", [rust_toolchain, rust_target, sys_name, sys_version, sys_target]),
elem("recipe", [rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target])
]
contexts = {
input = elem("target:ingredients", [rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target])
recipe = elem("target:recipe", [rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target])
input = elem("target:kitchen", [feat_set, sys_name, sys_version, sys_target])
deps = elem("target:preparing", [rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target])
rocksdb = elem("target:rocksdb", [cargo_profile, rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target])
}
args = {
cargo_profile = cargo_profile
cook_args = "--all-targets --no-build"
cargo_cmd = "chef cook --all-targets --no-build"
recipe_args = "--recipe-path=recipe.json"
color_args = ""
# Base path
CARGO_TARGET_DIR = "${cargo_tgt_dir_base}"
@@ -1167,56 +1190,80 @@ target "deps-base" {
CARGO_BUILD_RUSTFLAGS = (
cargo_profile == "release-native"?
join(" ", [
join(" ", rustflags),
join(" ", nightly_rustflags),
contains(split(",", cargo_feat_sets[feat_set]), "zstd_compression")?
"-Clink-arg=-l:libzstd.a": "",
contains(split(",", cargo_feat_sets[feat_set]), "io_uring")?
"-Clink-arg=-l:liburing.a": "",
join(" ", static_rustflags),
join(" ", static_nightly_rustflags),
join(" ", rmp_rustflags),
join(" ", native_rustflags),
"-C link-arg=-L/usr/lib/gcc/${sys_target}/14", #FIXME
contains(split(",", cargo_feat_sets[feat_set]), "zstd_compression")?
"-C link-arg=-l:libzstd.a": "",
contains(split(",", cargo_feat_sets[feat_set]), "io_uring")?
"-C link-arg=-l:liburing.a": "",
join(" ", static_libs),
sys_target == "aarch64-linux-gnu"?
"-C link-arg=-l:libgcc.a": "",
]):
cargo_profile == "release" && rust_toolchain == "nightly"?
join(" ", [
join(" ", rustflags),
join(" ", nightly_rustflags),
contains(split(",", cargo_feat_sets[feat_set]), "zstd_compression")?
"-Clink-arg=-l:libzstd.a": "",
contains(split(",", cargo_feat_sets[feat_set]), "io_uring")?
"-Clink-arg=-l:liburing.a": "",
join(" ", static_rustflags),
join(" ", static_nightly_rustflags),
"-C link-arg=-L/usr/lib/gcc/${sys_target}/14", #FIXME
contains(split(",", cargo_feat_sets[feat_set]), "zstd_compression")?
"-C link-arg=-l:libzstd.a": "",
contains(split(",", cargo_feat_sets[feat_set]), "io_uring")?
"-C link-arg=-l:liburing.a": "",
join(" ", static_libs),
sys_target == "aarch64-linux-gnu"?
"-C link-arg=-l:libgcc.a": "",
]):
cargo_profile == "release" || cargo_profile == "release-debuginfo"?
join(" ", [
contains(split(",", cargo_feat_sets[feat_set]), "zstd_compression")?
"-Clink-arg=-l:libzstd.a": "",
contains(split(",", cargo_feat_sets[feat_set]), "io_uring")?
"-Clink-arg=-l:liburing.a": "",
join(" ", rustflags),
join(" ", static_rustflags),
"-C link-arg=-L/usr/lib/gcc/${sys_target}/14", #FIXME
contains(split(",", cargo_feat_sets[feat_set]), "zstd_compression")?
"-C link-arg=-l:libzstd.a": "",
contains(split(",", cargo_feat_sets[feat_set]), "io_uring")?
"-C link-arg=-l:liburing.a": "",
join(" ", static_libs),
sys_target == "aarch64-linux-gnu"?
"-C link-arg=-l:libgcc.a": "",
]):
rust_toolchain == "stable"?
join(" ", [
contains(split(",", cargo_feat_sets[feat_set]), "zstd_compression")?
"-Clink-arg=-l:libzstd.a": "",
contains(split(",", cargo_feat_sets[feat_set]), "io_uring")?
"-Clink-arg=-l:liburing.a": "",
join(" ", rustflags),
join(" ", static_rustflags),
"-C link-arg=-L/usr/lib/gcc/${sys_target}/14", #FIXME
contains(split(",", cargo_feat_sets[feat_set]), "zstd_compression")?
"-C link-arg=-l:libzstd.a": "",
contains(split(",", cargo_feat_sets[feat_set]), "io_uring")?
"-C link-arg=-l:liburing.a": "",
join(" ", static_libs),
sys_target == "aarch64-linux-gnu"?
"-C link-arg=-l:libgcc.a": "",
]):
rust_toolchain == "nightly"?
join(" ", [
join(" ", rustflags),
join(" ", nightly_rustflags),
contains(split(",", cargo_feat_sets[feat_set]), "zstd_compression")?
"-Clink-arg=-lzstd": "",
contains(split(",", cargo_feat_sets[feat_set]), "io_uring")?
"-Clink-arg=-luring": "",
join(" ", dynamic_rustflags),
contains(split(",", cargo_feat_sets[feat_set]), "zstd_compression")?
"-C link-arg=-lzstd": "",
contains(split(",", cargo_feat_sets[feat_set]), "io_uring")?
"-C link-arg=-luring": "",
join(" ", dynamic_libs),
]):
join(" ", [])
join(" ", [
join(" ", rustflags),
])
)
}
}
@@ -1309,6 +1356,7 @@ target "recipe" {
]
contexts = {
input = elem("target:preparing", [rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target])
preparing = elem("target:preparing", [rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target])
}
}
@@ -1324,6 +1372,7 @@ target "preparing" {
]
contexts = {
input = elem("target:ingredients", [rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target])
ingredients = elem("target:ingredients", [rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target])
}
}
@@ -1332,17 +1381,18 @@ target "ingredients" {
tags = [
elem_tag("ingredients", [rust_toolchain, rust_target, feat_set, sys_name, sys_version, sys_target], "latest"),
]
cache_to = ["type=local,compression=zstd,mode=max"]
target = "ingredients"
cache_to = ["type=local,compression=zstd,mode=min"]
dockerfile = "${docker_dir}/Dockerfile.source"
matrix = rust_feat_sys
inherits = [
elem("source", [sys_name, sys_version, sys_target]),
elem("kitchen", [feat_set, sys_name, sys_version, sys_target]),
elem("rust", [rust_toolchain, rust_target, sys_name, sys_version, sys_target]),
elem("source", [sys_name, sys_version, sys_target]),
]
contexts = {
rust = elem("target:rust", [rust_toolchain, rust_target, sys_name, sys_version, sys_target])
input = elem("target:kitchen", [feat_set, sys_name, sys_version, sys_target])
rust = elem("target:rust", [rust_toolchain, rust_target, sys_name, sys_version, sys_target])
source = elem("target:source", [sys_name, sys_version, sys_target])
}
args = {
@@ -1400,7 +1450,7 @@ cargo_installs = [
"cargo-chef",
"cargo-audit",
"cargo-deb",
"cargo-arch",
#"cargo-arch",
"cargo-generate-rpm",
"lychee",
"mdbook",
@@ -1429,12 +1479,15 @@ target "rust" {
input = elem("target:rustup", [rust_target, sys_name, sys_version, sys_target])
}
args = {
rust_toolchain = rust_toolchain
rust_toolchain = (rust_toolchain == "stable"?
rust_msrv: rust_toolchain
)
rustup_components = join(" ", rustup_components)
cargo_installs = join(" ", cargo_installs)
CARGO_TERM_VERBOSE = CARGO_TERM_VERBOSE
RUSTUP_HOME = "/opt/rust/rustup/${sys_name}"
RUSTUP_HOME = "/opt/rust/rustup/${sys_name}/${sys_target}"
CARGO_HOME = "/opt/rust/cargo/${sys_name}/${sys_target}"
}
}
@@ -1639,8 +1692,8 @@ target "base" {
}
args = {
DEBIAN_FRONTEND="noninteractive"
var_lib_apt = "/var/lib/apt"
var_cache = "/var/cache"
var_lib_apt = "/var/lib/apt/${sys_name}/${sys_version}/${sys_target}"
var_cache = "/var/cache/${sys_name}/${sys_version}/${sys_target}"
packages = join(" ", base_pkgs)
}
}
@@ -1657,7 +1710,7 @@ target "system" {
]
target = "system"
output = ["type=cacheonly,compression=zstd,mode=min,compression-level=${cache_compress_level}"]
cache_to = ["type=local,compression=zstd,mode=min,compression-level=${cache_compress_level}"]
cache_to = ["type=local,compression=zstd,mode=max,compression-level=${cache_compress_level}"]
cache_from = ["type=local"]
dockerfile = "${docker_dir}/Dockerfile.system"
matrix = sys
+8 -10
View File
@@ -34,15 +34,15 @@ if test ! -z "$rust_toolchain"; then
fi
if test ! -z "$sys_name"; then
env_sys_name="[\"${sys_name}\"]"
env_sys_names="[\"${sys_name}\"]"
fi
if test ! -z "$sys_target"; then
env_sys_target="[\"${sys_target}\"]"
env_sys_targets="[\"${sys_target}\"]"
fi
if test ! -z "$sys_version"; then
env_sys_version="[\"${sys_version}\"]"
env_sys_versions="[\"${sys_version}\"]"
fi
set -a
@@ -56,9 +56,9 @@ sys_targets="${env_sys_targets:-$default_sys_targets}"
sys_versions="${env_sys_versions:-$default_sys_versions}"
docker_dir="$PWD/$BASEDIR"
runner_name=$(echo $RUNNER_NAME | cut -d"." -f1)
runner_num=$(echo $RUNNER_NAME | cut -d"." -f2)
builder_name="${GITHUB_ACTOR:-owo}"
toolchain_toml="$docker_dir/../rust-toolchain.toml"
rust_msrv=$(grep "channel = " "$toolchain_toml" | cut -d'=' -f2 | sed 's/\s"\|"$//g')
rocksdb_opt_level=3
rocksdb_portable=1
git_checkout="HEAD"
@@ -90,11 +90,9 @@ if test "$CI" = "true"; then
args="$args --allow=network.host"
fi
if test ! -z "$runner_num"; then
#cpu_num=$(expr $runner_num % $(nproc))
#args="$args --cpuset-cpus=${cpu_num}"
#args="$args --set *.args.nprocs=1"
# https://github.com/moby/buildkit/issues/1276
if test "$(uname)" = "Darwin"; then
nprocs=$(sysctl -n hw.logicalcpu)
args="$args --set *.args.nprocs=${nprocs}"
:
else
nprocs=$(nproc)
+3 -3
View File
@@ -45,7 +45,7 @@ fi
docker rm -f "$name" 2>/dev/null
trap 'set +x; date; echo -e "\033[1;41;37mFAIL\033[0m"' ERR
trap 'set +x; date; echo -e "\033[1;41;37mERROR\033[0m"' ERR
if test "$CI" = "true"; then
arg="-d $arg"
@@ -58,8 +58,8 @@ fi
echo -n "$cid" > "$name"
trap 'docker container stop $cid; set +x; date; echo -e "\033[1;41;37mFAIL\033[0m"' INT
trap 'docker container stop $cid; set +x; date; echo -e "\033[1;41;37mERROR\033[0m"' INT
docker logs -f "$cid"
docker wait "$cid" 2>/dev/null
echo -e "\033[1;42;30mPASS\033[0m"
echo -e "\033[1;42;30mACCEPT\033[0m"
+1
View File
@@ -10,6 +10,7 @@
- [Kubernetes](deploying/kubernetes.md)
- [Arch Linux](deploying/arch-linux.md)
- [Debian](deploying/debian.md)
- [Red Hat](deploying/redhat.md)
- [FreeBSD](deploying/freebsd.md)
- [TURN](turn.md)
- [Appservices](appservices.md)
+1
View File
@@ -6,5 +6,6 @@
- [Kubernetes](kubernetes.md)
- [Arch Linux](arch-linux.md)
- [Debian](debian.md)
- [Red Hat](redhat.md)
- [FreeBSD](freebsd.md)
- [Podman systemd](podman-systemd.md)
+1
View File
@@ -0,0 +1 @@
{{#include ../../rpm/README.md}}
Generated
+54 -71
View File
@@ -10,11 +10,11 @@
"nixpkgs-stable": "nixpkgs-stable"
},
"locked": {
"lastModified": 1738524606,
"narHash": "sha256-hPYEJ4juK3ph7kbjbvv7PlU1D9pAkkhl+pwx8fZY53U=",
"lastModified": 1748532342,
"narHash": "sha256-CvaKOUq8G10sghKpZhEB2UYjJoWhEkrDFggDgi7piUI=",
"owner": "zhaofengli",
"repo": "attic",
"rev": "ff8a897d1f4408ebbf4d45fa9049c06b3e1e3f4e",
"rev": "ce9373715fe3fac7a174a65a7e6d6baeba8cb4f9",
"type": "github"
},
"original": {
@@ -32,11 +32,11 @@
"nixpkgs": "nixpkgs_4"
},
"locked": {
"lastModified": 1737621947,
"narHash": "sha256-8HFvG7fvIFbgtaYAY2628Tb89fA55nPm2jSiNs0/Cws=",
"lastModified": 1748883665,
"narHash": "sha256-R0W7uAg+BLoHjMRMQ8+oiSbTq8nkGz5RDpQ+ZfxxP3A=",
"owner": "cachix",
"repo": "cachix",
"rev": "f65a3cd5e339c223471e64c051434616e18cc4f5",
"rev": "f707778d902af4d62d8dd92c269f8e70de09acbe",
"type": "github"
},
"original": {
@@ -63,11 +63,11 @@
"nixpkgs": "nixpkgs_2"
},
"locked": {
"lastModified": 1728672398,
"narHash": "sha256-KxuGSoVUFnQLB2ZcYODW7AVPAh9JqRlD5BrfsC/Q4qs=",
"lastModified": 1744206633,
"narHash": "sha256-pb5aYkE8FOoa4n123slgHiOf1UbNSnKe5pEZC+xXD5g=",
"owner": "cachix",
"repo": "cachix",
"rev": "aac51f698309fd0f381149214b7eee213c66ef0a",
"rev": "8a60090640b96f9df95d1ab99e5763a586be1404",
"type": "github"
},
"original": {
@@ -82,13 +82,13 @@
"locked": {
"lastModified": 1741891349,
"narHash": "sha256-YvrzOWcX7DH1drp5SGa+E/fc7wN3hqFtPbqPjZpOu1Q=",
"owner": "girlbossceo",
"owner": "matrix-construct",
"repo": "complement",
"rev": "e587b3df569cba411aeac7c20b6366d03c143745",
"type": "github"
},
"original": {
"owner": "girlbossceo",
"owner": "matrix-construct",
"ref": "main",
"repo": "complement",
"type": "github"
@@ -117,11 +117,11 @@
},
"crane_2": {
"locked": {
"lastModified": 1739936662,
"narHash": "sha256-x4syUjNUuRblR07nDPeLDP7DpphaBVbUaSoeZkFbGSk=",
"lastModified": 1748970125,
"narHash": "sha256-UDyigbDGv8fvs9aS95yzFfOKkEjx1LO3PL3DsKopohA=",
"owner": "ipetkov",
"repo": "crane",
"rev": "19de14aaeb869287647d9461cbd389187d8ecdb7",
"rev": "323b5746d89e04b22554b061522dfce9e4c49b18",
"type": "github"
},
"original": {
@@ -149,11 +149,11 @@
]
},
"locked": {
"lastModified": 1733323168,
"narHash": "sha256-d5DwB4MZvlaQpN6OQ4SLYxb5jA4UH5EtV5t5WOtjLPU=",
"lastModified": 1748273445,
"narHash": "sha256-5V0dzpNgQM0CHDsMzh+ludYeu1S+Y+IMjbaskSSdFh0=",
"owner": "cachix",
"repo": "devenv",
"rev": "efa9010b8b1cfd5dd3c7ed1e172a470c3b84a064",
"rev": "668a50d8b7bdb19a0131f53c9f6c25c9071e1ffb",
"type": "github"
},
"original": {
@@ -170,11 +170,11 @@
"rust-analyzer-src": "rust-analyzer-src"
},
"locked": {
"lastModified": 1740724364,
"narHash": "sha256-D1jLIueJx1dPrP09ZZwTrPf4cubV+TsFMYbpYYTVj6A=",
"lastModified": 1749883145,
"narHash": "sha256-RlcGw3vAnbI3cfZn8aFaovNUd7312VZh+/FDWkqdA7E=",
"owner": "nix-community",
"repo": "fenix",
"rev": "edf7d9e431cda8782e729253835f178a356d3aab",
"rev": "a804172f150bcf81262655324e583bb0cd0f28dd",
"type": "github"
},
"original": {
@@ -203,11 +203,11 @@
"flake-compat_2": {
"flake": false,
"locked": {
"lastModified": 1733328505,
"narHash": "sha256-NeCCThCEP3eCl2l/+27kNNK7QrwZB1IJCrXfrbv5oqU=",
"lastModified": 1747046372,
"narHash": "sha256-CIVLLkVgvHYbgI2UpXvIIBJ12HWgX+fjA8Xf8PUmqCY=",
"owner": "edolstra",
"repo": "flake-compat",
"rev": "ff81ac966bb2cae68946d5ed5fc4994f96d0ffec",
"rev": "9100a0f413b0c601e0533d1d94ffd501ce2e7885",
"type": "github"
},
"original": {
@@ -219,11 +219,11 @@
"flake-compat_3": {
"flake": false,
"locked": {
"lastModified": 1733328505,
"narHash": "sha256-NeCCThCEP3eCl2l/+27kNNK7QrwZB1IJCrXfrbv5oqU=",
"lastModified": 1747046372,
"narHash": "sha256-CIVLLkVgvHYbgI2UpXvIIBJ12HWgX+fjA8Xf8PUmqCY=",
"owner": "edolstra",
"repo": "flake-compat",
"rev": "ff81ac966bb2cae68946d5ed5fc4994f96d0ffec",
"rev": "9100a0f413b0c601e0533d1d94ffd501ce2e7885",
"type": "github"
},
"original": {
@@ -306,15 +306,14 @@
"nixpkgs": [
"cachix",
"nixpkgs"
],
"nixpkgs-stable": "nixpkgs-stable_2"
]
},
"locked": {
"lastModified": 1733318908,
"narHash": "sha256-SVQVsbafSM1dJ4fpgyBqLZ+Lft+jcQuMtEL3lQWx2Sk=",
"lastModified": 1747372754,
"narHash": "sha256-2Y53NGIX2vxfie1rOW0Qb86vjRZ7ngizoo+bnXU9D9k=",
"owner": "cachix",
"repo": "git-hooks.nix",
"rev": "6f4e2a2112050951a314d2733a994fbab94864c6",
"rev": "80479b6ec16fefd9c1db3ea13aeb038c60530f46",
"type": "github"
},
"original": {
@@ -364,11 +363,11 @@
"liburing": {
"flake": false,
"locked": {
"lastModified": 1740613216,
"narHash": "sha256-NpPOBqNND3Qe9IwqYs0mJLGTmIx7e6FgUEBAnJ+1ZLA=",
"lastModified": 1749816983,
"narHash": "sha256-p5hXfDe53Y4MVwL2+wKZYpy4OPGvqFFnOEvkMsFAO6c=",
"owner": "axboe",
"repo": "liburing",
"rev": "e1003e496e66f9b0ae06674869795edf772d5500",
"rev": "ad83d3ab64894c16eaf21ef869656a5bddb93ca4",
"type": "github"
},
"original": {
@@ -401,11 +400,11 @@
]
},
"locked": {
"lastModified": 1727438425,
"narHash": "sha256-X8ES7I1cfNhR9oKp06F6ir4Np70WGZU5sfCOuNBEwMg=",
"lastModified": 1745930071,
"narHash": "sha256-bYyjarS3qSNqxfgc89IoVz8cAFDkF9yPE63EJr+h50s=",
"owner": "domenkozar",
"repo": "nix",
"rev": "f6c5ae4c1b2e411e6b1e6a8181cc84363d6a7546",
"rev": "b455edf3505f1bf0172b39a735caef94687d0d9c",
"type": "github"
},
"original": {
@@ -484,29 +483,13 @@
"type": "github"
}
},
"nixpkgs-stable_2": {
"locked": {
"lastModified": 1730741070,
"narHash": "sha256-edm8WG19kWozJ/GqyYx2VjW99EdhjKwbY3ZwdlPAAlo=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "d063c1dd113c91ab27959ba540c0d9753409edf3",
"type": "github"
},
"original": {
"owner": "NixOS",
"ref": "nixos-24.05",
"repo": "nixpkgs",
"type": "github"
}
},
"nixpkgs_2": {
"locked": {
"lastModified": 1730531603,
"narHash": "sha256-Dqg6si5CqIzm87sp57j5nTaeBbWhHFaVyG7V6L8k3lY=",
"lastModified": 1733212471,
"narHash": "sha256-M1+uCoV5igihRfcUKrr1riygbe73/dzNnzPsmaLCmpo=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "7ffd9ae656aec493492b44d0ddfb28e79a1ea25d",
"rev": "55d15ad12a74eb7d4646254e13638ad0c4128776",
"type": "github"
},
"original": {
@@ -534,11 +517,11 @@
},
"nixpkgs_4": {
"locked": {
"lastModified": 1733212471,
"narHash": "sha256-M1+uCoV5igihRfcUKrr1riygbe73/dzNnzPsmaLCmpo=",
"lastModified": 1748190013,
"narHash": "sha256-R5HJFflOfsP5FBtk+zE8FpL8uqE7n62jqOsADvVshhE=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "55d15ad12a74eb7d4646254e13638ad0c4128776",
"rev": "62b852f6c6742134ade1abdd2a21685fd617a291",
"type": "github"
},
"original": {
@@ -550,11 +533,11 @@
},
"nixpkgs_5": {
"locked": {
"lastModified": 1740547748,
"narHash": "sha256-Ly2fBL1LscV+KyCqPRufUBuiw+zmWrlJzpWOWbahplg=",
"lastModified": 1749871736,
"narHash": "sha256-K9yBph93OLTNw02Q6e9CYFGrUhvEXnh45vrZqIRWfvQ=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "3a05eebede89661660945da1f151959900903b6a",
"rev": "6afe187897bef7933475e6af374c893f4c84a293",
"type": "github"
},
"original": {
@@ -567,16 +550,16 @@
"rocksdb": {
"flake": false,
"locked": {
"lastModified": 1741308171,
"narHash": "sha256-YdBvdQ75UJg5ffwNjxizpviCVwVDJnBkM8ZtGIduMgY=",
"owner": "girlbossceo",
"lastModified": 1749358049,
"narHash": "sha256-ZSjvAZBfZkJrBIpw8ANZMbJVb8AeuogvuAipGVE4Qe4=",
"owner": "matrix-construct",
"repo": "rocksdb",
"rev": "3ce04794bcfbbb0d2e6f81ae35fc4acf688b6986",
"rev": "cf7f65d0b377af019661c240f9165b3ef60640c3",
"type": "github"
},
"original": {
"owner": "girlbossceo",
"ref": "v9.11.1",
"owner": "matrix-construct",
"ref": "tuwunel-changes",
"repo": "rocksdb",
"type": "github"
}
@@ -599,11 +582,11 @@
"rust-analyzer-src": {
"flake": false,
"locked": {
"lastModified": 1740691488,
"narHash": "sha256-Fs6vBrByuiOf2WO77qeMDMTXcTGzrIMqLBv+lNeywwM=",
"lastModified": 1749829309,
"narHash": "sha256-t6x6/PKg8Shnkd3htrxf3WMgycfRLRWvN9JHAmGWf+s=",
"owner": "rust-lang",
"repo": "rust-analyzer",
"rev": "fe3eda77d3a7ce212388bda7b6cec8bffcc077e5",
"rev": "a497f4114ccf24978accb56190e60d1e1659e0c7",
"type": "github"
},
"original": {
+42 -33
View File
@@ -2,14 +2,14 @@
inputs = {
attic.url = "github:zhaofengli/attic?ref=main";
cachix.url = "github:cachix/cachix?ref=master";
complement = { url = "github:girlbossceo/complement?ref=main"; flake = false; };
complement = { url = "github:matrix-construct/complement?ref=main"; flake = false; };
crane = { url = "github:ipetkov/crane?ref=master"; };
fenix = { url = "github:nix-community/fenix?ref=main"; inputs.nixpkgs.follows = "nixpkgs"; };
flake-compat = { url = "github:edolstra/flake-compat?ref=master"; flake = false; };
flake-utils.url = "github:numtide/flake-utils?ref=main";
nix-filter.url = "github:numtide/nix-filter?ref=main";
nixpkgs.url = "github:NixOS/nixpkgs?ref=nixpkgs-unstable";
rocksdb = { url = "github:girlbossceo/rocksdb?ref=v9.11.1"; flake = false; };
rocksdb = { url = "github:matrix-construct/rocksdb?ref=tuwunel-changes"; flake = false; };
liburing = { url = "github:axboe/liburing?ref=master"; flake = false; };
};
@@ -53,7 +53,7 @@
};
rocksdb = (pkgs.rocksdb.override {
liburing = self.liburing;
}).overrideAttrs (old: {
}).overrideAttrs (final: old: {
src = inputs.rocksdb;
version = pkgs.lib.removePrefix
"v"
@@ -62,6 +62,15 @@
# we have this already at https://github.com/girlbossceo/rocksdb/commit/a935c0273e1ba44eacf88ce3685a9b9831486155
# unsetting this so i don't have to revert it and make this nix exclusive
patches = [];
postPatch =
pkgs.lib.optionalString (pkgs.lib.versionOlder final.version "8") ''
# Fix gcc-13 build failures due to missing <cstdint> and
# <system_error> includes, fixed upstream since 8.x
sed -e '1i #include <cstdint>' -i db/compaction/compaction_iteration_stats.h
sed -e '1i #include <cstdint>' -i table/block_based/data_block_hash_index.h
sed -e '1i #include <cstdint>' -i util/string_util.h
sed -e '1i #include <cstdint>' -i include/rocksdb/utilities/checkpoint.h
'';
cmakeFlags = pkgs.lib.subtractLists
[
# no real reason to have snappy or zlib, no one uses this
@@ -197,8 +206,8 @@
"jemalloc_stats"
# this is non-functional on nix for some reason
"hardened_malloc"
# conduwuit_mods is a development-only hot reload feature
"conduwuit_mods"
# tuwunel_mods is a development-only hot reload feature
"tuwunel_mods"
];
};
default-debug = scopeHost.main.override {
@@ -210,8 +219,8 @@
"experimental"
# this is non-functional on nix for some reason
"hardened_malloc"
# conduwuit_mods is a development-only hot reload feature
"conduwuit_mods"
# tuwunel_mods is a development-only hot reload feature
"tuwunel_mods"
];
};
# just a test profile used for things like CI and complement
@@ -223,8 +232,8 @@
"experimental"
# this is non-functional on nix for some reason
"hardened_malloc"
# conduwuit_mods is a development-only hot reload feature
"conduwuit_mods"
# tuwunel_mods is a development-only hot reload feature
"tuwunel_mods"
];
};
all-features = scopeHost.main.override {
@@ -238,8 +247,8 @@
"jemalloc_stats"
# this is non-functional on nix for some reason
"hardened_malloc"
# conduwuit_mods is a development-only hot reload feature
"conduwuit_mods"
# tuwunel_mods is a development-only hot reload feature
"tuwunel_mods"
];
};
all-features-debug = scopeHost.main.override {
@@ -252,8 +261,8 @@
"experimental"
# this is non-functional on nix for some reason
"hardened_malloc"
# conduwuit_mods is a development-only hot reload feature
"conduwuit_mods"
# tuwunel_mods is a development-only hot reload feature
"tuwunel_mods"
];
};
hmalloc = scopeHost.main.override { features = ["hardened_malloc"]; };
@@ -271,8 +280,8 @@
"jemalloc_stats"
# this is non-functional on nix for some reason
"hardened_malloc"
# conduwuit_mods is a development-only hot reload feature
"conduwuit_mods"
# tuwunel_mods is a development-only hot reload feature
"tuwunel_mods"
];
};
};
@@ -287,8 +296,8 @@
"experimental"
# this is non-functional on nix for some reason
"hardened_malloc"
# conduwuit_mods is a development-only hot reload feature
"conduwuit_mods"
# tuwunel_mods is a development-only hot reload feature
"tuwunel_mods"
];
};
};
@@ -352,8 +361,8 @@
"experimental"
# this is non-functional on nix for some reason
"hardened_malloc"
# conduwuit_mods is a development-only hot reload feature
"conduwuit_mods"
# tuwunel_mods is a development-only hot reload feature
"tuwunel_mods"
];
};
}
@@ -372,8 +381,8 @@
"jemalloc_stats"
# this is non-functional on nix for some reason
"hardened_malloc"
# conduwuit_mods is a development-only hot reload feature
"conduwuit_mods"
# tuwunel_mods is a development-only hot reload feature
"tuwunel_mods"
];
};
}
@@ -393,8 +402,8 @@
"jemalloc_stats"
# this is non-functional on nix for some reason
"hardened_malloc"
# conduwuit_mods is a development-only hot reload feature
"conduwuit_mods"
# tuwunel_mods is a development-only hot reload feature
"tuwunel_mods"
];
x86_64_haswell_target_optimised = (if (crossSystem == "x86_64-linux-gnu" || crossSystem == "x86_64-linux-musl") then true else false);
};
@@ -413,8 +422,8 @@
"experimental"
# this is non-functional on nix for some reason
"hardened_malloc"
# conduwuit_mods is a development-only hot reload feature
"conduwuit_mods"
# tuwunel_mods is a development-only hot reload feature
"tuwunel_mods"
];
};
}
@@ -471,8 +480,8 @@
"jemalloc_stats"
# this is non-functional on nix for some reason
"hardened_malloc"
# conduwuit_mods is a development-only hot reload feature
"conduwuit_mods"
# tuwunel_mods is a development-only hot reload feature
"tuwunel_mods"
];
};
};
@@ -494,8 +503,8 @@
"jemalloc_stats"
# this is non-functional on nix for some reason
"hardened_malloc"
# conduwuit_mods is a development-only hot reload feature
"conduwuit_mods"
# tuwunel_mods is a development-only hot reload feature
"tuwunel_mods"
];
x86_64_haswell_target_optimised = (if (crossSystem == "x86_64-linux-gnu" || crossSystem == "x86_64-linux-musl") then true else false);
};
@@ -516,8 +525,8 @@
"experimental"
# this is non-functional on nix for some reason
"hardened_malloc"
# conduwuit_mods is a development-only hot reload feature
"conduwuit_mods"
# tuwunel_mods is a development-only hot reload feature
"tuwunel_mods"
];
};
};
@@ -564,8 +573,8 @@
"jemalloc_stats"
# this is non-functional on nix for some reason
"hardened_malloc"
# conduwuit_mods is a development-only hot reload feature
"conduwuit_mods"
# tuwunel_mods is a development-only hot reload feature
"tuwunel_mods"
];
};
}));
+5 -3
View File
@@ -13,14 +13,16 @@ stdenv.mkDerivation {
root = inputs.self;
include = [
"book.toml"
"conduwuit-example.toml"
"tuwunel-example.toml"
"CODE_OF_CONDUCT.md"
"CONTRIBUTING.md"
"README.md"
"development.md"
"debian/conduwuit.service"
"debian/tuwunel.service"
"debian/README.md"
"arch/conduwuit.service"
"arch/tuwunel.service"
"rpm/tuwunel.service"
"rpm/README.md"
"docs"
"theme"
];
+4 -4
View File
@@ -15,7 +15,7 @@ LOG_FILE="${2:-complement_test_logs.jsonl}"
# A `.jsonl` file to write test results to
RESULTS_FILE="${3:-complement_test_results.jsonl}"
COMPLEMENT_BASE_IMAGE="${COMPLEMENT_BASE_IMAGE:-complement-conduwuit:main}"
COMPLEMENT_BASE_IMAGE="${COMPLEMENT_BASE_IMAGE:-complement-tuwunel:main}"
# Complement tests that are skipped due to flakiness/reliability issues or we don't implement such features and won't for a long time
SKIPPED_COMPLEMENT_TESTS='TestPartialStateJoin.*|TestRoomDeleteAlias/Parallel/Regular_users_can_add_and_delete_aliases_when_m.*|TestRoomDeleteAlias/Parallel/Can_delete_canonical_alias|TestUnbanViaInvite.*|TestRoomState/Parallel/GET_/publicRooms_lists.*"|TestRoomDeleteAlias/Parallel/Users_with_sufficient_power-level_can_delete_other.*'
@@ -35,20 +35,20 @@ toplevel="$(git rev-parse --show-toplevel)"
pushd "$toplevel" > /dev/null
if [ ! -f "complement_oci_image.tar.gz" ]; then
echo "building complement conduwuit image"
echo "building complement tuwunel image"
# if using macOS, use linux-complement
#bin/nix-build-and-cache just .#linux-complement
bin/nix-build-and-cache just .#complement
#nix build -L .#complement
echo "complement conduwuit image tar.gz built at \"result\""
echo "complement tuwunel image tar.gz built at \"result\""
echo "loading into docker"
docker load < result
popd > /dev/null
else
echo "skipping building a complement conduwuit image as complement_oci_image.tar.gz was already found, loading this"
echo "skipping building a complement tuwunel image as complement_oci_image.tar.gz was already found, loading this"
docker load < complement_oci_image.tar.gz
popd > /dev/null
+7 -7
View File
@@ -18,7 +18,7 @@ just() {
return
fi
# historical "conduit" store for compatibility purposes, same as conduwuit
# historical "conduit" store for compatibility purposes, same as tuwunel
nix run --inputs-from "$toplevel" attic -- \
login \
conduit \
@@ -52,26 +52,26 @@ just() {
withattic conduit || withattic conduit || withattic conduit || true
)
# main "conduwuit" store
# main "tuwunel" store
nix run --inputs-from "$toplevel" attic -- \
login \
conduwuit \
"${ATTIC_ENDPOINT:-https://attic.kennel.juneis.dog/conduwuit}" \
tuwunel \
"${ATTIC_ENDPOINT:-https://attic.kennel.juneis.dog/tuwunel}" \
"$ATTIC_TOKEN"
# Upload them to Attic (conduwuit store) and Cachix
# Upload them to Attic (tuwunel store) and Cachix
#
# Use `xargs` and a here-string because something would probably explode if
# several thousand arguments got passed to a command at once. Hopefully no
# store paths include a newline in them.
(
IFS=$'\n'
withattic conduwuit || withattic conduwuit || withattic conduwuit || true
withattic tuwunel || withattic tuwunel || withattic tuwunel || true
# push to cachix if available
if [ "$CACHIX_AUTH_TOKEN" ]; then
nix shell --inputs-from "$toplevel" cachix -c xargs \
cachix push conduwuit <<< "${cache[*]}"
cachix push tuwunel <<< "${cache[*]}"
fi
)
}
+6 -6
View File
@@ -31,7 +31,7 @@ let
"gzip_compression"
"zstd_compression"
# complement doesn't need hot reloading
"conduwuit_mods"
"tuwunel_mods"
# complement doesn't have URL preview media tests
"url_preview"
];
@@ -41,13 +41,13 @@ let
set -euxo pipefail
${lib.getExe' coreutils "env"} \
CONDUWUIT_SERVER_NAME="$SERVER_NAME" \
TUWUNEL_SERVER_NAME="$SERVER_NAME" \
${lib.getExe main'}
'';
in
dockerTools.buildImage {
name = "complement-conduwuit";
name = "complement-tuwunel";
tag = "main";
copyToRoot = buildEnv {
@@ -75,9 +75,9 @@ dockerTools.buildImage {
else [];
Env = [
"CONDUWUIT_TLS__KEY=${./private_key.key}"
"CONDUWUIT_TLS__CERTS=${./certificate.crt}"
"CONDUWUIT_CONFIG=${./config.toml}"
"TUWUNEL_TLS__KEY=${./private_key.key}"
"TUWUNEL_TLS__CERTS=${./certificate.crt}"
"TUWUNEL_CONFIG=${./config.toml}"
"RUST_BACKTRACE=full"
];
+3 -3
View File
@@ -25,8 +25,8 @@
"jemalloc_stats"
# this is non-functional on nix for some reason
"hardened_malloc"
# conduwuit_mods is a development-only hot reload feature
"conduwuit_mods"
# tuwunel_mods is a development-only hot reload feature
"tuwunel_mods"
]
, disable_release_max_log_level ? false
, features ? []
@@ -130,7 +130,7 @@ buildDepsOnlyEnv =
});
buildPackageEnv = {
CONDUWUIT_VERSION_EXTRA = inputs.self.shortRev or inputs.self.dirtyShortRev or "";
TUWUNEL_VERSION_EXTRA = inputs.self.shortRev or inputs.self.dirtyShortRev or "";
} // buildDepsOnlyEnv // {
# Only needed in static stdenv because these are transitive dependencies of rocksdb
CARGO_BUILD_RUSTFLAGS = buildDepsOnlyEnv.CARGO_BUILD_RUSTFLAGS
+36
View File
@@ -0,0 +1,36 @@
# Tuwunel for Red Hat
Information about downloading and deploying the Red Hat package. This may also be
referenced for other `rpm`-based distros such as CentOS.
### Installation
It is recommended to see the [generic deployment guide](../deploying/generic.md)
for further information if needed as usage of the RPM package is generally
related.
No `rpm` repository is currently offered yet, it is in the works/development.
### Configuration
When installed, the example config is placed at `/etc/tuwunel/tuwunel.toml`
as the default config. The config mentions things required to be changed before
starting.
You can tweak more detailed settings by uncommenting and setting the config
options in `/etc/tuwunel/tuwunel.toml`.
### Running
The package uses the [`tuwunel.service`](../configuration/examples.md#example-systemd-unit-file)
systemd unit file to start and stop Tuwunel. The binary is installed at `/usr/sbin/tuwunel`.
This package assumes by default that Tuwunel will be placed behind a reverse
proxy. The default config options apply (listening on `localhost` and TCP port
`8008`). Matrix federation requires a valid domain name and TLS, so you will
need to set up TLS certificates and renewal for it to work properly if you
intend to federate.
Consult various online documentation and guides on setting up a reverse proxy
and TLS. Caddy is documented at the [generic deployment guide](../deploying/generic.md#setting-up-the-reverse-proxy)
as it's the easiest and most user friendly.
+41
View File
@@ -0,0 +1,41 @@
#!/bin/sh
set -e
TUWUNEL_DATABASE_PATH=/var/lib/tuwunel
TUWUNEL_CONFIG_PATH=/etc/tuwunel
case "$1" in
configure)
# Create the `tuwunel` user if it does not exist yet.
if ! getent passwd tuwunel > /dev/null ; then
echo 'Adding system user for the tuwunel Matrix homeserver' 1>&2
useradd --system --user-group \
--create-home --home-dir "$TUWUNEL_DATABASE_PATH" \
--shell "/usr/sbin/nologin" \
tuwunel
fi
# Create the database path if it does not exist yet and fix up ownership
# and permissions for the config.
mkdir -v -p "$TUWUNEL_DATABASE_PATH"
# symlink the previous location for compatibility if it does not exist yet.
if ! test -L "/var/lib/matrix-conduit" ; then
ln -s -v "$TUWUNEL_DATABASE_PATH" "/var/lib/matrix-conduit"
fi
if ! test -L "/var/lib/conduwuit" ; then
ln -s -v "$TUWUNEL_DATABASE_PATH" "/var/lib/conduwuit"
fi
chown -v tuwunel:tuwunel -R "$TUWUNEL_DATABASE_PATH"
chown -v tuwunel:tuwunel -R "$TUWUNEL_CONFIG_PATH"
chmod -v 740 "$TUWUNEL_DATABASE_PATH"
echo ''
echo 'Make sure you edit the example config at /etc/tuwunel/tuwunel.toml before starting!'
echo 'To start the server, run: systemctl start tuwunel.service'
echo ''
;;
esac
+65
View File
@@ -0,0 +1,65 @@
[Unit]
Description=Tuwunel Matrix homeserver
Wants=network-online.target
After=network-online.target
Alias=matrix-tuwunel.service
Documentation=https://tuwunel.chat/
[Service]
DynamicUser=yes
User=tuwunel
Group=tuwunel
Type=notify
Environment="TUWUNEL_CONFIG=/etc/tuwunel/tuwunel.toml"
ExecStart=/usr/sbin/tuwunel
ReadWritePaths=/var/lib/tuwunel /etc/tuwunel
AmbientCapabilities=
CapabilityBoundingSet=
DevicePolicy=closed
LockPersonality=yes
MemoryDenyWriteExecute=yes
NoNewPrivileges=yes
#ProcSubset=pid
ProtectClock=yes
ProtectControlGroups=yes
ProtectHome=yes
ProtectHostname=yes
ProtectKernelLogs=yes
ProtectKernelModules=yes
ProtectKernelTunables=yes
ProtectProc=invisible
ProtectSystem=strict
PrivateDevices=yes
PrivateMounts=yes
PrivateTmp=yes
PrivateUsers=yes
PrivateIPC=yes
RemoveIPC=yes
RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX
RestrictNamespaces=yes
RestrictRealtime=yes
RestrictSUIDSGID=yes
SystemCallArchitectures=native
SystemCallFilter=@system-service @resources
SystemCallFilter=~@clock @debug @module @mount @reboot @swap @cpu-emulation @obsolete @timer @chown @setuid @privileged @keyring @ipc
SystemCallErrorNumber=EPERM
RuntimeDirectory=tuwunel
RuntimeDirectoryMode=0750
Restart=on-failure
RestartSec=5
TimeoutStopSec=2m
TimeoutStartSec=2m
StartLimitInterval=1m
StartLimitBurst=5
[Install]
WantedBy=multi-user.target
+2
View File
@@ -81,9 +81,11 @@ tuwunel-database.workspace = true
tuwunel-macros.workspace = true
tuwunel-service.workspace = true
const-str.workspace = true
ctor.workspace = true
futures.workspace = true
log.workspace = true
ruma.workspace = true
serde.workspace = true
serde_json.workspace = true
serde_yaml.workspace = true
tokio.workspace = true
+61 -1
View File
@@ -2,6 +2,7 @@ use std::{
collections::HashMap,
fmt::Write,
iter::once,
str::FromStr,
time::{Instant, SystemTime},
};
@@ -11,9 +12,10 @@ use ruma::{
OwnedRoomOrAliasId, OwnedServerName, RoomId, RoomVersionId,
api::federation::event::get_room_state, events::AnyStateEvent, serde::Raw,
};
use serde::Serialize;
use tracing_subscriber::EnvFilter;
use tuwunel_core::{
Err, Result, debug_error, err, info,
Err, Result, debug_error, err, info, jwt,
matrix::{
Event,
pdu::{PduEvent, PduId, RawPduId},
@@ -22,6 +24,7 @@ use tuwunel_core::{
utils::{
stream::{IterStream, ReadyExt},
string::EMPTY,
time::now_secs,
},
warn,
};
@@ -975,3 +978,60 @@ pub(super) async fn trim_memory(&self) -> Result {
writeln!(self, "done").await
}
#[admin_command]
pub(super) async fn create_jwt(
&self,
user: String,
exp_from_now: Option<u64>,
nbf_from_now: Option<u64>,
issuer: Option<String>,
audience: Option<String>,
) -> Result {
use jwt::{Algorithm, EncodingKey, Header, encode};
#[derive(Serialize)]
struct Claim {
sub: String,
iss: String,
aud: String,
exp: usize,
nbf: usize,
}
let config = &self.services.config.jwt;
if config.format.as_str() != "HMAC" {
return Err!("This command only supports HMAC key format, not {}.", config.format);
}
let key = EncodingKey::from_secret(config.key.as_ref());
let alg = Algorithm::from_str(config.algorithm.as_str()).map_err(|e| {
err!(Config("jwt.algorithm", "JWT algorithm is not recognized or configured {e}"))
})?;
let header = Header { alg, ..Default::default() };
let claim = Claim {
sub: user,
iss: issuer.unwrap_or_default(),
aud: audience.unwrap_or_default(),
exp: exp_from_now
.and_then(|val| now_secs().checked_add(val))
.map(TryInto::try_into)
.and_then(Result::ok)
.unwrap_or(usize::MAX),
nbf: nbf_from_now
.and_then(|val| now_secs().checked_add(val))
.map(TryInto::try_into)
.and_then(Result::ok)
.unwrap_or(0),
};
encode(&header, &claim, &key)
.map_err(|e| err!("Failed to encode JWT: {e}"))
.map(async |token| self.write_str(&token).await)?
.await
}
+22
View File
@@ -234,6 +234,28 @@ pub(super) enum DebugCommand {
level: Option<i32>,
},
/// - Create a JWT token for login
CreateJwt {
/// Localpart of the user's MXID
user: String,
/// Set expiration time in seconds from now.
#[arg(long)]
exp_from_now: Option<u64>,
/// Set not-before time in seconds from now.
#[arg(long)]
nbf_from_now: Option<u64>,
/// Claim an issuer.
#[arg(long)]
issuer: Option<String>,
/// Claim an audience.
#[arg(long)]
audience: Option<String>,
},
/// - Developer test stubs
#[command(subcommand)]
#[allow(non_snake_case)]
+1
View File
@@ -73,6 +73,7 @@ axum-extra.workspace = true
axum.workspace = true
base64.workspace = true
bytes.workspace = true
ctor.workspace = true
tuwunel-core.workspace = true
tuwunel-service.workspace = true
const-str.workspace = true
-455
View File
@@ -1,455 +0,0 @@
use std::time::Duration;
use axum::extract::State;
use axum_client_ip::InsecureClientIp;
use futures::{FutureExt, StreamExt, TryFutureExt};
use ruma::{
OwnedUserId, UserId,
api::client::{
session::{
get_login_token,
get_login_types::{
self,
v3::{ApplicationServiceLoginType, PasswordLoginType, TokenLoginType},
},
login::{
self,
v3::{DiscoveryInfo, HomeserverInfo},
},
logout, logout_all,
},
uiaa,
},
};
use tuwunel_core::{
Err, Error, Result, debug, debug_error, err, info, utils,
utils::{hash, stream::ReadyExt},
};
use tuwunel_service::{Services, uiaa::SESSION_ID_LENGTH};
use super::{DEVICE_ID_LENGTH, TOKEN_LENGTH};
use crate::Ruma;
/// # `GET /_matrix/client/v3/login`
///
/// Get the supported login types of this server. One of these should be used as
/// the `type` field when logging in.
#[tracing::instrument(skip_all, fields(%client), name = "login")]
pub(crate) async fn get_login_types_route(
State(services): State<crate::State>,
InsecureClientIp(client): InsecureClientIp,
_body: Ruma<get_login_types::v3::Request>,
) -> Result<get_login_types::v3::Response> {
Ok(get_login_types::v3::Response::new(vec![
get_login_types::v3::LoginType::Password(PasswordLoginType::default()),
get_login_types::v3::LoginType::ApplicationService(ApplicationServiceLoginType::default()),
get_login_types::v3::LoginType::Token(TokenLoginType {
get_login_token: services.server.config.login_via_existing_session,
}),
]))
}
/// Authenticates the given user by its ID and its password.
///
/// Returns the user ID if successful, and an error otherwise.
#[tracing::instrument(skip_all, fields(%user_id), name = "password")]
async fn password_login(
services: &Services,
user_id: &UserId,
lowercased_user_id: &UserId,
password: &str,
) -> Result<OwnedUserId> {
let (hash, user_id) = services
.users
.password_hash(user_id)
.map_ok(|hash| (hash, user_id))
.or_else(|_| {
services
.users
.password_hash(lowercased_user_id)
.map_ok(|hash| (hash, lowercased_user_id))
})
.await?;
if hash.is_empty() {
return Err!(Request(UserDeactivated("The user has been deactivated")));
}
hash::verify_password(password, &hash)
.inspect_err(|e| debug_error!("{e}"))
.map_err(|_| err!(Request(Forbidden("Wrong username or password."))))?;
Ok(user_id.to_owned())
}
/// Authenticates the given user through the configured LDAP server.
///
/// Creates the user if the user is found in the LDAP and do not already have an
/// account.
#[tracing::instrument(skip_all, fields(%user_id), name = "ldap")]
async fn ldap_login(
services: &Services,
user_id: &UserId,
lowercased_user_id: &UserId,
password: &str,
) -> Result<OwnedUserId> {
let (user_dn, is_ldap_admin) = match services.config.ldap.bind_dn.as_ref() {
| Some(bind_dn) if bind_dn.contains("{username}") =>
(bind_dn.replace("{username}", lowercased_user_id.localpart()), false),
| _ => {
debug!("Searching user in LDAP");
let dns = services.users.search_ldap(user_id).await?;
if dns.len() >= 2 {
return Err!(Ldap("LDAP search returned two or more results"));
}
let Some((user_dn, is_admin)) = dns.first() else {
return password_login(services, user_id, lowercased_user_id, password).await;
};
(user_dn.clone(), *is_admin)
},
};
let user_id = services
.users
.auth_ldap(&user_dn, password)
.await
.map(|()| lowercased_user_id.to_owned())?;
// LDAP users are automatically created on first login attempt. This is a very
// common feature that can be seen on many services using a LDAP provider for
// their users (synapse, Nextcloud, Jellyfin, ...).
//
// LDAP users are crated with a dummy password but non empty because an empty
// password is reserved for deactivated accounts. The tuwunel password field
// will never be read to login a LDAP user so it's not an issue.
if !services.users.exists(lowercased_user_id).await {
services
.users
.create(lowercased_user_id, Some("*"), Some("ldap"))
.await?;
}
let is_tuwunel_admin = services
.admin
.user_is_admin(lowercased_user_id)
.await;
if is_ldap_admin && !is_tuwunel_admin {
services
.admin
.make_user_admin(lowercased_user_id)
.await?;
} else if !is_ldap_admin && is_tuwunel_admin {
services
.admin
.revoke_admin(lowercased_user_id)
.await?;
}
Ok(user_id)
}
/// # `POST /_matrix/client/v3/login`
///
/// Authenticates the user and returns an access token it can use in subsequent
/// requests.
///
/// - The user needs to authenticate using their password (or if enabled using a
/// json web token)
/// - If `device_id` is known: invalidates old access token of that device
/// - If `device_id` is unknown: creates a new device
/// - Returns access token that is associated with the user and device
///
/// Note: You can use [`GET
/// /_matrix/client/r0/login`](fn.get_supported_versions_route.html) to see
/// supported login types.
#[tracing::instrument(skip_all, fields(%client), name = "login")]
pub(crate) async fn login_route(
State(services): State<crate::State>,
InsecureClientIp(client): InsecureClientIp,
body: Ruma<login::v3::Request>,
) -> Result<login::v3::Response> {
let emergency_mode_enabled = services.config.emergency_password.is_some();
// Validate login method
// TODO: Other login methods
let user_id = match &body.login_info {
#[allow(deprecated)]
| login::v3::LoginInfo::Password(login::v3::Password {
identifier,
password,
user,
..
}) => {
debug!("Got password login type");
let user_id =
if let Some(uiaa::UserIdentifier::UserIdOrLocalpart(user_id)) = identifier {
UserId::parse_with_server_name(user_id, &services.config.server_name)
} else if let Some(user) = user {
UserId::parse_with_server_name(user, &services.config.server_name)
} else {
return Err!(Request(Unknown(
debug_warn!(?body.login_info, "Valid identifier or username was not provided (invalid or unsupported login type?)")
)));
}
.map_err(|e| err!(Request(InvalidUsername(warn!("Username is invalid: {e}")))))?;
let lowercased_user_id = UserId::parse_with_server_name(
user_id.localpart().to_lowercase(),
&services.config.server_name,
)?;
if !services.globals.user_is_local(&user_id)
|| !services
.globals
.user_is_local(&lowercased_user_id)
{
return Err!(Request(Unknown("User ID does not belong to this homeserver")));
}
if cfg!(feature = "ldap") && services.config.ldap.enable {
ldap_login(&services, &user_id, &lowercased_user_id, password)
.boxed()
.await?
} else {
password_login(&services, &user_id, &lowercased_user_id, password)
.boxed()
.await?
}
},
| login::v3::LoginInfo::Token(login::v3::Token { token }) => {
debug!("Got token login type");
if !services.server.config.login_via_existing_session {
return Err!(Request(Unknown("Token login is not enabled.")));
}
services
.users
.find_from_login_token(token)
.await?
},
#[allow(deprecated)]
| login::v3::LoginInfo::ApplicationService(login::v3::ApplicationService {
identifier,
user,
}) => {
debug!("Got appservice login type");
let Some(ref info) = body.appservice_info else {
return Err!(Request(MissingToken("Missing appservice token.")));
};
let user_id =
if let Some(uiaa::UserIdentifier::UserIdOrLocalpart(user_id)) = identifier {
UserId::parse_with_server_name(user_id, &services.config.server_name)
} else if let Some(user) = user {
UserId::parse_with_server_name(user, &services.config.server_name)
} else {
return Err!(Request(Unknown(
debug_warn!(?body.login_info, "Valid identifier or username was not provided (invalid or unsupported login type?)")
)));
}
.map_err(|e| err!(Request(InvalidUsername(warn!("Username is invalid: {e}")))))?;
if !services.globals.user_is_local(&user_id) {
return Err!(Request(Unknown("User ID does not belong to this homeserver")));
}
if !info.is_user_match(&user_id) && !emergency_mode_enabled {
return Err!(Request(Exclusive("Username is not in an appservice namespace.")));
}
user_id
},
| _ => {
debug!("/login json_body: {:?}", &body.json_body);
return Err!(Request(Unknown(
debug_warn!(?body.login_info, "Invalid or unsupported login type")
)));
},
};
// Generate new device id if the user didn't specify one
let device_id = body
.device_id
.clone()
.unwrap_or_else(|| utils::random_string(DEVICE_ID_LENGTH).into());
// Generate a new token for the device
let token = utils::random_string(TOKEN_LENGTH);
// Determine if device_id was provided and exists in the db for this user
let device_exists = if body.device_id.is_some() {
services
.users
.all_device_ids(&user_id)
.ready_any(|v| v == device_id)
.await
} else {
false
};
if device_exists {
services
.users
.set_token(&user_id, &device_id, &token)
.await?;
} else {
services
.users
.create_device(
&user_id,
&device_id,
&token,
body.initial_device_display_name.clone(),
Some(client.to_string()),
)
.await?;
}
// send client well-known if specified so the client knows to reconfigure itself
let client_discovery_info: Option<DiscoveryInfo> = services
.server
.config
.well_known
.client
.as_ref()
.map(|server| DiscoveryInfo::new(HomeserverInfo::new(server.to_string())));
info!("{user_id} logged in");
#[allow(deprecated)]
Ok(login::v3::Response {
user_id,
access_token: token,
device_id,
well_known: client_discovery_info,
expires_in: None,
home_server: Some(services.config.server_name.clone()),
refresh_token: None,
})
}
/// # `POST /_matrix/client/v1/login/get_token`
///
/// Allows a logged-in user to get a short-lived token which can be used
/// to log in with the m.login.token flow.
///
/// <https://spec.matrix.org/v1.13/client-server-api/#post_matrixclientv1loginget_token>
#[tracing::instrument(skip_all, fields(%client), name = "login_token")]
pub(crate) async fn login_token_route(
State(services): State<crate::State>,
InsecureClientIp(client): InsecureClientIp,
body: Ruma<get_login_token::v1::Request>,
) -> Result<get_login_token::v1::Response> {
if !services.server.config.login_via_existing_session {
return Err!(Request(Forbidden("Login via an existing session is not enabled")));
}
// This route SHOULD have UIA
// TODO: How do we make only UIA sessions that have not been used before valid?
let (sender_user, sender_device) = body.sender();
let mut uiaainfo = uiaa::UiaaInfo {
flows: vec![uiaa::AuthFlow { stages: vec![uiaa::AuthType::Password] }],
completed: Vec::new(),
params: Box::default(),
session: None,
auth_error: None,
};
match &body.auth {
| Some(auth) => {
let (worked, uiaainfo) = services
.uiaa
.try_auth(sender_user, sender_device, auth, &uiaainfo)
.await?;
if !worked {
return Err(Error::Uiaa(uiaainfo));
}
// Success!
},
| _ => match body.json_body.as_ref() {
| Some(json) => {
uiaainfo.session = Some(utils::random_string(SESSION_ID_LENGTH));
services
.uiaa
.create(sender_user, sender_device, &uiaainfo, json);
return Err(Error::Uiaa(uiaainfo));
},
| _ => {
return Err!(Request(NotJson("No JSON body was sent when required.")));
},
},
}
let login_token = utils::random_string(TOKEN_LENGTH);
let expires_in = services
.users
.create_login_token(sender_user, &login_token);
Ok(get_login_token::v1::Response {
expires_in: Duration::from_millis(expires_in),
login_token,
})
}
/// # `POST /_matrix/client/v3/logout`
///
/// Log out the current device.
///
/// - Invalidates access token
/// - Deletes device metadata (device id, device display name, last seen ip,
/// last seen ts)
/// - Forgets to-device events
/// - Triggers device list updates
#[tracing::instrument(skip_all, fields(%client), name = "logout")]
pub(crate) async fn logout_route(
State(services): State<crate::State>,
InsecureClientIp(client): InsecureClientIp,
body: Ruma<logout::v3::Request>,
) -> Result<logout::v3::Response> {
services
.users
.remove_device(body.sender_user(), body.sender_device())
.await;
Ok(logout::v3::Response::new())
}
/// # `POST /_matrix/client/r0/logout/all`
///
/// Log out all devices of this user.
///
/// - Invalidates all access tokens
/// - Deletes all device metadata (device id, device display name, last seen ip,
/// last seen ts)
/// - Forgets all to-device events
/// - Triggers device list updates
///
/// Note: This is equivalent to calling [`GET
/// /_matrix/client/r0/logout`](fn.logout_route.html) from each device of this
/// user.
#[tracing::instrument(skip_all, fields(%client), name = "logout")]
pub(crate) async fn logout_all_route(
State(services): State<crate::State>,
InsecureClientIp(client): InsecureClientIp,
body: Ruma<logout_all::v3::Request>,
) -> Result<logout_all::v3::Response> {
services
.users
.all_device_ids(body.sender_user())
.for_each(|device_id| {
services
.users
.remove_device(body.sender_user(), device_id)
})
.await;
Ok(logout_all::v3::Response::new())
}
+48
View File
@@ -0,0 +1,48 @@
use ruma::{
OwnedUserId, UserId,
api::client::{
session::login::v3::{ApplicationService, Request},
uiaa,
},
};
use tuwunel_core::{Err, Result, err};
use tuwunel_service::Services;
use crate::Ruma;
pub(super) async fn handle_login(
services: &Services,
body: &Ruma<Request>,
info: &ApplicationService,
) -> Result<OwnedUserId> {
#[allow(deprecated)]
let ApplicationService { identifier, user } = info;
let Some(ref info) = body.appservice_info else {
return Err!(Request(MissingToken("Missing appservice token.")));
};
let user_id = if let Some(uiaa::UserIdentifier::UserIdOrLocalpart(user_id)) = identifier {
UserId::parse_with_server_name(user_id, &services.config.server_name)
} else if let Some(user) = user {
UserId::parse_with_server_name(user, &services.config.server_name)
} else {
return Err!(Request(Unknown(debug_warn!(
?body.login_info,
"Valid identifier or username was not provided (invalid or unsupported login type?)"
))));
}
.map_err(|e| err!(Request(InvalidUsername(warn!("Username is invalid: {e}")))))?;
if !services.globals.user_is_local(&user_id) {
return Err!(Request(Unknown("User ID does not belong to this homeserver")));
}
let emergency_mode_enabled = services.config.emergency_password.is_some();
if !info.is_user_match(&user_id) && !emergency_mode_enabled {
return Err!(Request(Exclusive("Username is not in an appservice namespace.")));
}
Ok(user_id)
}
+117
View File
@@ -0,0 +1,117 @@
use std::str::FromStr;
use jwt::{Algorithm, DecodingKey, Validation, decode};
use ruma::{
OwnedUserId, UserId,
api::client::session::login::v3::{Request, Token},
};
use serde::Deserialize;
use tuwunel_core::{Err, Result, at, config::JwtConfig, debug, err, jwt, warn};
use tuwunel_service::Services;
use crate::Ruma;
#[derive(Debug, Deserialize)]
struct Claim {
/// Subject is the localpart of the User MXID
sub: String,
}
pub(super) async fn handle_login(
services: &Services,
_body: &Ruma<Request>,
info: &Token,
) -> Result<OwnedUserId> {
let config = &services.config.jwt;
if !config.enable {
return Err!(Request(Unknown("JWT login is not enabled.")));
}
let claim = validate(config, &info.token)?;
let local = claim.sub.to_lowercase();
let server = &services.server.name;
let user_id = UserId::parse_with_server_name(local, server).map_err(|e| {
err!(Request(InvalidUsername("JWT subject is not a valid user MXID: {e}")))
})?;
if !services.users.exists(&user_id).await {
if !config.register_user {
return Err!(Request(NotFound("User {user_id} is not registered on this server.")));
}
services
.users
.create(&user_id, Some("*"), Some("jwt"))
.await?;
}
Ok(user_id)
}
fn validate(config: &JwtConfig, token: &str) -> Result<Claim> {
let verifier = init_verifier(config)?;
let validator = init_validator(config)?;
decode::<Claim>(token, &verifier, &validator)
.map(|decoded| (decoded.header, decoded.claims))
.inspect(|(head, claim)| debug!(?head, ?claim, "JWT token decoded"))
.map_err(|e| err!(Request(Forbidden("Invalid JWT token: {e}"))))
.map(at!(1))
}
fn init_verifier(config: &JwtConfig) -> Result<DecodingKey> {
let key = &config.key;
let format = config.format.as_str();
Ok(match format {
| "HMAC" => DecodingKey::from_secret(key.as_bytes()),
| "HMACB64" => DecodingKey::from_base64_secret(key.as_str())
.map_err(|e| err!(Config("jwt.key", "JWT key is not valid base64: {e}")))?,
| "ECDSA" => DecodingKey::from_ec_pem(key.as_bytes())
.map_err(|e| err!(Config("jwt.key", "JWT key is not valid PEM: {e}")))?,
| _ => return Err!(Config("jwt.format", "Key format {format:?} is not supported.")),
})
}
fn init_validator(config: &JwtConfig) -> Result<Validation> {
let alg = config.algorithm.as_str();
let alg = Algorithm::from_str(alg).map_err(|e| {
err!(Config("jwt.algorithm", "JWT algorithm is not recognized or configured {e}"))
})?;
let mut validator = Validation::new(alg);
let mut required_spec_claims: Vec<_> = ["sub"].into();
validator.validate_exp = config.validate_exp;
if config.require_exp {
required_spec_claims.push("exp");
}
validator.validate_nbf = config.validate_nbf;
if config.require_nbf {
required_spec_claims.push("nbf");
}
if !config.audience.is_empty() {
required_spec_claims.push("aud");
validator.set_audience(&config.audience);
}
if !config.issuer.is_empty() {
required_spec_claims.push("iss");
validator.set_issuer(&config.issuer);
}
if cfg!(debug_assertions) && !config.validate_signature {
warn!("JWT signature validation is disabled!");
validator.insecure_disable_signature_validation();
}
validator.set_required_spec_claims(&required_spec_claims);
debug!(?validator, "JWT configured");
Ok(validator)
}
+75
View File
@@ -0,0 +1,75 @@
use ruma::{OwnedUserId, UserId};
use tuwunel_core::{Err, Result, debug};
use tuwunel_service::Services;
use super::password_login;
/// Authenticates the given user through the configured LDAP server.
///
/// Creates the user if the user is found in the LDAP and do not already have an
/// account.
#[tracing::instrument(skip_all, fields(%user_id), name = "ldap")]
pub(super) async fn ldap_login(
services: &Services,
user_id: &UserId,
lowercased_user_id: &UserId,
password: &str,
) -> Result<OwnedUserId> {
let (user_dn, is_ldap_admin) = match services.config.ldap.bind_dn.as_ref() {
| Some(bind_dn) if bind_dn.contains("{username}") =>
(bind_dn.replace("{username}", lowercased_user_id.localpart()), false),
| _ => {
debug!("Searching user in LDAP");
let dns = services.users.search_ldap(user_id).await?;
if dns.len() >= 2 {
return Err!(Ldap("LDAP search returned two or more results"));
}
let Some((user_dn, is_admin)) = dns.first() else {
return password_login(services, user_id, lowercased_user_id, password).await;
};
(user_dn.clone(), *is_admin)
},
};
let user_id = services
.users
.auth_ldap(&user_dn, password)
.await
.map(|()| lowercased_user_id.to_owned())?;
// LDAP users are automatically created on first login attempt. This is a very
// common feature that can be seen on many services using a LDAP provider for
// their users (synapse, Nextcloud, Jellyfin, ...).
//
// LDAP users are crated with a dummy password but non empty because an empty
// password is reserved for deactivated accounts. The tuwunel password field
// will never be read to login a LDAP user so it's not an issue.
if !services.users.exists(lowercased_user_id).await {
services
.users
.create(lowercased_user_id, Some("*"), Some("ldap"))
.await?;
}
let is_tuwunel_admin = services
.admin
.user_is_admin(lowercased_user_id)
.await;
if is_ldap_admin && !is_tuwunel_admin {
services
.admin
.make_user_admin(lowercased_user_id)
.await?;
} else if !is_ldap_admin && is_tuwunel_admin {
services
.admin
.revoke_admin(lowercased_user_id)
.await?;
}
Ok(user_id)
}
+62
View File
@@ -0,0 +1,62 @@
use axum::extract::State;
use axum_client_ip::InsecureClientIp;
use futures::StreamExt;
use ruma::api::client::session::{logout, logout_all};
use tuwunel_core::Result;
use crate::Ruma;
/// # `POST /_matrix/client/v3/logout`
///
/// Log out the current device.
///
/// - Invalidates access token
/// - Deletes device metadata (device id, device display name, last seen ip,
/// last seen ts)
/// - Forgets to-device events
/// - Triggers device list updates
#[tracing::instrument(skip_all, fields(%client), name = "logout")]
pub(crate) async fn logout_route(
State(services): State<crate::State>,
InsecureClientIp(client): InsecureClientIp,
body: Ruma<logout::v3::Request>,
) -> Result<logout::v3::Response> {
services
.users
.remove_device(body.sender_user(), body.sender_device())
.await;
Ok(logout::v3::Response::new())
}
/// # `POST /_matrix/client/r0/logout/all`
///
/// Log out all devices of this user.
///
/// - Invalidates all access tokens
/// - Deletes all device metadata (device id, device display name, last seen ip,
/// last seen ts)
/// - Forgets all to-device events
/// - Triggers device list updates
///
/// Note: This is equivalent to calling [`GET
/// /_matrix/client/r0/logout`](fn.logout_route.html) from each device of this
/// user.
#[tracing::instrument(skip_all, fields(%client), name = "logout")]
pub(crate) async fn logout_all_route(
State(services): State<crate::State>,
InsecureClientIp(client): InsecureClientIp,
body: Ruma<logout_all::v3::Request>,
) -> Result<logout_all::v3::Response> {
services
.users
.all_device_ids(body.sender_user())
.for_each(|device_id| {
services
.users
.remove_device(body.sender_user(), device_id)
})
.await;
Ok(logout_all::v3::Response::new())
}
+147
View File
@@ -0,0 +1,147 @@
mod appservice;
mod jwt;
mod ldap;
mod logout;
mod password;
mod token;
use axum::extract::State;
use axum_client_ip::InsecureClientIp;
use ruma::api::client::session::{
get_login_types::{
self,
v3::{
ApplicationServiceLoginType, JwtLoginType, LoginType, PasswordLoginType,
TokenLoginType,
},
},
login::{
self,
v3::{DiscoveryInfo, HomeserverInfo, LoginInfo},
},
};
use tuwunel_core::{Err, Result, info, utils, utils::stream::ReadyExt};
use self::{ldap::ldap_login, password::password_login};
pub(crate) use self::{
logout::{logout_all_route, logout_route},
token::login_token_route,
};
use super::{DEVICE_ID_LENGTH, TOKEN_LENGTH};
use crate::Ruma;
/// # `GET /_matrix/client/v3/login`
///
/// Get the supported login types of this server. One of these should be used as
/// the `type` field when logging in.
#[tracing::instrument(skip_all, fields(%client), name = "login")]
pub(crate) async fn get_login_types_route(
State(services): State<crate::State>,
InsecureClientIp(client): InsecureClientIp,
_body: Ruma<get_login_types::v3::Request>,
) -> Result<get_login_types::v3::Response> {
Ok(get_login_types::v3::Response::new(vec![
LoginType::Password(PasswordLoginType::default()),
LoginType::ApplicationService(ApplicationServiceLoginType::default()),
LoginType::Jwt(JwtLoginType::default()),
LoginType::Token(TokenLoginType {
get_login_token: services.config.login_via_existing_session,
}),
]))
}
/// # `POST /_matrix/client/v3/login`
///
/// Authenticates the user and returns an access token it can use in subsequent
/// requests.
///
/// - The user needs to authenticate using their password (or if enabled using a
/// json web token)
/// - If `device_id` is known: invalidates old access token of that device
/// - If `device_id` is unknown: creates a new device
/// - Returns access token that is associated with the user and device
///
/// Note: You can use [`GET
/// /_matrix/client/r0/login`](fn.get_supported_versions_route.html) to see
/// supported login types.
#[tracing::instrument(name = "login", skip_all, fields(%client, ?body.login_info))]
pub(crate) async fn login_route(
State(services): State<crate::State>,
InsecureClientIp(client): InsecureClientIp,
body: Ruma<login::v3::Request>,
) -> Result<login::v3::Response> {
// Validate login method
let user_id = match &body.login_info {
| LoginInfo::Password(info) => password::handle_login(&services, &body, info).await?,
| LoginInfo::Token(info) => token::handle_login(&services, &body, info).await?,
| LoginInfo::Jwt(info) => jwt::handle_login(&services, &body, info).await?,
| LoginInfo::ApplicationService(info) =>
appservice::handle_login(&services, &body, info).await?,
| _ => {
return Err!(Request(Unknown(debug_warn!(
?body.login_info,
?body.json_body,
"Invalid or unsupported login type",
))));
},
};
// Generate a new token for the device
let access_token = utils::random_string(TOKEN_LENGTH);
// Generate new device id if the user didn't specify one
let device_id = body
.device_id
.clone()
.unwrap_or_else(|| utils::random_string(DEVICE_ID_LENGTH).into());
// Determine if device_id was provided and exists in the db for this user
let device_exists = services
.users
.all_device_ids(&user_id)
.ready_any(|v| v == device_id)
.await;
if !device_exists {
services
.users
.create_device(
&user_id,
&device_id,
&access_token,
body.initial_device_display_name.clone(),
Some(client.to_string()),
)
.await?;
} else {
services
.users
.set_access_token(&user_id, &device_id, &access_token)
.await?;
}
info!("{user_id} logged in");
let home_server = services.server.name.clone().into();
// send client well-known if specified so the client knows to reconfigure itself
let well_known: Option<DiscoveryInfo> = services
.config
.well_known
.client
.as_ref()
.map(ToString::to_string)
.map(HomeserverInfo::new)
.map(DiscoveryInfo::new);
#[allow(deprecated)]
Ok(login::v3::Response {
user_id,
access_token,
device_id,
home_server,
well_known,
expires_in: None,
refresh_token: None,
})
}
+100
View File
@@ -0,0 +1,100 @@
use futures::{FutureExt, TryFutureExt};
use ruma::{
OwnedUserId, UserId,
api::client::{
session::login::v3::{Password, Request},
uiaa,
},
};
use tuwunel_core::{Err, Result, debug_error, err, utils::hash};
use tuwunel_service::Services;
use super::ldap_login;
use crate::Ruma;
pub(super) async fn handle_login(
services: &Services,
body: &Ruma<Request>,
info: &Password,
) -> Result<OwnedUserId> {
#[allow(deprecated)]
let Password { identifier, password, user, .. } = info;
let user_id = if let Some(uiaa::UserIdentifier::UserIdOrLocalpart(user_id)) = identifier {
UserId::parse_with_server_name(user_id, &services.config.server_name)
} else if let Some(user) = user {
UserId::parse_with_server_name(user, &services.config.server_name)
} else {
return Err!(Request(Unknown(debug_warn!(
?body.login_info,
"Valid identifier or username was not provided (invalid or unsupported login type?)"
))));
}
.map_err(|e| err!(Request(InvalidUsername(warn!("Username is invalid: {e}")))))?;
let lowercased_user_id = UserId::parse_with_server_name(
user_id.localpart().to_lowercase(),
&services.config.server_name,
)?;
let user_is_remote = !services.globals.user_is_local(&user_id)
|| !services
.globals
.user_is_local(&lowercased_user_id);
if user_is_remote {
return Err!(Request(Unknown("User ID does not belong to this homeserver")));
}
if cfg!(feature = "ldap") && services.config.ldap.enable {
ldap_login(services, &user_id, &lowercased_user_id, password)
.boxed()
.await
} else {
password_login(services, &user_id, &lowercased_user_id, password).await
}
}
/// Authenticates the given user by its ID and its password.
///
/// Returns the user ID if successful, and an error otherwise.
#[tracing::instrument(skip_all, fields(%user_id), name = "password")]
pub(super) async fn password_login(
services: &Services,
user_id: &UserId,
lowercased_user_id: &UserId,
password: &str,
) -> Result<OwnedUserId> {
// Restrict login to accounts only of type 'password', including untyped
// legacy accounts which are equivalent to 'password'.
if services
.users
.origin(user_id)
.await
.is_ok_and(|origin| origin != "password")
{
return Err!(Request(Forbidden("Account does not permit password login.")));
}
let (hash, user_id) = services
.users
.password_hash(user_id)
.map_ok(|hash| (hash, user_id))
.or_else(|_| {
services
.users
.password_hash(lowercased_user_id)
.map_ok(|hash| (hash, lowercased_user_id))
})
.await?;
if hash.is_empty() {
return Err!(Request(UserDeactivated("The user has been deactivated")));
}
hash::verify_password(password, &hash)
.inspect_err(|e| debug_error!("{e}"))
.map_err(|_| err!(Request(Forbidden("Wrong username or password."))))?;
Ok(user_id.to_owned())
}
+100
View File
@@ -0,0 +1,100 @@
use std::time::Duration;
use axum::extract::State;
use axum_client_ip::InsecureClientIp;
use ruma::{
OwnedUserId,
api::client::{
session::{
get_login_token,
login::v3::{Request, Token},
},
uiaa,
},
};
use tuwunel_core::{Err, Result, utils::random_string};
use tuwunel_service::{Services, uiaa::SESSION_ID_LENGTH};
use super::TOKEN_LENGTH;
use crate::Ruma;
pub(super) async fn handle_login(
services: &Services,
_body: &Ruma<Request>,
info: &Token,
) -> Result<OwnedUserId> {
let Token { token } = info;
if !services.config.login_via_existing_session {
return Err!(Request(Unknown("Token login is not enabled.")));
}
services.users.find_from_login_token(token).await
}
/// # `POST /_matrix/client/v1/login/get_token`
///
/// Allows a logged-in user to get a short-lived token which can be used
/// to log in with the m.login.token flow.
///
/// <https://spec.matrix.org/v1.13/client-server-api/#post_matrixclientv1loginget_token>
#[tracing::instrument(skip_all, fields(%client), name = "login_token")]
pub(crate) async fn login_token_route(
State(services): State<crate::State>,
InsecureClientIp(client): InsecureClientIp,
body: Ruma<get_login_token::v1::Request>,
) -> Result<get_login_token::v1::Response> {
if !services.config.login_via_existing_session {
return Err!(Request(Forbidden("Login via an existing session is not enabled")));
}
// This route SHOULD have UIA
// TODO: How do we make only UIA sessions that have not been used before valid?
let (sender_user, sender_device) = body.sender();
let password_flow = uiaa::AuthFlow { stages: vec![uiaa::AuthType::Password] };
let mut uiaainfo = uiaa::UiaaInfo {
flows: vec![password_flow],
completed: Vec::new(),
params: Box::default(),
session: None,
auth_error: None,
};
match &body.auth {
| Some(auth) => {
let (worked, uiaainfo) = services
.uiaa
.try_auth(sender_user, sender_device, auth, &uiaainfo)
.await?;
if !worked {
return Err!(Uiaa(uiaainfo));
}
// Success!
},
| _ => match body.json_body.as_ref() {
| Some(json) => {
uiaainfo.session = Some(random_string(SESSION_ID_LENGTH));
services
.uiaa
.create(sender_user, sender_device, &uiaainfo, json);
return Err!(Uiaa(uiaainfo));
},
| _ => return Err!(Request(NotJson("No JSON body was sent when required."))),
},
}
let login_token = random_string(TOKEN_LENGTH);
let expires_in = services
.users
.create_login_token(sender_user, &login_token);
Ok(get_login_token::v1::Response {
expires_in: Duration::from_millis(expires_in),
login_token,
})
}
+3 -3
View File
@@ -60,7 +60,7 @@ pub(crate) async fn get_mutual_rooms_route(
})
}
/// # `DELETE /_matrix/client/unstable/uk.tcpip.msc4133/profile/:user_id/us.cloke.msc4175.tz`
/// # `DELETE /_matrix/client/unstable/uk.tcpip.msc4133/profile/{user_id}/us.cloke.msc4175.tz`
///
/// Deletes the `tz` (timezone) of a user, as per MSC4133 and MSC4175.
///
@@ -88,7 +88,7 @@ pub(crate) async fn delete_timezone_key_route(
Ok(delete_timezone_key::unstable::Response {})
}
/// # `PUT /_matrix/client/unstable/uk.tcpip.msc4133/profile/:user_id/us.cloke.msc4175.tz`
/// # `PUT /_matrix/client/unstable/uk.tcpip.msc4133/profile/{user_id}/us.cloke.msc4175.tz`
///
/// Updates the `tz` (timezone) of a user, as per MSC4133 and MSC4175.
///
@@ -270,7 +270,7 @@ pub(crate) async fn delete_profile_key_route(
Ok(delete_profile_key::unstable::Response {})
}
/// # `GET /_matrix/client/unstable/uk.tcpip.msc4133/profile/:user_id/us.cloke.msc4175.tz`
/// # `GET /_matrix/client/unstable/uk.tcpip.msc4133/profile/{user_id}/us.cloke.msc4175.tz`
///
/// Returns the `timezone` of the user as per MSC4133 and MSC4175.
///
+16 -16
View File
@@ -124,23 +124,23 @@ pub fn build(router: Router<State>, server: &Server) -> Router<State> {
// Ruma doesn't have support for multiple paths for a single endpoint yet, and these routes
// share one Ruma request / response type pair with {get,send}_state_event_for_key_route
.route(
"/_matrix/client/r0/rooms/:room_id/state/:event_type",
"/_matrix/client/r0/rooms/{room_id}/state/{event_type}",
get(client::get_state_events_for_empty_key_route)
.put(client::send_state_event_for_empty_key_route),
)
.route(
"/_matrix/client/v3/rooms/:room_id/state/:event_type",
"/_matrix/client/v3/rooms/{room_id}/state/{event_type}",
get(client::get_state_events_for_empty_key_route)
.put(client::send_state_event_for_empty_key_route),
)
// These two endpoints allow trailing slashes
.route(
"/_matrix/client/r0/rooms/:room_id/state/:event_type/",
"/_matrix/client/r0/rooms/{room_id}/state/{event_type}/",
get(client::get_state_events_for_empty_key_route)
.put(client::send_state_event_for_empty_key_route),
)
.route(
"/_matrix/client/v3/rooms/:room_id/state/:event_type/",
"/_matrix/client/v3/rooms/{room_id}/state/{event_type}/",
get(client::get_state_events_for_empty_key_route)
.put(client::send_state_event_for_empty_key_route),
)
@@ -179,7 +179,7 @@ pub fn build(router: Router<State>, server: &Server) -> Router<State> {
.ruma_route(&client::get_mutual_rooms_route)
.ruma_route(&client::get_room_summary)
.route(
"/_matrix/client/unstable/im.nheko.summary/rooms/:room_id_or_alias/summary",
"/_matrix/client/unstable/im.nheko.summary/rooms/{room_id_or_alias}/summary",
get(client::get_room_summary_legacy)
)
.ruma_route(&client::well_known_support)
@@ -193,7 +193,7 @@ pub fn build(router: Router<State>, server: &Server) -> Router<State> {
.ruma_route(&server::get_server_version_route)
.route("/_matrix/key/v2/server", get(server::get_server_keys_route))
.route(
"/_matrix/key/v2/server/:key_id",
"/_matrix/key/v2/server/{key_id}",
get(server::get_server_keys_deprecated_route),
)
.ruma_route(&server::get_public_rooms_route)
@@ -227,9 +227,9 @@ pub fn build(router: Router<State>, server: &Server) -> Router<State> {
.route("/_tuwunel/local_user_count", get(client::tuwunel_local_user_count));
} else {
router = router
.route("/_matrix/federation/*path", any(federation_disabled))
.route("/_matrix/federation/{*path}", any(federation_disabled))
.route("/.well-known/matrix/server", any(federation_disabled))
.route("/_matrix/key/*path", any(federation_disabled))
.route("/_matrix/key/{*path}", any(federation_disabled))
.route("/_tuwunel/local_user_count", any(federation_disabled));
}
@@ -247,27 +247,27 @@ pub fn build(router: Router<State>, server: &Server) -> Router<State> {
get(client::get_media_preview_legacy_legacy_route),
)
.route(
"/_matrix/media/v1/download/:server_name/:media_id",
"/_matrix/media/v1/download/{server_name}/{media_id}",
get(client::get_content_legacy_legacy_route),
)
.route(
"/_matrix/media/v1/download/:server_name/:media_id/:file_name",
"/_matrix/media/v1/download/{server_name}/{media_id}/{file_name}",
get(client::get_content_as_filename_legacy_legacy_route),
)
.route(
"/_matrix/media/v1/thumbnail/:server_name/:media_id",
"/_matrix/media/v1/thumbnail/{server_name}/{media_id}",
get(client::get_content_thumbnail_legacy_legacy_route),
);
} else {
router = router
.route("/_matrix/media/v1/*path", any(legacy_media_disabled))
.route("/_matrix/media/v1/{*path}", any(legacy_media_disabled))
.route("/_matrix/media/v3/config", any(legacy_media_disabled))
.route("/_matrix/media/v3/download/*path", any(legacy_media_disabled))
.route("/_matrix/media/v3/thumbnail/*path", any(legacy_media_disabled))
.route("/_matrix/media/v3/download/{*path}", any(legacy_media_disabled))
.route("/_matrix/media/v3/thumbnail/{*path}", any(legacy_media_disabled))
.route("/_matrix/media/v3/preview_url", any(redirect_legacy_preview))
.route("/_matrix/media/r0/config", any(legacy_media_disabled))
.route("/_matrix/media/r0/download/*path", any(legacy_media_disabled))
.route("/_matrix/media/r0/thumbnail/*path", any(legacy_media_disabled))
.route("/_matrix/media/r0/download/{*path}", any(legacy_media_disabled))
.route("/_matrix/media/r0/thumbnail/{*path}", any(legacy_media_disabled))
.route("/_matrix/media/r0/preview_url", any(redirect_legacy_preview));
}
-2
View File
@@ -1,6 +1,5 @@
use std::{mem, ops::Deref};
use async_trait::async_trait;
use axum::{body::Body, extract::FromRequest};
use bytes::{BufMut, Bytes, BytesMut};
use ruma::{
@@ -79,7 +78,6 @@ where
fn deref(&self) -> &Self::Target { &self.body }
}
#[async_trait]
impl<T> FromRequest<State, Body> for Args<T>
where
T: IncomingRequest + Send + Sync + 'static,
+21 -21
View File
@@ -48,16 +48,22 @@ pub(super) async fn auth(
json_body: Option<&CanonicalJsonValue>,
metadata: &Metadata,
) -> Result<Auth> {
let bearer: Option<TypedHeader<Authorization<Bearer>>> = request.parts.extract().await?;
let bearer: Option<TypedHeader<Authorization<Bearer>>> =
request.parts.extract().await.unwrap_or(None);
let token = match &bearer {
| Some(TypedHeader(Authorization(bearer))) => Some(bearer.token()),
| None => request.query.access_token.as_deref(),
};
let token = if let Some(token) = token {
match services.appservice.find_from_token(token).await {
match services
.appservice
.find_from_access_token(token)
.await
{
| Some(reg_info) => Token::Appservice(Box::new(reg_info)),
| _ => match services.users.find_from_token(token).await {
| _ => match services.users.find_from_access_token(token).await {
| Ok((user_id, device_id)) => Token::User((user_id, device_id)),
| _ => Token::Invalid,
},
@@ -80,10 +86,9 @@ pub(super) async fn auth(
// already
},
| Token::None | Token::Invalid => {
return Err(Error::BadRequest(
ErrorKind::MissingToken,
"Missing or invalid access token.",
));
return Err!(Request(MissingToken(
"Missing or invalid access token."
)));
},
}
}
@@ -104,10 +109,9 @@ pub(super) async fn auth(
// already
},
| Token::None | Token::Invalid => {
return Err(Error::BadRequest(
ErrorKind::MissingToken,
"Missing or invalid access token.",
));
return Err!(Request(MissingToken(
"Missing or invalid access token."
)));
},
}
}
@@ -138,10 +142,10 @@ pub(super) async fn auth(
appservice_info: None,
})
} else {
Err(Error::BadRequest(ErrorKind::MissingToken, "Missing access token."))
Err!(Request(MissingToken("Missing access token.")))
}
},
| _ => Err(Error::BadRequest(ErrorKind::MissingToken, "Missing access token.")),
| _ => Err!(Request(MissingToken("Missing access token."))),
},
| (
AuthScheme::AccessToken | AuthScheme::AccessTokenOptional | AuthScheme::None,
@@ -164,14 +168,10 @@ pub(super) async fn auth(
appservice_info: None,
}),
| (AuthScheme::ServerSignatures, Token::Appservice(_) | Token::User(_)) =>
Err(Error::BadRequest(
ErrorKind::Unauthorized,
"Only server signatures should be used on this endpoint.",
)),
| (AuthScheme::AppserviceToken, Token::User(_)) => Err(Error::BadRequest(
ErrorKind::Unauthorized,
"Only appservice access tokens should be used on this endpoint.",
)),
Err!(Request(Unauthorized("Only server signatures should be used on this endpoint."))),
| (AuthScheme::AppserviceToken, Token::User(_)) => Err!(Request(Unauthorized(
"Only appservice access tokens should be used on this endpoint."
))),
| (AuthScheme::None, Token::Invalid) => {
// OpenID federation endpoint uses a query param with the same name, drop this
// once query params for user auth are removed from the spec. This is
+2
View File
@@ -13,6 +13,8 @@ pub(crate) async fn get_server_version_route(
server: Some(get_server_version::v1::Server {
name: Some(tuwunel_core::version::name().into()),
version: Some(tuwunel_core::version::version().into()),
compiler: tuwunel_core::info::rustc::version().map(Into::into),
..Default::default()
}),
})
}
+1
View File
@@ -78,6 +78,7 @@ http-body-util.workspace = true
http.workspace = true
ipaddress.workspace = true
itertools.workspace = true
jsonwebtoken.workspace = true
ldap3.workspace = true
libc.workspace = true
libloading.workspace = true
+9 -2
View File
@@ -33,9 +33,16 @@ pub static malloc_conf: &[u8] = const_str::concat_bytes!(
0
);
#[cfg(all(feature = "jemalloc_conf", feature = "jemalloc_prof"))]
#[cfg(all(
feature = "jemalloc_conf",
feature = "jemalloc_prof",
target_arch = "x86_64",
))]
const MALLOC_CONF_PROF: &str = ",prof_active:false";
#[cfg(all(feature = "jemalloc_conf", not(feature = "jemalloc_prof")))]
#[cfg(all(
feature = "jemalloc_conf",
any(not(feature = "jemalloc_prof"), not(target_arch = "x86_64")),
))]
const MALLOC_CONF_PROF: &str = "";
#[global_allocator]
+110 -3
View File
@@ -52,7 +52,7 @@ use crate::{Result, err, error::Error, utils::sys};
### For more information, see:
### https://tuwunel.chat/configuration.html
"#,
ignore = "catchall well_known tls blurhashing allow_invalid_tls_certificates ldap"
ignore = "catchall well_known tls blurhashing allow_invalid_tls_certificates ldap jwt"
)]
pub struct Config {
/// The server_name is the pretty name of this server. It is used as a
@@ -1623,7 +1623,7 @@ pub struct Config {
/// Sentry.io crash/panic reporting, performance monitoring/metrics, etc.
/// This is NOT enabled by default. tuwunel's default Sentry reporting
/// endpoint domain is `o4506996327251968.ingest.us.sentry.io`.
/// endpoint domain is `o4509498990067712.ingest.us.sentry.io`.
#[serde(default)]
pub sentry: bool,
@@ -1814,6 +1814,10 @@ pub struct Config {
#[serde(default)]
pub ldap: LdapConfig,
// external structure; separate section
#[serde(default)]
pub jwt: JwtConfig,
#[serde(flatten)]
#[allow(clippy::zero_sized_map_values)]
// this is a catchall, the map shouldn't be zero at runtime
@@ -1991,6 +1995,99 @@ pub struct LdapConfig {
pub admin_filter: String,
}
#[derive(Clone, Debug, Default, Deserialize)]
#[config_example_generator(filename = "tuwunel-example.toml", section = "global.jwt")]
pub struct JwtConfig {
/// Enable JWT logins
///
/// default: false
#[serde(default)]
pub enable: bool,
/// Validation key, also called 'secret' in Synapse config. The type of key
/// can be configured in 'format', but defaults to the common HMAC which
/// is a plaintext shared-secret, so you should keep this value private.
///
/// display: sensitive
/// default:
#[serde(default, alias = "secret")]
pub key: String,
/// Format of the 'key'. Only HMAC, ECDSA, and B64HMAC are supported
/// Binary keys cannot be pasted into this config, so B64HMAC is an
/// alternative to HMAC for properly random secret strings.
/// - HMAC is a plaintext shared-secret private-key.
/// - B64HMAC is a base64-encoded version of HMAC.
/// - ECDSA is a PEM-encoded public-key.
///
/// default: "HMAC"
#[serde(default = "default_jwt_format")]
pub format: String,
/// Automatically create new user from a valid claim, otherwise access is
/// denied for an unknown even with an authentic token.
///
/// default: true
#[serde(default = "true_fn")]
pub register_user: bool,
/// JWT algorithm
///
/// default: "HS256"
#[serde(default = "default_jwt_algorithm")]
pub algorithm: String,
/// Optional audience claim list. The token must claim one or more values
/// from this list when set.
///
/// default: []
#[serde(default)]
pub audience: Vec<String>,
/// Optional issuer claim list. The token must claim one or more values
/// from this list when set.
///
/// default: []
#[serde(default)]
pub issuer: Vec<String>,
/// Require expiration claim in the token. This defaults to false for
/// synapse migration compatibility.
///
/// default: false
#[serde(default)]
pub require_exp: bool,
/// Require not-before claim in the token. This defaults to false for
/// synapse migration compatibility.
///
/// default: false
#[serde(default)]
pub require_nbf: bool,
/// Validate expiration time of the token when present. Whether or not it is
/// required depends on require_exp, but when present this ensures the token
/// is not used after a time.
///
/// default: true
#[serde(default = "true_fn")]
pub validate_exp: bool,
/// Validate not-before time of the token when present. Whether or not it is
/// required depends on require_nbf, but when present this ensures the token
/// is not used before a time.
///
/// default: true
#[serde(default = "true_fn")]
pub validate_nbf: bool,
/// Bypass validation for diagnostic/debug use only.
///
/// default: true
#[serde(default = "true_fn")]
pub validate_signature: bool,
}
#[derive(Deserialize, Clone, Debug)]
#[serde(transparent)]
struct ListeningPort {
@@ -2312,7 +2409,13 @@ fn default_url_preview_max_spider_size() -> usize {
fn default_new_user_displayname_suffix() -> String { "🎔".to_owned() }
fn default_sentry_endpoint() -> Option<Url> { None }
fn default_sentry_endpoint() -> Option<Url> {
let url = "https://8994b1762a6a95af9502a7900edabc4c@o4509498990067712.ingest.us.sentry.io/4509498993213440"
.try_into()
.expect("default sentry url is invalid");
Some(url)
}
fn default_sentry_traces_sample_rate() -> f32 { 0.15 }
@@ -2386,3 +2489,7 @@ fn default_ldap_uid_attribute() -> String { String::from("uid") }
fn default_ldap_mail_attribute() -> String { String::from("mail") }
fn default_ldap_name_attribute() -> String { String::from("givenName") }
fn default_jwt_algorithm() -> String { "HS256".to_owned() }
fn default_jwt_format() -> String { "HMAC".to_owned() }
+2 -1
View File
@@ -100,7 +100,8 @@ pub fn trap() {
#[must_use]
pub fn panic_str(p: &Box<dyn Any + Send>) -> &'static str {
p.downcast_ref::<&str>()
(**p)
.downcast_ref::<&str>()
.copied()
.unwrap_or_default()
}
+13
View File
@@ -10,6 +10,9 @@ use std::{
use crate::utils::exchange;
// Capture rustc version during compilation.
tuwunel_macros::rustc_version! {}
/// Raw capture of rustc flags used to build each crate in the project. Informed
/// by rustc_flags_capture macro (one in each crate's mod.rs). This is
/// done during static initialization which is why it's mutex-protected and pub.
@@ -23,6 +26,16 @@ static FEATURES: OnceLock<Vec<&'static str>> = OnceLock::new();
/// List of features enabled for the project.
pub fn features() -> &'static Vec<&'static str> { FEATURES.get_or_init(init_features) }
/// Version of the rustc compiler used during build.
#[inline]
#[must_use]
pub fn version() -> Option<&'static str> {
RUSTC_VERSION
.len()
.gt(&0)
.then_some(RUSTC_VERSION)
}
fn init_features() -> Vec<&'static str> {
let mut features = Vec::new();
FLAGS
+28 -7
View File
@@ -9,6 +9,8 @@ use std::sync::OnceLock;
static BRANDING: &str = "Tuwunel";
static SEMANTIC: &str = env!("CARGO_PKG_VERSION");
tuwunel_macros::git_commit! {}
tuwunel_macros::git_semantic! {}
static VERSION: OnceLock<String> = OnceLock::new();
static USER_AGENT: OnceLock<String> = OnceLock::new();
@@ -23,17 +25,36 @@ pub fn version() -> &'static str { VERSION.get_or_init(init_version) }
#[inline]
pub fn user_agent() -> &'static str { USER_AGENT.get_or_init(init_user_agent) }
fn init_user_agent() -> String { format!("{}/{}", name(), version()) }
fn init_user_agent() -> String { format!("{}/{}", name(), semantic()) }
fn init_version() -> String {
option_env!("TUWUNEL_VERSION_EXTRA")
.or(option_env!("CONDUWUIT_VERSION_EXTRA"))
.or(option_env!("CONDUIT_VERSION_EXTRA"))
.map_or(SEMANTIC.to_owned(), |extra| {
if extra.is_empty() {
SEMANTIC.to_owned()
} else {
format!("{SEMANTIC} ({extra})")
}
.map_or_else(detailed, |extra| {
extra
.is_empty()
.then(detailed)
.unwrap_or_else(|| format!("{} ({extra})", detailed()))
})
}
fn detailed() -> String {
let tag_dirty = semantic()
.rsplit_once('-')
.is_some_and(|(_, s)| !s.is_empty());
if !GIT_COMMIT.is_empty() && tag_dirty {
format!("{} ({})", semantic(), GIT_COMMIT)
} else {
semantic().to_owned()
}
}
fn semantic() -> &'static str {
if !GIT_SEMANTIC.is_empty() {
GIT_SEMANTIC
} else {
SEMANTIC
}
}
+1
View File
@@ -14,6 +14,7 @@ pub mod utils;
pub use ::arrayvec;
pub use ::http;
pub use ::jsonwebtoken as jwt;
pub use ::ruma;
pub use ::smallstr;
pub use ::smallvec;
+9 -2
View File
@@ -7,11 +7,18 @@ use crate::{Result, err};
#[inline]
#[must_use]
#[allow(clippy::as_conversions, clippy::cast_possible_truncation)]
pub fn now_millis() -> u64 {
pub fn now_millis() -> u64 { now().as_millis() as u64 }
#[inline]
#[must_use]
pub fn now_secs() -> u64 { now().as_secs() }
#[inline]
#[must_use]
pub fn now() -> Duration {
UNIX_EPOCH
.elapsed()
.expect("positive duration after epoch")
.as_millis() as u64
}
#[inline]
+1
View File
@@ -48,6 +48,7 @@ zstd_compression = [
[dependencies]
async-channel.workspace = true
const-str.workspace = true
ctor.workspace = true
futures.workspace = true
log.workspace = true
minicbor.workspace = true
+2 -2
View File
@@ -17,8 +17,8 @@ pub type Buffer<const CAP: usize = DEF_STACK_CAP> = SmallVec<[Byte; CAP]>;
pub type Slice = [Byte];
pub type Byte = u8;
pub const KEY_STACK_CAP: usize = 128;
pub const VAL_STACK_CAP: usize = 512;
pub const KEY_STACK_CAP: usize = 128 - 16;
pub const VAL_STACK_CAP: usize = 512 - 16;
pub const DEF_STACK_CAP: usize = KEY_STACK_CAP;
#[inline]
+64
View File
@@ -0,0 +1,64 @@
use std::{process::Command, str};
use proc_macro::TokenStream;
use quote::quote;
pub(super) fn semantic(_args: TokenStream) -> TokenStream {
static ARGS: &[&str] = &["describe", "--tags", "--abbrev=1"];
let output = git(ARGS);
let output = output
.strip_prefix('v')
.map(str::to_string)
.unwrap_or(output);
let output = output
.rsplit_once('-')
.map(|(s, _)| s)
.map(str::to_string)
.unwrap_or(output);
let ret = quote! {
static GIT_SEMANTIC: &'static str = #output;
};
ret.into()
}
pub(super) fn commit(_args: TokenStream) -> TokenStream {
static ARGS: &[&str] = &["describe", "--always", "--dirty", "--abbrev=10"];
let output = git(ARGS);
let ret = quote! {
static GIT_COMMIT: &'static str = #output;
};
ret.into()
}
pub(super) fn describe(_args: TokenStream) -> TokenStream {
static ARGS: &[&str] =
&["describe", "--dirty", "--tags", "--always", "--broken", "--abbrev=10"];
let output = git(ARGS);
let ret = quote! {
static GIT_DESCRIBE: &'static str = #output;
};
ret.into()
}
fn git(args: &[&str]) -> String {
Command::new("git")
.args(args)
.output()
.map(|output| {
str::from_utf8(&output.stdout)
.map(str::trim)
.map(String::from)
.ok()
})
.ok()
.flatten()
.unwrap_or_default()
}
+13
View File
@@ -4,6 +4,7 @@ mod admin;
mod cargo;
mod config;
mod debug;
mod git;
mod implement;
mod refutable;
mod rustc;
@@ -41,6 +42,9 @@ pub fn recursion_depth(args: TokenStream, input: TokenStream) -> TokenStream {
#[proc_macro]
pub fn rustc_flags_capture(args: TokenStream) -> TokenStream { rustc::flags_capture(args) }
#[proc_macro]
pub fn rustc_version(args: TokenStream) -> TokenStream { rustc::version(args) }
#[proc_macro_attribute]
pub fn refutable(args: TokenStream, input: TokenStream) -> TokenStream {
attribute_macro::<ItemFn, _>(args, input, refutable::refutable)
@@ -56,6 +60,15 @@ pub fn config_example_generator(args: TokenStream, input: TokenStream) -> TokenS
attribute_macro::<ItemStruct, _>(args, input, config::example_generator)
}
#[proc_macro]
pub fn git_semantic(args: TokenStream) -> TokenStream { git::semantic(args) }
#[proc_macro]
pub fn git_commit(args: TokenStream) -> TokenStream { git::commit(args) }
#[proc_macro]
pub fn git_describe(args: TokenStream) -> TokenStream { git::describe(args) }
fn attribute_macro<I, F>(args: TokenStream, input: TokenStream, func: F) -> TokenStream
where
F: Fn(I, &[Meta]) -> Result<TokenStream>,
+35 -4
View File
@@ -1,11 +1,13 @@
use std::{process::Command, str};
use proc_macro::TokenStream;
use quote::quote;
use crate::utils::get_crate_name;
pub(super) fn flags_capture(args: TokenStream) -> TokenStream {
let cargo_crate_name = std::env::var("CARGO_CRATE_NAME");
let crate_name = match cargo_crate_name.as_ref() {
| Err(_) => return args,
| Ok(crate_name) => crate_name.trim_start_matches("tuwunel_"),
let Some(crate_name) = get_crate_name() else {
return args;
};
let flag = std::env::args().collect::<Vec<_>>();
@@ -27,3 +29,32 @@ pub(super) fn flags_capture(args: TokenStream) -> TokenStream {
ret.into()
}
pub(super) fn version(args: TokenStream) -> TokenStream {
let Some(_) = get_crate_name() else {
return args;
};
let rustc_path = std::env::args()
.next()
.expect("Path to rustc executable");
let version = Command::new(rustc_path)
.args(["-V"])
.output()
.map(|output| {
str::from_utf8(&output.stdout)
.map(str::trim)
.map(String::from)
.ok()
})
.ok()
.flatten()
.unwrap_or_default();
let ret = quote! {
static RUSTC_VERSION: &'static str = #version;
};
ret.into()
}
+12
View File
@@ -4,6 +4,18 @@ use syn::{Expr, ExprLit, Generics, Lit, Meta, MetaNameValue, parse_str};
use crate::Result;
pub(crate) fn get_crate_name() -> Option<String> {
let cargo_crate_name = std::env::var("CARGO_CRATE_NAME");
match cargo_crate_name.as_ref() {
| Err(_) => None,
| Ok(crate_name) => Some(
crate_name
.trim_start_matches("tuwunel_")
.to_owned(),
),
}
}
pub(crate) fn get_simple_settings(args: &[Meta]) -> HashMap<String, String> {
args.iter().fold(HashMap::new(), |mut map, arg| {
let Meta::NameValue(MetaNameValue { path, value, .. }) = arg else {
+13 -6
View File
@@ -39,12 +39,18 @@ name = "tuwunel"
summary = """\
High performance Matrix homeserver written in Rust"""
require-sh = false
post_install_script = "rpm/postinst"
assets = [
{ source = "target/release/tuwunel", dest = "/usr/sbin/tuwunel", mode = "755" },
{ source = "README.md", dest = "/usr/share/doc/tuwunel/", mode = "644", doc = true },
{ source = "tuwunel-example.toml", dest = "/etc/tuwunel/tuwunel.toml", mode = "640", config = "noreplace" },
{ source = "rpm/tuwunel.service", dest = "/etc/systemd/system/", mode = "644" },
]
[package.metadata.arch]
name = "tuwunel"
pkgdesc = "High performance Matrix homeserver written in Rust"
[features]
default = [
"brotli_compression",
@@ -116,7 +122,7 @@ perf_measurements = [
"dep:tracing-flame",
"dep:tracing-opentelemetry",
"dep:opentelemetry_sdk",
"dep:opentelemetry-jaeger",
#"dep:opentelemetry-jaeger",
"tuwunel-core/perf_measurements",
"tuwunel-core/sentry_telemetry",
]
@@ -146,7 +152,7 @@ systemd = [
]
# enable the tokio_console server ncompatible with release_max_log_level
tokio_console = [
"dep:console-subscriber",
#"dep:console-subscriber",
"tokio/tracing",
]
url_preview = [
@@ -173,12 +179,13 @@ tuwunel-router.workspace = true
tuwunel-service.workspace = true
clap.workspace = true
console-subscriber.optional = true
console-subscriber.workspace = true
#console-subscriber.optional = true
#console-subscriber.workspace = true
const-str.workspace = true
ctor.workspace = true
log.workspace = true
opentelemetry-jaeger.optional = true
opentelemetry-jaeger.workspace = true
#opentelemetry-jaeger.optional = true
#opentelemetry-jaeger.workspace = true
opentelemetry.optional = true
opentelemetry.workspace = true
opentelemetry_sdk.optional = true
+7 -2
View File
@@ -77,9 +77,11 @@ pub(crate) fn init(
(None, None)
};
#[cfg(tuwunel_disable)]
let jaeger_filter = EnvFilter::try_new(&config.jaeger_filter)
.map_err(|e| err!(Config("jaeger_filter", "{e}.")))?;
#[cfg(tuwunel_disable)]
let jaeger_layer = config.allow_jaeger.then(|| {
opentelemetry::global::set_text_map_propagator(
opentelemetry_jaeger::Propagator::new(),
@@ -100,7 +102,10 @@ pub(crate) fn init(
Some(telemetry.with_filter(jaeger_reload_filter))
});
#[cfg(tuwunel_disable)]
let subscriber = subscriber.with(flame_layer).with(jaeger_layer);
let subscriber = subscriber.with(flame_layer);
(subscriber, flame_guard)
};
@@ -117,7 +122,7 @@ pub(crate) fn init(
// compile-time and runtime conditions to elide it, each of those changing the
// subscriber's type.
let (console_enabled, console_disabled_reason) = tokio_console_enabled(config);
#[cfg(all(feature = "tokio_console", tokio_unstable))]
#[cfg(all(feature = "tokio_console", tokio_unstable, tuwunel_disable))]
if console_enabled {
let console_layer = console_subscriber::ConsoleLayer::builder()
.with_default_env()
@@ -139,7 +144,7 @@ pub(crate) fn init(
}
fn tokio_console_enabled(config: &Config) -> (bool, &'static str) {
if !cfg!(all(feature = "tokio_console", tokio_unstable)) {
if !cfg!(all(feature = "tokio_console", tokio_unstable, tuwunel_disable)) {
return (false, "");
}
+1
View File
@@ -100,6 +100,7 @@ axum-server.workspace = true
axum.workspace = true
bytes.workspace = true
const-str.workspace = true
ctor.workspace = true
futures.workspace = true
http.workspace = true
http-body-util.workspace = true
+1 -1
View File
@@ -30,7 +30,7 @@ use tuwunel_core::{
type MakeService = IntoMakeServiceWithConnectInfo<Router, net::SocketAddr>;
const NULL_ADDR: net::SocketAddr = net::SocketAddr::new(IpAddr::V4(Ipv4Addr::new(0, 0, 0, 0)), 0);
const NULL_ADDR: net::SocketAddr = net::SocketAddr::new(IpAddr::V4(Ipv4Addr::UNSPECIFIED), 0);
const FINI_POLL_INTERVAL: Duration = Duration::from_millis(750);
#[tracing::instrument(skip_all, level = "debug")]
+1
View File
@@ -82,6 +82,7 @@ async-trait.workspace = true
base64.workspace = true
bytes.workspace = true
const-str.workspace = true
ctor.workspace = true
either.workspace = true
futures.workspace = true
hickory-resolver.workspace = true
+1 -1
View File
@@ -115,7 +115,7 @@ impl Service {
.map(|info| info.registration)
}
pub async fn find_from_token(&self, token: &str) -> Option<RegistrationInfo> {
pub async fn find_from_access_token(&self, token: &str) -> Option<RegistrationInfo> {
self.read()
.await
.values()
+1 -1
View File
@@ -75,7 +75,7 @@ pub enum SendingEvent {
pub type EduBuf = SmallVec<[u8; EDU_BUF_CAP]>;
pub type EduVec = SmallVec<[EduBuf; EDU_VEC_CAP]>;
const EDU_BUF_CAP: usize = 128;
const EDU_BUF_CAP: usize = 128 - 16;
const EDU_VEC_CAP: usize = 1;
#[async_trait]
+30 -58
View File
@@ -6,12 +6,12 @@ use std::{
use ruma::{
CanonicalJsonValue, DeviceId, OwnedDeviceId, OwnedUserId, UserId,
api::client::{
error::ErrorKind,
error::{ErrorKind, StandardErrorBody},
uiaa::{AuthData, AuthType, Password, UiaaInfo, UserIdentifier},
},
};
use tuwunel_core::{
Err, Error, Result, err, error, implement, utils,
Err, Result, err, error, implement, utils,
utils::{hash, string::EMPTY},
};
use tuwunel_database::{Deserialized, Json, Map};
@@ -67,14 +67,15 @@ pub async fn read_tokens(&self) -> Result<HashSet<String>> {
.as_ref()
{
match std::fs::read_to_string(file) {
| Err(e) => error!("Failed to read the registration token file: {e}"),
| Ok(text) => {
text.split_ascii_whitespace().for_each(|token| {
tokens.insert(token.to_owned());
});
},
| Err(e) => error!("Failed to read the registration token file: {e}"),
}
}
if let Some(token) = &self.services.config.registration_token {
tokens.insert(token.to_owned());
}
@@ -93,25 +94,14 @@ pub fn create(
) {
// TODO: better session error handling (why is uiaainfo.session optional in
// ruma?)
self.set_uiaa_request(
user_id,
device_id,
uiaainfo
.session
.as_ref()
.expect("session should be set"),
json_body,
);
let session = uiaainfo
.session
.as_ref()
.expect("session should be set");
self.update_uiaa_session(
user_id,
device_id,
uiaainfo
.session
.as_ref()
.expect("session should be set"),
Some(uiaainfo),
);
self.set_uiaa_request(user_id, device_id, session, json_body);
self.update_uiaa_session(user_id, device_id, session, Some(uiaainfo));
}
#[implement(Service)]
@@ -148,40 +138,35 @@ pub async fn try_auth(
} else if let Some(username) = user {
username
} else {
return Err(Error::BadRequest(
ErrorKind::Unrecognized,
"Identifier type not recognized.",
));
return Err!(Request(Unrecognized("Identifier type not recognized.")));
};
#[cfg(not(feature = "element_hacks"))]
let Some(UserIdentifier::UserIdOrLocalpart(username)) = identifier else {
return Err(Error::BadRequest(
ErrorKind::Unrecognized,
"Identifier type not recognized.",
));
return Err!(Request(Unrecognized("Identifier type not recognized.")));
};
let user_id_from_username = UserId::parse_with_server_name(
username.clone(),
self.services.globals.server_name(),
)
.map_err(|_| Error::BadRequest(ErrorKind::InvalidParam, "User ID is invalid."))?;
.map_err(|_| err!(Request(InvalidParam("User ID is invalid."))))?;
// Check if the access token being used matches the credentials used for UIAA
if user_id.localpart() != user_id_from_username.localpart() {
return Err!(Request(Forbidden("User ID and access token mismatch.")));
}
let user_id = user_id_from_username;
// Check if password is correct
let user_id = user_id_from_username;
if let Ok(hash) = self.services.users.password_hash(&user_id).await {
let hash_matches = hash::verify_password(password, &hash).is_ok();
if !hash_matches {
uiaainfo.auth_error = Some(ruma::api::client::error::StandardErrorBody {
uiaainfo.auth_error = Some(StandardErrorBody {
kind: ErrorKind::forbidden(),
message: "Invalid username or password.".to_owned(),
});
return Ok((false, uiaainfo));
}
}
@@ -196,10 +181,11 @@ pub async fn try_auth(
.completed
.push(AuthType::RegistrationToken);
} else {
uiaainfo.auth_error = Some(ruma::api::client::error::StandardErrorBody {
uiaainfo.auth_error = Some(StandardErrorBody {
kind: ErrorKind::forbidden(),
message: "Invalid registration token.".to_owned(),
});
return Ok((false, uiaainfo));
}
},
@@ -221,30 +207,19 @@ pub async fn try_auth(
completed = true;
}
let session = uiaainfo
.session
.as_ref()
.expect("session is always set");
if !completed {
self.update_uiaa_session(
user_id,
device_id,
uiaainfo
.session
.as_ref()
.expect("session is always set"),
Some(&uiaainfo),
);
self.update_uiaa_session(user_id, device_id, session, Some(&uiaainfo));
return Ok((false, uiaainfo));
}
// UIAA was successful! Remove this session and return true
self.update_uiaa_session(
user_id,
device_id,
uiaainfo
.session
.as_ref()
.expect("session is always set"),
None,
);
self.update_uiaa_session(user_id, device_id, session, None);
Ok((true, uiaainfo))
}
@@ -258,6 +233,7 @@ fn set_uiaa_request(
request: &CanonicalJsonValue,
) {
let key = (user_id.to_owned(), device_id.to_owned(), session.to_owned());
self.userdevicesessionid_uiaarequest
.write()
.expect("locked for writing")
@@ -271,13 +247,8 @@ pub fn get_uiaa_request(
device_id: Option<&DeviceId>,
session: &str,
) -> Option<CanonicalJsonValue> {
let key = (
user_id.to_owned(),
device_id
.unwrap_or_else(|| EMPTY.into())
.to_owned(),
session.to_owned(),
);
let device_id = device_id.unwrap_or_else(|| EMPTY.into());
let key = (user_id.to_owned(), device_id.to_owned(), session.to_owned());
self.userdevicesessionid_uiaarequest
.read()
@@ -313,6 +284,7 @@ async fn get_uiaa_session(
session: &str,
) -> Result<UiaaInfo> {
let key = (user_id, device_id, session);
self.db
.userdevicesessionid_uiaainfo
.qry(&key)
+276
View File
@@ -0,0 +1,276 @@
use std::sync::Arc;
use futures::{Stream, StreamExt};
use ruma::{
DeviceId, MilliSecondsSinceUnixEpoch, OwnedDeviceId, OwnedUserId, UserId,
api::client::device::Device, events::AnyToDeviceEvent, serde::Raw,
};
use serde_json::json;
use tuwunel_core::{
Err, Result, at, implement,
utils::{self, ReadyExt, stream::TryIgnore},
};
use tuwunel_database::{Deserialized, Ignore, Interfix, Json, Map};
/// Adds a new device to a user.
#[implement(super::Service)]
pub async fn create_device(
&self,
user_id: &UserId,
device_id: &DeviceId,
token: &str,
initial_device_display_name: Option<String>,
client_ip: Option<String>,
) -> Result {
if !self.exists(user_id).await {
return Err!(Request(InvalidParam(error!(
"Called create_device for non-existent user {user_id}"
))));
}
let key = (user_id, device_id);
let val = Device {
device_id: device_id.into(),
display_name: initial_device_display_name,
last_seen_ip: client_ip,
last_seen_ts: Some(MilliSecondsSinceUnixEpoch::now()),
};
increment(&self.db.userid_devicelistversion, user_id.as_bytes());
self.db.userdeviceid_metadata.put(key, Json(val));
self.set_access_token(user_id, device_id, token)
.await
}
/// Removes a device from a user.
#[implement(super::Service)]
pub async fn remove_device(&self, user_id: &UserId, device_id: &DeviceId) {
let userdeviceid = (user_id, device_id);
// Remove tokens
if let Ok(old_token) = self
.db
.userdeviceid_token
.qry(&userdeviceid)
.await
{
self.db.userdeviceid_token.del(userdeviceid);
self.db.token_userdeviceid.remove(&old_token);
}
// Remove todevice events
let prefix = (user_id, device_id, Interfix);
self.db
.todeviceid_events
.keys_prefix_raw(&prefix)
.ignore_err()
.ready_for_each(|key| self.db.todeviceid_events.remove(key))
.await;
// TODO: Remove onetimekeys
increment(&self.db.userid_devicelistversion, user_id.as_bytes());
self.db.userdeviceid_metadata.del(userdeviceid);
self.mark_device_key_update(user_id).await;
}
/// Returns an iterator over all device ids of this user.
#[implement(super::Service)]
pub fn all_device_ids<'a>(
&'a self,
user_id: &'a UserId,
) -> impl Stream<Item = &DeviceId> + Send + 'a {
let prefix = (user_id, Interfix);
self.db
.userdeviceid_metadata
.keys_prefix(&prefix)
.ignore_err()
.map(|(_, device_id): (Ignore, &DeviceId)| device_id)
}
/// Replaces the access token of one device.
#[implement(super::Service)]
pub async fn set_access_token(
&self,
user_id: &UserId,
device_id: &DeviceId,
token: &str,
) -> Result {
let key = (user_id, device_id);
if self
.db
.userdeviceid_metadata
.qry(&key)
.await
.is_err()
{
return Err!(Database(error!(
?user_id,
?device_id,
"User does not exist or device has no metadata."
)));
}
// Remove old token
if let Ok(old_token) = self.db.userdeviceid_token.qry(&key).await {
self.db.token_userdeviceid.remove(&old_token);
// It will be removed from userdeviceid_token by the insert later
}
// Assign token to user device combination
self.db.userdeviceid_token.put_raw(key, token);
self.db.token_userdeviceid.raw_put(token, key);
Ok(())
}
/// Find out which user an access token belongs to.
#[implement(super::Service)]
pub async fn find_from_access_token(&self, token: &str) -> Result<(OwnedUserId, OwnedDeviceId)> {
self.db
.token_userdeviceid
.get(token)
.await
.deserialized()
}
#[implement(super::Service)]
pub async fn get_access_token(&self, user_id: &UserId, device_id: &DeviceId) -> Result<String> {
let key = (user_id, device_id);
self.db
.userdeviceid_token
.qry(&key)
.await
.deserialized()
}
#[implement(super::Service)]
pub async fn add_to_device_event(
&self,
sender: &UserId,
target_user_id: &UserId,
target_device_id: &DeviceId,
event_type: &str,
content: serde_json::Value,
) {
let count = self.services.globals.next_count().unwrap();
let key = (target_user_id, target_device_id, count);
self.db.todeviceid_events.put(
key,
Json(json!({
"type": event_type,
"sender": sender,
"content": content,
})),
);
}
#[implement(super::Service)]
pub fn get_to_device_events<'a>(
&'a self,
user_id: &'a UserId,
device_id: &'a DeviceId,
since: Option<u64>,
to: Option<u64>,
) -> impl Stream<Item = Raw<AnyToDeviceEvent>> + Send + 'a {
type Key<'a> = (&'a UserId, &'a DeviceId, u64);
let from = (user_id, device_id, since.map_or(0, |since| since.saturating_add(1)));
self.db
.todeviceid_events
.stream_from(&from)
.ignore_err()
.ready_take_while(move |((user_id_, device_id_, count), _): &(Key<'_>, _)| {
user_id == *user_id_ && device_id == *device_id_ && to.is_none_or(|to| *count <= to)
})
.map(at!(1))
}
#[implement(super::Service)]
pub async fn remove_to_device_events<Until>(
&self,
user_id: &UserId,
device_id: &DeviceId,
until: Until,
) where
Until: Into<Option<u64>> + Send,
{
type Key<'a> = (&'a UserId, &'a DeviceId, u64);
let until = until.into().unwrap_or(u64::MAX);
let from = (user_id, device_id, until);
self.db
.todeviceid_events
.rev_keys_from(&from)
.ignore_err()
.ready_take_while(move |(user_id_, device_id_, _): &Key<'_>| {
user_id == *user_id_ && device_id == *device_id_
})
.ready_for_each(|key: Key<'_>| {
self.db.todeviceid_events.del(key);
})
.await;
}
#[implement(super::Service)]
pub async fn update_device_metadata(
&self,
user_id: &UserId,
device_id: &DeviceId,
device: &Device,
) -> Result {
increment(&self.db.userid_devicelistversion, user_id.as_bytes());
let key = (user_id, device_id);
self.db
.userdeviceid_metadata
.put(key, Json(device));
Ok(())
}
/// Get device metadata.
#[implement(super::Service)]
pub async fn get_device_metadata(
&self,
user_id: &UserId,
device_id: &DeviceId,
) -> Result<Device> {
self.db
.userdeviceid_metadata
.qry(&(user_id, device_id))
.await
.deserialized()
}
#[implement(super::Service)]
pub async fn get_devicelist_version(&self, user_id: &UserId) -> Result<u64> {
self.db
.userid_devicelistversion
.get(user_id)
.await
.deserialized()
}
#[implement(super::Service)]
pub fn all_devices_metadata<'a>(
&'a self,
user_id: &'a UserId,
) -> impl Stream<Item = Device> + Send + 'a {
let key = (user_id, Interfix);
self.db
.userdeviceid_metadata
.stream_prefix(&key)
.ignore_err()
.map(|(_, val): (Ignore, Device)| val)
}
//TODO: this is an ABA
fn increment(db: &Arc<Map>, key: &[u8]) {
let old = db.get_blocking(key);
let new = utils::increment(old.ok().as_deref());
db.insert(key, new);
}
+515
View File
@@ -0,0 +1,515 @@
use std::{collections::BTreeMap, mem};
use futures::{Stream, StreamExt, TryFutureExt};
use ruma::{
DeviceId, KeyId, OneTimeKeyAlgorithm, OneTimeKeyId, OneTimeKeyName, OwnedKeyId, RoomId, UInt,
UserId,
api::client::error::ErrorKind,
encryption::{CrossSigningKey, DeviceKeys, OneTimeKey},
serde::Raw,
};
use tuwunel_core::{
Err, Error, Result, err, implement,
utils::{ReadyExt, stream::TryIgnore, string::Unquoted},
};
use tuwunel_database::{Deserialized, Ignore, Json};
#[implement(super::Service)]
pub async fn add_one_time_key(
&self,
user_id: &UserId,
device_id: &DeviceId,
one_time_key_key: &KeyId<OneTimeKeyAlgorithm, OneTimeKeyName>,
one_time_key_value: &Raw<OneTimeKey>,
) -> Result {
// All devices have metadata
// Only existing devices should be able to call this, but we shouldn't assert
// either...
let key = (user_id, device_id);
if self
.db
.userdeviceid_metadata
.qry(&key)
.await
.is_err()
{
return Err!(Database(error!(
?user_id,
?device_id,
"User does not exist or device has no metadata."
)));
}
let mut key = user_id.as_bytes().to_vec();
key.push(0xFF);
key.extend_from_slice(device_id.as_bytes());
key.push(0xFF);
// TODO: Use DeviceKeyId::to_string when it's available (and update everything,
// because there are no wrapping quotation marks anymore)
key.extend_from_slice(
serde_json::to_string(one_time_key_key)
.expect("DeviceKeyId::to_string always works")
.as_bytes(),
);
self.db
.onetimekeyid_onetimekeys
.raw_put(key, Json(one_time_key_value));
let count = self.services.globals.next_count().unwrap();
self.db
.userid_lastonetimekeyupdate
.raw_put(user_id, count);
Ok(())
}
#[implement(super::Service)]
pub async fn last_one_time_keys_update(&self, user_id: &UserId) -> u64 {
self.db
.userid_lastonetimekeyupdate
.get(user_id)
.await
.deserialized()
.unwrap_or(0)
}
#[implement(super::Service)]
pub async fn take_one_time_key(
&self,
user_id: &UserId,
device_id: &DeviceId,
key_algorithm: &OneTimeKeyAlgorithm,
) -> Result<(OwnedKeyId<OneTimeKeyAlgorithm, OneTimeKeyName>, Raw<OneTimeKey>)> {
let count = self.services.globals.next_count()?.to_be_bytes();
self.db
.userid_lastonetimekeyupdate
.insert(user_id, count);
let mut prefix = user_id.as_bytes().to_vec();
prefix.push(0xFF);
prefix.extend_from_slice(device_id.as_bytes());
prefix.push(0xFF);
prefix.push(b'"'); // Annoying quotation mark
prefix.extend_from_slice(key_algorithm.as_ref().as_bytes());
prefix.push(b':');
let one_time_key = self
.db
.onetimekeyid_onetimekeys
.raw_stream_prefix(&prefix)
.ignore_err()
.map(|(key, val)| {
self.db.onetimekeyid_onetimekeys.remove(key);
let key = key
.rsplit(|&b| b == 0xFF)
.next()
.ok_or_else(|| err!(Database("OneTimeKeyId in db is invalid.")))
.unwrap();
let key = serde_json::from_slice(key)
.map_err(|e| err!(Database("OneTimeKeyId in db is invalid. {e}")))
.unwrap();
let val = serde_json::from_slice(val)
.map_err(|e| err!(Database("OneTimeKeys in db are invalid. {e}")))
.unwrap();
(key, val)
})
.next()
.await;
one_time_key.ok_or_else(|| err!(Request(NotFound("No one-time-key found"))))
}
#[implement(super::Service)]
pub async fn count_one_time_keys(
&self,
user_id: &UserId,
device_id: &DeviceId,
) -> BTreeMap<OneTimeKeyAlgorithm, UInt> {
type KeyVal<'a> = ((Ignore, Ignore, &'a Unquoted), Ignore);
let mut algorithm_counts = BTreeMap::<OneTimeKeyAlgorithm, _>::new();
let query = (user_id, device_id);
self.db
.onetimekeyid_onetimekeys
.stream_prefix(&query)
.ignore_err()
.ready_for_each(|((Ignore, Ignore, device_key_id), Ignore): KeyVal<'_>| {
let one_time_key_id: &OneTimeKeyId = device_key_id
.as_str()
.try_into()
.expect("Invalid DeviceKeyID in database");
let count: &mut UInt = algorithm_counts
.entry(one_time_key_id.algorithm())
.or_default();
*count = count.saturating_add(1_u32.into());
})
.await;
algorithm_counts
}
#[implement(super::Service)]
pub async fn add_device_keys(
&self,
user_id: &UserId,
device_id: &DeviceId,
device_keys: &Raw<DeviceKeys>,
) {
let key = (user_id, device_id);
self.db.keyid_key.put(key, Json(device_keys));
self.mark_device_key_update(user_id).await;
}
#[implement(super::Service)]
pub async fn add_cross_signing_keys(
&self,
user_id: &UserId,
master_key: &Option<Raw<CrossSigningKey>>,
self_signing_key: &Option<Raw<CrossSigningKey>>,
user_signing_key: &Option<Raw<CrossSigningKey>>,
notify: bool,
) -> Result {
// TODO: Check signatures
let mut prefix = user_id.as_bytes().to_vec();
prefix.push(0xFF);
if let Some(master_key) = master_key {
let (master_key_key, _) = parse_master_key(user_id, master_key)?;
self.db
.keyid_key
.insert(&master_key_key, master_key.json().get().as_bytes());
self.db
.userid_masterkeyid
.insert(user_id.as_bytes(), &master_key_key);
}
// Self-signing key
if let Some(self_signing_key) = self_signing_key {
let mut self_signing_key_ids = self_signing_key
.deserialize()
.map_err(|e| err!(Request(InvalidParam("Invalid self signing key: {e:?}"))))?
.keys
.into_values();
let self_signing_key_id = self_signing_key_ids
.next()
.ok_or(Error::BadRequest(
ErrorKind::InvalidParam,
"Self signing key contained no key.",
))?;
if self_signing_key_ids.next().is_some() {
return Err(Error::BadRequest(
ErrorKind::InvalidParam,
"Self signing key contained more than one key.",
));
}
let mut self_signing_key_key = prefix.clone();
self_signing_key_key.extend_from_slice(self_signing_key_id.as_bytes());
self.db
.keyid_key
.insert(&self_signing_key_key, self_signing_key.json().get().as_bytes());
self.db
.userid_selfsigningkeyid
.insert(user_id.as_bytes(), &self_signing_key_key);
}
// User-signing key
if let Some(user_signing_key) = user_signing_key {
let user_signing_key_id = parse_user_signing_key(user_signing_key)?;
let user_signing_key_key = (user_id, &user_signing_key_id);
self.db
.keyid_key
.put_raw(user_signing_key_key, user_signing_key.json().get().as_bytes());
self.db
.userid_usersigningkeyid
.raw_put(user_id, user_signing_key_key);
}
if notify {
self.mark_device_key_update(user_id).await;
}
Ok(())
}
#[implement(super::Service)]
pub async fn sign_key(
&self,
target_id: &UserId,
key_id: &str,
signature: (String, String),
sender_id: &UserId,
) -> Result {
let key = (target_id, key_id);
let mut cross_signing_key: serde_json::Value = self
.db
.keyid_key
.qry(&key)
.await
.map_err(|_| err!(Request(InvalidParam("Tried to sign nonexistent key"))))?
.deserialized()
.map_err(|e| err!(Database(debug_warn!("key in keyid_key is invalid: {e:?}"))))?;
let signatures = cross_signing_key
.get_mut("signatures")
.ok_or_else(|| err!(Database(debug_warn!("key in keyid_key has no signatures field"))))?
.as_object_mut()
.ok_or_else(|| {
err!(Database(debug_warn!("key in keyid_key has invalid signatures field.")))
})?
.entry(sender_id.to_string())
.or_insert_with(|| serde_json::Map::new().into());
signatures
.as_object_mut()
.ok_or_else(|| {
err!(Database(debug_warn!("signatures in keyid_key for a user is invalid.")))
})?
.insert(signature.0, signature.1.into());
let key = (target_id, key_id);
self.db
.keyid_key
.put(key, Json(cross_signing_key));
self.mark_device_key_update(target_id).await;
Ok(())
}
#[implement(super::Service)]
#[inline]
pub fn keys_changed<'a>(
&'a self,
user_id: &'a UserId,
from: u64,
to: Option<u64>,
) -> impl Stream<Item = &UserId> + Send + 'a {
self.keys_changed_user_or_room(user_id.as_str(), from, to)
.map(|(user_id, ..)| user_id)
}
#[implement(super::Service)]
#[inline]
pub fn room_keys_changed<'a>(
&'a self,
room_id: &'a RoomId,
from: u64,
to: Option<u64>,
) -> impl Stream<Item = (&UserId, u64)> + Send + 'a {
self.keys_changed_user_or_room(room_id.as_str(), from, to)
}
#[implement(super::Service)]
fn keys_changed_user_or_room<'a>(
&'a self,
user_or_room_id: &'a str,
from: u64,
to: Option<u64>,
) -> impl Stream<Item = (&UserId, u64)> + Send + 'a {
type KeyVal<'a> = ((&'a str, u64), &'a UserId);
let to = to.unwrap_or(u64::MAX);
let start = (user_or_room_id, from.saturating_add(1));
self.db
.keychangeid_userid
.stream_from(&start)
.ignore_err()
.ready_take_while(move |((prefix, count), _): &KeyVal<'_>| {
*prefix == user_or_room_id && *count <= to
})
.map(|((_, count), user_id): KeyVal<'_>| (user_id, count))
}
#[implement(super::Service)]
pub async fn mark_device_key_update(&self, user_id: &UserId) {
let count = self.services.globals.next_count().unwrap();
self.services
.state_cache
.rooms_joined(user_id)
// Don't send key updates to unencrypted rooms
.filter(|room_id| self.services.state_accessor.is_encrypted_room(room_id))
.ready_for_each(|room_id| {
let key = (room_id, count);
self.db.keychangeid_userid.put_raw(key, user_id);
})
.await;
let key = (user_id, count);
self.db.keychangeid_userid.put_raw(key, user_id);
}
#[implement(super::Service)]
pub async fn get_device_keys<'a>(
&'a self,
user_id: &'a UserId,
device_id: &DeviceId,
) -> Result<Raw<DeviceKeys>> {
let key_id = (user_id, device_id);
self.db
.keyid_key
.qry(&key_id)
.await
.deserialized()
}
#[implement(super::Service)]
pub async fn get_key<F>(
&self,
key_id: &[u8],
sender_user: Option<&UserId>,
user_id: &UserId,
allowed_signatures: &F,
) -> Result<Raw<CrossSigningKey>>
where
F: Fn(&UserId) -> bool + Send + Sync,
{
let key: serde_json::Value = self
.db
.keyid_key
.get(key_id)
.await
.deserialized()?;
let cleaned = clean_signatures(key, sender_user, user_id, allowed_signatures)?;
let raw_value = serde_json::value::to_raw_value(&cleaned)?;
Ok(Raw::from_json(raw_value))
}
#[implement(super::Service)]
pub async fn get_master_key<F>(
&self,
sender_user: Option<&UserId>,
user_id: &UserId,
allowed_signatures: &F,
) -> Result<Raw<CrossSigningKey>>
where
F: Fn(&UserId) -> bool + Send + Sync,
{
let key_id = self.db.userid_masterkeyid.get(user_id).await?;
self.get_key(&key_id, sender_user, user_id, allowed_signatures)
.await
}
#[implement(super::Service)]
pub async fn get_self_signing_key<F>(
&self,
sender_user: Option<&UserId>,
user_id: &UserId,
allowed_signatures: &F,
) -> Result<Raw<CrossSigningKey>>
where
F: Fn(&UserId) -> bool + Send + Sync,
{
let key_id = self
.db
.userid_selfsigningkeyid
.get(user_id)
.await?;
self.get_key(&key_id, sender_user, user_id, allowed_signatures)
.await
}
#[implement(super::Service)]
pub async fn get_user_signing_key(&self, user_id: &UserId) -> Result<Raw<CrossSigningKey>> {
self.db
.userid_usersigningkeyid
.get(user_id)
.and_then(|key_id| self.db.keyid_key.get(&*key_id))
.await
.deserialized()
}
pub fn parse_master_key(
user_id: &UserId,
master_key: &Raw<CrossSigningKey>,
) -> Result<(Vec<u8>, CrossSigningKey)> {
let mut prefix = user_id.as_bytes().to_vec();
prefix.push(0xFF);
let master_key = master_key
.deserialize()
.map_err(|_| Error::BadRequest(ErrorKind::InvalidParam, "Invalid master key"))?;
let mut master_key_ids = master_key.keys.values();
let master_key_id = master_key_ids
.next()
.ok_or(Error::BadRequest(ErrorKind::InvalidParam, "Master key contained no key."))?;
if master_key_ids.next().is_some() {
return Err(Error::BadRequest(
ErrorKind::InvalidParam,
"Master key contained more than one key.",
));
}
let mut master_key_key = prefix.clone();
master_key_key.extend_from_slice(master_key_id.as_bytes());
Ok((master_key_key, master_key))
}
pub(super) fn parse_user_signing_key(user_signing_key: &Raw<CrossSigningKey>) -> Result<String> {
let mut user_signing_key_ids = user_signing_key
.deserialize()
.map_err(|_| err!(Request(InvalidParam("Invalid user signing key"))))?
.keys
.into_values();
let user_signing_key_id = user_signing_key_ids
.next()
.ok_or(err!(Request(InvalidParam("User signing key contained no key."))))?;
if user_signing_key_ids.next().is_some() {
return Err!(Request(InvalidParam("User signing key contained more than one key.")));
}
Ok(user_signing_key_id)
}
/// Ensure that a user only sees signatures from themselves and the target user
fn clean_signatures<F>(
mut cross_signing_key: serde_json::Value,
sender_user: Option<&UserId>,
user_id: &UserId,
allowed_signatures: &F,
) -> Result<serde_json::Value>
where
F: Fn(&UserId) -> bool + Send + Sync,
{
if let Some(signatures) = cross_signing_key
.get_mut("signatures")
.and_then(|v| v.as_object_mut())
{
// Don't allocate for the full size of the current signatures, but require
// at most one resize if nothing is dropped
let new_capacity = signatures.len() / 2;
for (user, signature) in
mem::replace(signatures, serde_json::Map::with_capacity(new_capacity))
{
let sid = <&UserId>::try_from(user.as_str())
.map_err(|_| Error::bad_database("Invalid user ID in database."))?;
if sender_user == Some(user_id) || sid == user_id || allowed_signatures(sid) {
signatures.insert(user, signature);
}
}
}
Ok(cross_signing_key)
}
+153
View File
@@ -0,0 +1,153 @@
#![cfg(feature = "ldap")]
use itertools::Itertools;
use ldap3::{LdapConnAsync, Scope, SearchEntry};
use ruma::UserId;
use tuwunel_core::{Result, debug, err, error, implement, result::LogErr, trace};
/// Performs a LDAP search for the given user.
///
/// Returns the list of matching users, with a boolean for each result set
/// to true if the user is an admin.
#[implement(super::Service)]
pub async fn search_ldap(&self, user_id: &UserId) -> Result<Vec<(String, bool)>> {
let localpart = user_id.localpart().to_owned();
let lowercased_localpart = localpart.to_lowercase();
let config = &self.services.server.config.ldap;
let uri = config
.uri
.as_ref()
.ok_or_else(|| err!(Ldap(error!("LDAP URI is not configured."))))?;
debug!(?uri, "LDAP creating connection...");
let (conn, mut ldap) = LdapConnAsync::new(uri.as_str())
.await
.map_err(|e| err!(Ldap(error!(?user_id, "LDAP connection setup error: {e}"))))?;
let driver = self.services.server.runtime().spawn(async move {
match conn.drive().await {
| Err(e) => error!("LDAP connection error: {e}"),
| Ok(()) => debug!("LDAP connection completed."),
}
});
match (&config.bind_dn, &config.bind_password_file) {
| (Some(bind_dn), Some(bind_password_file)) => {
let bind_pw = String::from_utf8(std::fs::read(bind_password_file)?)?;
ldap.simple_bind(bind_dn, bind_pw.trim())
.await
.and_then(ldap3::LdapResult::success)
.map_err(|e| err!(Ldap(error!("LDAP bind error: {e}"))))?;
},
| (..) => {},
}
let attr = [&config.uid_attribute, &config.name_attribute];
let user_filter = &config
.filter
.replace("{username}", &lowercased_localpart);
let (entries, _result) = ldap
.search(&config.base_dn, Scope::Subtree, user_filter, &attr)
.await
.and_then(ldap3::SearchResult::success)
.inspect(|(entries, result)| trace!(?entries, ?result, "LDAP Search"))
.map_err(|e| err!(Ldap(error!(?attr, ?user_filter, "LDAP search error: {e}"))))?;
let mut dns = entries
.into_iter()
.filter_map(|entry| {
let search_entry = SearchEntry::construct(entry);
debug!(?search_entry, "LDAP search entry");
search_entry
.attrs
.get(&config.uid_attribute)
.into_iter()
.chain(search_entry.attrs.get(&config.name_attribute))
.any(|ids| ids.contains(&localpart) || ids.contains(&lowercased_localpart))
.then_some((search_entry.dn, false))
})
.collect_vec();
if !config.admin_base_dn.is_empty() {
let admin_base_dn = if config.admin_base_dn.is_empty() {
&config.base_dn
} else {
&config.admin_base_dn
};
let admin_filter = &config
.admin_filter
.replace("{username}", &lowercased_localpart);
let (admin_entries, _result) = ldap
.search(admin_base_dn, Scope::Subtree, admin_filter, &attr)
.await
.and_then(ldap3::SearchResult::success)
.inspect(|(entries, result)| trace!(?entries, ?result, "LDAP Admin Search"))
.map_err(|e| {
err!(Ldap(error!(?attr, ?user_filter, "Ldap admin search error: {e}")))
})?;
let mut admin_dns = admin_entries
.into_iter()
.filter_map(|entry| {
let search_entry = SearchEntry::construct(entry);
debug!(?search_entry, "LDAP search entry");
search_entry
.attrs
.get(&config.uid_attribute)
.into_iter()
.chain(search_entry.attrs.get(&config.name_attribute))
.any(|ids| ids.contains(&localpart) || ids.contains(&lowercased_localpart))
.then_some((search_entry.dn, true))
})
.collect_vec();
dns.append(&mut admin_dns);
}
ldap.unbind()
.await
.map_err(|e| err!(Ldap(error!("LDAP unbind error: {e}"))))?;
driver.await.log_err().ok();
Ok(dns)
}
#[implement(super::Service)]
pub async fn auth_ldap(&self, user_dn: &str, password: &str) -> Result {
let config = &self.services.server.config.ldap;
let uri = config
.uri
.as_ref()
.ok_or_else(|| err!(Ldap(error!("LDAP URI is not configured."))))?;
debug!(?uri, "LDAP creating connection...");
let (conn, mut ldap) = LdapConnAsync::new(uri.as_str())
.await
.map_err(|e| err!(Ldap(error!(?user_dn, "LDAP connection setup error: {e}"))))?;
let driver = self.services.server.runtime().spawn(async move {
match conn.drive().await {
| Err(e) => error!("LDAP connection error: {e}"),
| Ok(()) => debug!("LDAP connection completed."),
}
});
ldap.simple_bind(user_dn, password)
.await
.and_then(ldap3::LdapResult::success)
.map_err(|e| err!(Request(Forbidden(debug_error!("LDAP authentication error: {e}")))))?;
ldap.unbind()
.await
.map_err(|e| err!(Ldap(error!("LDAP unbind error: {e}"))))?;
driver.await.log_err().ok();
Ok(())
}
+18 -991
View File
File diff suppressed because it is too large. Load diff
+87
View File
@@ -0,0 +1,87 @@
use futures::{Stream, StreamExt, TryFutureExt};
use ruma::UserId;
use tuwunel_core::{Result, implement, utils::stream::TryIgnore};
use tuwunel_database::{Deserialized, Ignore, Interfix, Json};
/// Gets a specific user profile key
#[implement(super::Service)]
pub async fn profile_key(
&self,
user_id: &UserId,
profile_key: &str,
) -> Result<serde_json::Value> {
let key = (user_id, profile_key);
self.db
.useridprofilekey_value
.qry(&key)
.await
.deserialized()
}
/// Gets all the user's profile keys and values in an iterator
#[implement(super::Service)]
pub fn all_profile_keys<'a>(
&'a self,
user_id: &'a UserId,
) -> impl Stream<Item = (String, serde_json::Value)> + 'a + Send {
type KeyVal = ((Ignore, String), serde_json::Value);
let prefix = (user_id, Interfix);
self.db
.useridprofilekey_value
.stream_prefix(&prefix)
.ignore_err()
.map(|((_, key), val): KeyVal| (key, val))
}
/// Sets a new profile key value, removes the key if value is None
#[implement(super::Service)]
pub fn set_profile_key(
&self,
user_id: &UserId,
profile_key: &str,
profile_key_value: Option<serde_json::Value>,
) {
// TODO: insert to the stable MSC4175 key when it's stable
let key = (user_id, profile_key);
if let Some(value) = profile_key_value {
self.db
.useridprofilekey_value
.put(key, Json(value));
} else {
self.db.useridprofilekey_value.del(key);
}
}
/// Get the timezone of a user.
#[implement(super::Service)]
pub async fn timezone(&self, user_id: &UserId) -> Result<String> {
// TODO: transparently migrate unstable key usage to the stable key once MSC4133
// and MSC4175 are stable, likely a remove/insert in this block.
// first check the unstable prefix then check the stable prefix
let unstable_key = (user_id, "us.cloke.msc4175.tz");
let stable_key = (user_id, "m.tz");
self.db
.useridprofilekey_value
.qry(&unstable_key)
.or_else(|_| self.db.useridprofilekey_value.qry(&stable_key))
.await
.deserialized()
}
/// Sets a new timezone or removes it if timezone is None.
#[implement(super::Service)]
pub fn set_timezone(&self, user_id: &UserId, timezone: Option<String>) {
// TODO: insert to the stable MSC4175 key when it's stable
let key = (user_id, "us.cloke.msc4175.tz");
if let Some(timezone) = timezone {
self.db
.useridprofilekey_value
.put_raw(key, &timezone);
} else {
self.db.useridprofilekey_value.del(key);
}
}
+67 -1
View File
@@ -1407,7 +1407,7 @@
# Sentry.io crash/panic reporting, performance monitoring/metrics, etc.
# This is NOT enabled by default. tuwunel's default Sentry reporting
# endpoint domain is `o4506996327251968.ingest.us.sentry.io`.
# endpoint domain is `o4509498990067712.ingest.us.sentry.io`.
#
#sentry = false
@@ -1712,3 +1712,69 @@
# example: "(objectClass=tuwunelAdmin)" or "(uid={username})"
#
#admin_filter = false
[global.jwt]
# Enable JWT logins
#
#enable = false
# Validation key, also called 'secret' in Synapse config. The type of key
# can be configured in 'format', but defaults to the common HMAC which
# is a plaintext shared-secret, so you should keep this value private.
#
#key =
# Format of the 'key'. Only HMAC, ECDSA, and B64HMAC are supported
# Binary keys cannot be pasted into this config, so B64HMAC is an
# alternative to HMAC for properly random secret strings.
# - HMAC is a plaintext shared-secret private-key.
# - B64HMAC is a base64-encoded version of HMAC.
# - ECDSA is a PEM-encoded public-key.
#
#format = "HMAC"
# Automatically create new user from a valid claim, otherwise access is
# denied for an unknown even with an authentic token.
#
#register_user = true
# JWT algorithm
#
#algorithm = "HS256"
# Optional audience claim list. The token must claim one or more values
# from this list when set.
#
#audience = []
# Optional issuer claim list. The token must claim one or more values
# from this list when set.
#
#issuer = []
# Require expiration claim in the token. This defaults to false for
# synapse migration compatibility.
#
#require_exp = false
# Require not-before claim in the token. This defaults to false for
# synapse migration compatibility.
#
#require_nbf = false
# Validate expiration time of the token when present. Whether or not it is
# required depends on require_exp, but when present this ensures the token
# is not used after a time.
#
#validate_exp = true
# Validate not-before time of the token when present. Whether or not it is
# required depends on require_nbf, but when present this ensures the token
# is not used before a time.
#
#validate_nbf = true
# Bypass validation for diagnostic/debug use only.
#
#validate_signature = true