Compare commits

...
49 Commits
Author SHA1 Message Date
timedout a535449f15 chore: Release 2026-11-10 20:12:29 +00:00
timedout 7e3022f6ae chore: Add changelog frags 2026-11-10 20:11:33 +00:00
timedout c7dc06c693 fix: SEC34
Reviewed-By: Jade Ellis <jade@ellis.link>
Reviewed-By: Ginger <ginger@gingershaped.computer>
Reviewed-By: Jacob Taylor <jacob@explodie.org>
2026-11-10 20:07:18 +00:00
timedout 52562c4227 fix: SEC15
Reviewed-By: Jade Ellis <jade@ellis.link>
Reviewed-By: Ginger <ginger@gingershaped.computer>
2026-11-10 19:23:24 +00:00
timedout e9d0737e50 fix: SEC31
Reviewed-By: Jade Ellis <jade@ellis.link>
Reviewed-By: Ginger <ginger@gingershaped.computer>
2026-11-10 19:22:51 +00:00
Henry-HilesandGinger 005ee45feb fix: SEC39
Reviewed-By: timedout <git@nexy7574.co.uk>
Co-Authored-By: Ginger <ginger@gingershaped.computer>
2026-11-10 19:22:15 +00:00
timedout 46f71d55f7 fix: SEC36
Reviewed-By: Jacob Taylor <jacob@explodie.org>
Reviewed-By: Jade Ellis <jade@ellis.link>
Reviewed-By: Ginger <ginger@gingershaped.computer>
2026-11-10 19:20:08 +00:00
timedout 046074bbc2 chore: Release 2026-09-28 02:04:54 +01:00
timedout 550dd0160a fix: SEC8
Reviewed-By: Ginger <ginger@gingershaped.computer>
Reviewed-By: Jade <jade@ellis.link>
Reviewed-By: Jacob Taylor <jacob@explodie.org>
2026-09-28 02:00:37 +01:00
timedout c0ad864d57 fix: Make unembed unsigned run in bg 2026-09-27 22:58:08 +01:00
timedout 451c44800b chore: Add news frag 2026-09-27 19:19:50 +01:00
timedout a61026912c feat: Add a migration to fix up unsigned data 2026-09-27 18:31:46 +01:00
timedout 05d6922351 style: Take an optional context instead of deconstructed parameters 2026-09-27 17:33:26 +01:00
timedout f0a0596d12 fix: Remove undocumented prev_sender unsigned field 2026-09-27 17:32:47 +01:00
timedout fbedf994db fix: Don't unnecessarily embed prev_content in populate_unsigned 2026-09-27 17:32:47 +01:00
timedout 1de16fed9e style: English is a difficult language 2026-09-27 17:32:47 +01:00
timedout 4812837d13 fix: Don't allow set_unsigned to panic
While an error in this function is probably bad, detonating a metric tonne of TNT on the call tree is probably worse
2026-09-27 17:32:47 +01:00
timedout 90793eaaf4 fix: Actually set unsigned in PDU after set_unsigned 2026-09-27 17:32:47 +01:00
timedout 744928b206 style: Update docstring for Pdu.set_unsigned 2026-09-27 17:32:47 +01:00
timedout 05ee24df7d fix: Move helper to helpers.rs, rename field, don't recurse unsigned in redacted_because
Renamed `redacted_because_id` to `org.continuwuity.redacted_by` for proper namespacing; `get_unsigned_context` now lives in helpers.rs instead of `mod.rs`. The helper function also now explicitly wipes `unsigned` for `redacted_because` to avoid leaks (see comment).
2026-09-27 17:32:46 +01:00
timedout 89eecd2bf7 fix: Record redacted_because_id instead of redacted_because 2026-09-27 17:32:46 +01:00
timedout b2072baf59 perf: Strip unsigned over federation and generate it on the fly locally 2026-09-27 17:32:46 +01:00
timedout 286897de88 feat: Don't send unsigned data over federation 2026-09-27 17:32:46 +01:00
stratself 3ee4ab7c09 chore: Add changelog for #2259 2026-09-26 23:03:21 +00:00
stratself 173091624c docs(styling): Use lighter colors for nav bars in dark mode 2026-09-26 23:03:21 +00:00
stratself 0f21f42787 docs(landing): Style action buttons to add punch and match c10y's themes
The buttons are an attempt to match colors found on c10y's account
management landing page. With less round corners and a punchier color
scheme they should be more visible than previous versions (which were
reported to be a bit dull)

The donation button is stylized in green
2026-09-26 23:03:21 +00:00
stratself 31a0c4c9b5 docs: Add OpenCollective link and donate button 2026-09-26 23:03:21 +00:00
stratself 8d60b8dada docs: Stricter shortcut link redirections
Avoids some recursion and 404 endpoint issues. Serves as good
examples for future additions.

The "Intermittent federation failures to a specific server"
section is now ID'd as #intermittent for easier reachability.
2026-09-26 23:01:15 +00:00
Renovate Bot 84222bfc7b chore(deps): update rust crate smallvec to v1.16.2 2026-09-26 05:04:12 +00:00
Renovate Bot a4d2d4a720 chore(deps): update rust-zerover-patch-updates 2026-09-25 05:02:56 +00:00
Renovate BotandGinger 02d245afe5 chore(deps): Update ruma digest to e7384f0
Co-authored-by: Ginger <ginger@gingershaped.computer>
2026-09-24 14:16:55 -04:00
Renovate Bot 3e5ea8f850 chore(deps): update node-patch-updates to v2.0.22 2026-09-24 14:11:17 +00:00
Renovate Bot 5878594698 chore(deps): update dependency node to v24 2026-09-24 14:11:11 +00:00
Renovate Bot 4e8643147d chore(deps): update rust-non-major 2026-09-24 13:57:09 +00:00
Ginger 4ea4e15edb chore: Reformat with latest nightly 2026-09-24 09:53:59 -04:00
Erwan Leboucher cee69b4c8d fix: Walk auth events once instead of re-queueing parents 2026-09-22 21:19:25 +02:00
Renovate Bot 457b884491 chore(deps): update github-actions-digest 2026-09-22 05:03:35 +00:00
timedout d3ed175264 fix(stateres): Inject dummy create event when none is available
I'm not sure how this doesn't cause more panics. When running an auth check on the create event during iterative auth checks, there's obviously no create event in the base state nor auth map, so the call to fetch_state() will return nothing and thus panic.
This just inserts the incoming event if it's a create event and there isn't an existing one in the auth state. It's not used for the create event checks anyway so this has no effect aside from plugging the panic.
2026-09-22 02:04:39 +01:00
timedout 7924e96589 chore: add todo to fix 2026-09-22 00:16:27 +00:00
timedout d1b1274900 style: Unchain if statement 2026-09-22 00:16:27 +00:00
timedout 6299df3552 style: Document fetch_and_handle_auth_events control flow 2026-09-22 00:16:27 +00:00
Erwan Leboucher 20d146bd5e fix: Throttle client-triggered remote event fetches 2026-09-22 00:16:27 +00:00
Erwan Leboucher 03f60c7002 fix: Don't re-fetch auth events that can never be resolved 2026-09-22 00:16:27 +00:00
Erwan Leboucher ed1e634579 feat(push): Add MSC4174 web push 2026-09-21 23:35:05 +00:00
Renovate Bot dbcffe8af0 chore(deps): update ghcr.io/renovatebot/renovate docker tag to v44.104.2 2026-09-21 05:02:42 +00:00
timedout 8891adfba3 fix(stateres): Enforce state ownership checks against creators too
Previously being a room creator short-circuited the `can_send_event`
check, which allowed creators to send state events "owned" (state key
starting with @ but not being the sender's user ID), in effect skipping
auth check 9 in v12 rooms. This removes that short circuit condition,
since `Infinite < Int` always returns false anyway, allowing the state
key check block to execute.

Fixes #2266
2026-09-18 18:57:57 +01:00
Renovate Bot 4c8dc59f7a chore(deps): update rust-non-major 2026-09-17 05:03:15 +00:00
timedout 1a8e5ccc09 fix: Gracefully handle extremityless rooms when gapfilling
See also: #2259
2026-09-17 00:06:17 +01:00
timedout c8720a6ebc chore: Update admin room announcement 2026-09-16 22:12:25 +01:00
97 changed files with 4512 additions and 1648 deletions

No files matched your search

@@ -52,7 +52,7 @@ runs:
- name: Set up Docker Buildx
if: ${{ env.BUILTIN_REGISTRY_ENABLED == 'true' }}
uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4
with:
# Use persistent BuildKit if BUILDKIT_ENDPOINT is set (e.g. tcp://buildkit:8125)
driver: ${{ env.BUILDKIT_ENDPOINT != '' && 'remote' || 'docker-container' }}
@@ -67,7 +67,7 @@ runs:
uses: ./.forgejo/actions/rust-toolchain
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4
uses: docker/setup-buildx-action@f87e5991a6d7451dcb8d9637bfbc97413f497069 # v4
with:
# Use persistent BuildKit if BUILDKIT_ENDPOINT is set (e.g. tcp://buildkit:8125)
driver: ${{ env.BUILDKIT_ENDPOINT != '' && 'remote' || 'docker-container' }}
@@ -75,7 +75,7 @@ runs:
- name: Set up QEMU
if: ${{ env.BUILDKIT_ENDPOINT == '' }}
uses: docker/setup-qemu-action@1f40c72289eff860ee54a304f1438e3cff362e0a # v4
uses: docker/setup-qemu-action@99012661954931238ded8c8b007157a8430204e1 # v4
- name: Login to builtin registry
if: ${{ env.BUILTIN_REGISTRY_ENABLED == 'true' }}
+1 -1
View File
@@ -71,7 +71,7 @@ runs:
- name: Install timelord-cli and git-warp-time
if: steps.check-binaries.outputs.need-install == 'true'
uses: https://github.com/taiki-e/install-action@3f74d7c16a4242f1c95561e98edc25d36adb4375 # v2
uses: https://github.com/taiki-e/install-action@94c31af3204a9f15ab40b35ad084410b905bbc73 # v2
with:
tool: git-warp-time,timelord-cli@3.0.1
+1 -1
View File
@@ -34,7 +34,7 @@ jobs:
if: steps.runner-env.outputs.node_major == '' || steps.runner-env.outputs.node_major < '20'
uses: https://github.com/actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
with:
node-version: 22
node-version: 24
- name: Cache npm dependencies
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # v6
+2 -2
View File
@@ -62,7 +62,7 @@ jobs:
registry_password: ${{ secrets.BUILTIN_REGISTRY_PASSWORD || secrets.GITHUB_TOKEN }}
- name: Build and push Docker image by digest
id: build
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7
uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7
with:
context: .
file: "docker/Dockerfile"
@@ -150,7 +150,7 @@ jobs:
registry_password: ${{ secrets.BUILTIN_REGISTRY_PASSWORD || secrets.GITHUB_TOKEN }}
- name: Build and push max-perf Docker image by digest
id: build
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7
uses: docker/build-push-action@c3c9e263c25d99ce0380d002d59b67737d91b0dc # v7
with:
context: .
file: "docker/Dockerfile"
+1 -1
View File
@@ -43,7 +43,7 @@ jobs:
name: Renovate
runs-on: ubuntu-latest
container:
image: ghcr.io/renovatebot/renovate:44.83.2@sha256:7c83dd134ede3fd12ffbdaa67e426aafebd592f4ffdff8fdbae92d48f98324af
image: ghcr.io/renovatebot/renovate:44.104.2@sha256:d19051d0a908e098c7fbba04a16435ef136208bfa47105d59a9fe47059e2bd25
options: --tmpfs /tmp:exec
steps:
- name: Checkout
+17 -29
View File
@@ -10,18 +10,18 @@ repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v6.0.0
hooks:
- id: fix-byte-order-marker
- id: check-case-conflict
- id: check-symlinks
- id: destroyed-symlinks
- id: check-yaml
- id: check-json
- id: check-toml
- id: end-of-file-fixer
- id: trailing-whitespace
- id: mixed-line-ending
- id: check-merge-conflict
- id: check-added-large-files
- id: fix-byte-order-marker
- id: check-case-conflict
- id: check-symlinks
- id: destroyed-symlinks
- id: check-yaml
- id: check-json
- id: check-toml
- id: end-of-file-fixer
- id: trailing-whitespace
- id: mixed-line-ending
- id: check-merge-conflict
- id: check-added-large-files
- repo: https://github.com/crate-ci/typos
rev: v1.50.2
@@ -29,12 +29,12 @@ repos:
- id: typos
- id: typos
name: commit-msg-typos
stages: [commit-msg]
stages: [ commit-msg ]
- repo: https://github.com/crate-ci/committed
rev: v1.1.11
hooks:
- id: committed
- id: committed
- repo: local
hooks:
@@ -42,10 +42,10 @@ repos:
name: cargo fmt
entry: cargo +nightly fmt --
language: system
types: [rust]
types: [ rust ]
pass_filenames: false
stages:
- pre-commit
- pre-commit
- repo: local
hooks:
@@ -54,18 +54,6 @@ repos:
entry: cargo clippy -- -D warnings
language: system
pass_filenames: false
types: [rust]
types: [ rust ]
stages:
- pre-push
- repo: local
hooks:
- id: docs-build
name: docs build
entry: npm run docs:build
language: system
pass_filenames: false
files: docs/
types: [mdx]
stages:
- pre-commit
+29
View File
@@ -1,3 +1,32 @@
# Continuwuity v26.9.2 (2026-11-10)
## Bugfixes
- SEC15, reported by @stratself (SEC15)
- SEC31, reported by @tulir (SEC31)
- SEC34, reported by @jade (SEC34)
- SEC36, reported by @nex (SEC36)
- SEC39, reported by @Henry-Hiles (SEC39)
# Continuwuity v26.9.1 (2026-09-28)
## Features
- The server now calculates some data for `unsigned` data when sending it to clients, rather than embedding it at rest. This brings decreased storage costs and improved access control checks.
Contributed by @nex. ([#2270](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2270))
- Added encrypted Web Push ([MSC4174](https://github.com/matrix-org/matrix-spec-proposals/pull/4174)), enabled by default. Contributed by @eleboucher.
## Bugfixes
- Fixed exponential blowup in auth event fetching for deep auth chains, which caused runaway CPU and RAM until restart. Contributed by @eleboucher. (auth-event-walk)
- Fixed events being repeatedly re-requested over federation when they were fetched successfully but could not be resolved, and throttled remote event fetches made on behalf of clients. Contributed by @eleboucher. ([#2090](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2090))
## Improved Documentation
- Add Open Collective links to docs and restyle action buttons. Contributed by @stratself ([#2269](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2269))
# Continuwuity v26.9.0 (2026-09-16)
## Features
Generated
+300 -125
View File
File diff suppressed because it is too large. Load diff
+17 -2
View File
@@ -12,7 +12,7 @@ license = "Apache-2.0"
# See also `rust-toolchain.toml`
readme = "README.md"
repository = "https://forgejo.ellis.link/continuwuation/continuwuity"
version = "26.9.0"
version = "26.9.2"
[workspace.metadata.crane]
name = "conduwuit"
@@ -66,6 +66,15 @@ default-features = false
version = "0.13.0"
default-features = false
[workspace.dependencies.web-push-native]
version = "0.5.0"
default-features = false
[workspace.dependencies.p256]
version = "0.13.2"
default-features = false
features = ["ecdsa", "std"]
# used for checking if an IP is in specific subnets / CIDR ranges easier
[workspace.dependencies.ipaddress]
version = "0.2.0"
@@ -83,6 +92,10 @@ version = "0.1.3"
[workspace.dependencies.http]
version = "1.3.1"
# parses the HTTP-date form of Retry-After
[workspace.dependencies.httpdate]
version = "1.0.3"
[workspace.dependencies.regex]
version = "1.11.1"
@@ -334,7 +347,7 @@ version = "1.1.1"
# Used for matrix spec type definitions and helpers
[workspace.dependencies.ruma]
git = "https://github.com/ruma/ruma.git"
rev = "83d80c8d869e0ba09becbf7e7378eefc7377cfdb"
rev = "e7384f01eacd454c593683de50491201e2160fd1"
features = [
"appservice-api-c",
"client-api",
@@ -962,6 +975,8 @@ let_underscore_future = { level = "allow", priority = 1 }
# rust doesnt understand conduwuit's custom log macros
literal_string_with_formatting_args = { level = "allow", priority = 1 }
# This lint is stupid, there's no point defining generics for simple trait params
impl-trait-in-params = { level = "allow", priority = 1 }
needless_raw_string_hashes = "allow"
+70 -30
View File
@@ -891,48 +891,88 @@
#proxy = "none"
# Servers listed here will be used to gather public keys of other servers
# (notary trusted key servers).
# (trusted key notary servers), as well as serve as trusted servers for
# other operations like backfill and event fetching.
#
# Currently, continuwuity doesn't support inbound batched key requests, so
# Currently, continuwuity doesn't support inbound key requests, so
# this list should only contain other Synapse servers.
#
# example: ["matrix.org", "tchncs.de"]
# example: ["matrix.org", "starstruck.systems"]
#
# It is possible to restrict which signing keys trusted servers are
# allowed to sign responses with. Without configuring this, all responses
# from trusted servers will be implicitly trusted, which is fine for most
# users. Strict verification is only necessary if TLS interception is part
# of your threat model. Even without strict verification, signatures from
# the originating server must be valid, regardless of whether the trusted
# server's is valid or not.
#
# To enable strict validation, you need to pass a TOML table containing
# the trusted server name, and any public keys (not key IDs!) that you
# expect to sign responses. You can fetch the current public keys a remote
# server uses with `!admin federation get-signing-keys <server name>`.
# Only add public keys that aren't expired, and make sure to remove any
# that expire at a later date!
#
# Example:
# ```ignore
# trusted_servers = [{server_name = "starstruck.systems", verify_keys=["Z+WZUaLLgkclXn1iQVzYIiJMjMEyltxg5LG8Dndu34A"]}]
# ```
#
# You can also mix-and-match strict verification and lax verification
# trusted servers:
#
# ```ignore
# trusted_servers = [
# {server_name = "starstruck.systems", verify_keys=["Z+WZUaLLgkclXn1iQVzYIiJMjMEyltxg5LG8Dndu34A"]},
# "matrix.org"
# ]
# ```
#
# Trusted servers are queried in parallel, so order does not matter. You
# should not have more than five entries. You can set this value to an
# empty list (`[]`) to operate without trusted server assistance, but this
# is discouraged for performance and reliability reasons.
#
#trusted_servers = ["matrix.org"]
# Whether to query the servers listed in trusted_servers first or query
# the origin server first. For best security, querying the origin server
# first is advised to minimize the exposure to a compromised trusted
# server. For maximum federation/join performance this can be set to true,
# however other options exist to query trusted servers first under
# specific high-load circumstances and should be evaluated before setting
# this to true.
# Whether to prioritise trusted server lookups over origin server lookups.
#
# By default, both notaries and origin servers are queried in parallel,
# and handled on a first-come-first-serve basis, meaning the server always
# uses the fastest response(s).
#
# If you wish to prioritise handling responses from trusted servers
# *before* handling responses from origin servers, set this to true.
#
#query_trusted_key_servers_first = false
# Whether to query the servers listed in trusted_servers first
# specifically on room joins. This option limits the exposure to a
# compromised trusted server to room joins only. The join operation
# requires gathering keys from many origin servers which can cause
# significant delays. Therefor this defaults to true to mitigate
# unexpected delays out-of-the-box. The security-paranoid or those willing
# to tolerate delays are advised to set this to false. Note that setting
# query_trusted_key_servers_first to true causes this option to be
# ignored.
# Any old signing keys that were used to sign events using this server
# name.
#
#query_trusted_key_servers_first_on_join = true
# Only query trusted servers for keys and never the origin server. This is
# intended for clusters or custom deployments using their trusted_servers
# as forwarding-agents to cache and deduplicate requests. Notary servers
# do not act as forwarding-agents by default, therefor do not enable this
# unless you know exactly what you are doing.
# You do not need to populate this unless you previously used a different
# server software with the same server_name configured, and then have
# migrated to continuwuity.
#
#only_query_trusted_key_servers = false
# Maximum number of keys to request in each trusted server batch query.
# The expiry timestamp should be as exact as you can get it, as making it
# too old may inadvertently invalidate historical events sent by your
# server, and potentially make it impossible for you to use some rooms as
# a result. If your key was compromised, and you set the expiry timestamp
# too late, some forged events may continue to be allowed.
#
#trusted_server_batch_size = 1024
# If not configured, continuwuity will discover its own historical keys
# through trusted servers. For improved security, you can set this to an
# empty map ({}) to disable this behaviour, if you know you have no old
# signing keys.
#
# Example:
# {"ed25519:1"={expired_ts=1790506890859,key="d29vZndvb2Z3b29md29vZg"}}
#
# The map key should be the key ID, expired_ts should be unix
# milliseconds, and the actual key value should be the unpadded base64
# representation of the public key component of the expired signing key.
#
#old_verify_keys =
# Max log level for continuwuity. Allows debug, info, warn, or error.
#
+3
View File
@@ -15,6 +15,9 @@ hero:
- theme: alt
text: Star on GitHub
link: https://github.com/continuwuity/continuwuity
- theme: alt
text: Donate to us!
link: https://opencollective.com/continuwuity
image:
src: /assets/logo.svg
alt: continuwuity logo
@@ -6,10 +6,10 @@
"message": "Welcome to Continuwuity! Important announcements about the project will appear here."
},
{
"id": 19,
"id": 20,
"mention_room": true,
"date": "2026-08-24",
"message": "[Continuwuity 26.8.1](https://forgejo.ellis.link/continuwuation/continuwuity/releases/tag/v26.8.1) has just been released - this is a bug fix and feature release, with one small security patch. Please read the release notes for more information, before updating as soon as possible.\n\nAny questions? Ask in our [support room](https://matrix.to/#/#continuwuity:continuwuity.org)."
"date": "2026-09-16",
"message": "[Continuwuity 26.9.0](https://forgejo.ellis.link/continuwuation/continuwuity/releases/tag/v26.9.0) has just been released - this update features a bunch of bug fixes, and some bandwidth saving improvements for sliding sync.\n\nAny questions? Ask in our [support room](https://matrix.to/#/#continuwuity:continuwuity.org)!"
}
]
}
+1 -1
View File
@@ -64,7 +64,7 @@ Matrix federation is extremely heavy and sends wild amounts of DNS requests. Thi
Follow the [**DNS tuning guide**](./guides/dns) for details on setting it up.
### Intermittent federation failures to a specific server
### Intermittent federation failures to a specific server {#intermittent}
There may be circumstances where servers fail to connect to each other, probably due to a bad DNS cache. In such cases, issuing `!admin debug ping <SERVER_NAME>` would return some errors, and `!admin query resolver cache <SERVER_NAME>` would return a wrong destination endpoint.
+16 -16
View File
@@ -455,17 +455,17 @@
}
},
"node_modules/@rspress/core": {
"version": "2.0.21",
"resolved": "https://registry.npmjs.org/@rspress/core/-/core-2.0.21.tgz",
"integrity": "sha512-QxhVqhHhN4guA24MlvlsFTb96pE/X8gBd5RBfp4j1kn5tAu1nJ7A9itKcRAuEw0WYJHcsWtI+WUnd7JPioHtlw==",
"version": "2.0.22",
"resolved": "https://registry.npmjs.org/@rspress/core/-/core-2.0.22.tgz",
"integrity": "sha512-mS/+xJYxVSR72GAjTz8GIGcJwwb2NSbbduq1fQOSv7k73EJ2aa0Q/o14nnnI4DRmRSEmK0t81vkVGcKHWQzPcA==",
"dev": true,
"license": "MIT",
"dependencies": {
"@mdx-js/mdx": "^3.1.1",
"@mdx-js/react": "^3.1.1",
"@rsbuild/core": "^2.1.13",
"@rsbuild/core": "^2.2.5",
"@rsbuild/plugin-react": "~2.1.0",
"@rspress/shared": "2.0.21",
"@rspress/shared": "2.0.22",
"@shikijs/rehype": "^4.2.0",
"@types/mdast": "^4.0.4",
"@types/react": "^19.2.18",
@@ -484,7 +484,7 @@
"react-dom": "^19.2.8",
"react-lazy-with-preload": "^2.2.1",
"react-render-to-markdown": "19.1.0",
"react-router-dom": "^7.18.2",
"react-router-dom": "^7.18.3",
"rehype-external-links": "^3.0.0",
"rehype-raw": "^7.0.0",
"remark-cjk-friendly": "^2.3.1",
@@ -508,9 +508,9 @@
}
},
"node_modules/@rspress/plugin-client-redirects": {
"version": "2.0.21",
"resolved": "https://registry.npmjs.org/@rspress/plugin-client-redirects/-/plugin-client-redirects-2.0.21.tgz",
"integrity": "sha512-21giX10Blom+dOJBJOxXZUS+pjez1Tb9iNA7L7km8uUkn5L45KfdQvYqbBHY6Nv6L46zaDdz0Mt97eAqBPgNoA==",
"version": "2.0.22",
"resolved": "https://registry.npmjs.org/@rspress/plugin-client-redirects/-/plugin-client-redirects-2.0.22.tgz",
"integrity": "sha512-pyOLIfxji1xbh//H0egdmhXVdO5hfp9dLXCTnZKcet9K3FJpre21/syuhRbRwWviMpcKtE73PZVcSaNVTYQYZw==",
"dev": true,
"license": "MIT",
"engines": {
@@ -521,9 +521,9 @@
}
},
"node_modules/@rspress/plugin-sitemap": {
"version": "2.0.21",
"resolved": "https://registry.npmjs.org/@rspress/plugin-sitemap/-/plugin-sitemap-2.0.21.tgz",
"integrity": "sha512-PldjjLDFqtQZ3yPHHsWxmxGNlmoNH7nSx76ww7QkPzkO9rPrlr71sBLZR3MMc1s6cd5Csyd+gHjfIJiP7o77XQ==",
"version": "2.0.22",
"resolved": "https://registry.npmjs.org/@rspress/plugin-sitemap/-/plugin-sitemap-2.0.22.tgz",
"integrity": "sha512-2SNLW8Oh4osCyqnzea1WDVA9469Luwu0piNuh5HNjlzcgjguzynA3447WRFpqBL1Z/B7H5IB5SMcwYaMsgFhvQ==",
"dev": true,
"license": "MIT",
"engines": {
@@ -534,13 +534,13 @@
}
},
"node_modules/@rspress/shared": {
"version": "2.0.21",
"resolved": "https://registry.npmjs.org/@rspress/shared/-/shared-2.0.21.tgz",
"integrity": "sha512-siCWkv1a4n6y7JdVRII+fwCRZGZtG5KGFfKi2w/Tt33UnIgh4XGqozZ5F/K0iZ/QUwg0wi93SoVdOQm2Q+wf8w==",
"version": "2.0.22",
"resolved": "https://registry.npmjs.org/@rspress/shared/-/shared-2.0.22.tgz",
"integrity": "sha512-1UXwG6gAhWBw5vClv3wb2Oy8hPXYNgQwQ5zWXEG8auY5ZcMywvHCBi6pFJtYNOgXe5327lNgiOG0S1R8l0OLeg==",
"dev": true,
"license": "MIT",
"dependencies": {
"@rsbuild/core": "^2.1.13",
"@rsbuild/core": "^2.2.5",
"@shikijs/rehype": "^4.2.0",
"@types/react": "^19.2.18",
"mdast-util-mdx-jsx": "^3.2.0",
+11 -7
View File
@@ -52,6 +52,13 @@ export default defineConfig({
mode: 'link',
content: 'https://github.com/continuwuity/continuwuity',
},
{
icon: {
svg: `<svg role="img" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><title>Open Collective</title><path fill="currentColor" d="M12 0C5.373 0 0 5.373 0 12s5.373 12 12 12c2.54 0 4.894-.79 6.834-2.135l-3.107-3.109a7.715 7.715 0 1 1 0-13.512l3.107-3.109A11.943 11.943 0 0 0 12 0zm9.865 5.166l-3.109 3.107A7.67 7.67 0 0 1 19.715 12a7.682 7.682 0 0 1-.959 3.727l3.109 3.107A11.943 11.943 0 0 0 24 12c0-2.54-.79-4.894-2.135-6.834z"/></svg>`
},
mode: 'link',
content: 'https://opencollective.com/continuwuity',
},
],
lastUpdated: true,
enableContentAnimation: true,
@@ -65,16 +72,13 @@ export default defineConfig({
}),
pluginClientRedirects({
redirects: [{
from: '/configuration/examples',
from: '^/config_reference$',
to: '/reference/config'
}, {
from: '/admin_reference',
from: '^/admin_reference$',
to: '/reference/admin'
}, {
from: '/server_reference',
to: '/reference/server'
}, {
from: '/community$',
from: '^/community$',
to: '/community/guidelines'
}, {
from: "^/turn$",
@@ -83,7 +87,7 @@ export default defineConfig({
from: "^/livekit$",
to: "/calls/livekit",
}, {
from: "^/dns",
from: "^/dns$",
to: "/guides/dns",
}, {
from: "^/source(|_code)$",
+5 -59
View File
@@ -694,11 +694,11 @@ impl crate::Context<'_> {
| Ok(value) => match self
.services
.server_keys
.verify_json(&value, &room_version_rules)
.verify_event_json(&value, &room_version_rules)
.await
{
| Err(e) => return Err!("Signature verification failed: {e}"),
| Ok(()) => write!(self, "Signature correct"),
| Ok(_) => write!(self, "Signature correct"),
},
}
.await
@@ -714,7 +714,7 @@ impl crate::Context<'_> {
let msg = match self
.services
.server_keys
.verify_event(&event, &room_version_rules)
.verify_event_json(&event, &room_version_rules)
.await
{
| Err(e) => return Err(e),
@@ -849,7 +849,7 @@ impl crate::Context<'_> {
for result in remote_state_response.pdus.iter().map(|pdu| {
self.services
.server_keys
.validate_and_add_event_id(pdu, &room_version_rules)
.verify_event_json_no_fetch_add_event_id(pdu, &room_version_rules)
}) {
let Ok((event_id, value)) = result.await else {
continue;
@@ -887,7 +887,7 @@ impl crate::Context<'_> {
for result in remote_state_response.auth_chain.iter().map(|pdu| {
self.services
.server_keys
.validate_and_add_event_id(pdu, &room_version_rules)
.verify_event_json_no_fetch_add_event_id(pdu, &room_version_rules)
}) {
let Ok((event_id, value)) = result.await else {
continue;
@@ -943,60 +943,6 @@ impl crate::Context<'_> {
.await
}
pub(super) async fn get_signing_keys(
&self,
server_name: Option<OwnedServerName>,
notary: Option<OwnedServerName>,
query: bool,
) -> Result {
let server_name = server_name.unwrap_or_else(|| self.services.server.name.clone());
if let Some(notary) = notary {
let signing_keys = self
.services
.server_keys
.notary_request(&notary, &server_name)
.await?;
let out = format!("```rs\n{signing_keys:#?}\n```");
return self.write_str(&out).await;
}
let signing_keys = if query {
self.services
.server_keys
.server_request(&server_name)
.await?
} else {
self.services
.server_keys
.signing_keys_for(&server_name)
.await?
};
let out = format!("```rs\n{signing_keys:#?}\n```");
self.write_str(&out).await
}
pub(super) async fn get_verify_keys(&self, server_name: Option<OwnedServerName>) -> Result {
let server_name = server_name.unwrap_or_else(|| self.services.server.name.clone());
let keys = self
.services
.server_keys
.verify_keys_for(&server_name)
.await;
let mut out = String::new();
writeln!(out, "| Key ID | Public Key |")?;
writeln!(out, "| --- | --- |")?;
for (key_id, key) in keys {
writeln!(out, "| {key_id} | {key:?} |")?;
}
self.write_str(&out).await
}
pub(super) async fn resolve_true_destination(
&self,
server_name: OwnedServerName,
-16
View File
@@ -103,22 +103,6 @@ pub enum DebugCommand {
event_id: OwnedEventId,
},
/// Get and display signing keys from local cache or remote server.
GetSigningKeys {
server_name: Option<OwnedServerName>,
#[arg(long)]
notary: Option<OwnedServerName>,
#[arg(short, long)]
query: bool,
},
/// Get and display signing keys from local cache or remote server.
GetVerifyKeys {
server_name: Option<OwnedServerName>,
},
/// Sends a federation request to the remote server's
/// `/_matrix/federation/v1/version` endpoint and measures the latency it
/// took for the server to respond
+27 -1
View File
@@ -2,7 +2,7 @@ use std::fmt::Write;
use conduwuit::{Err, Result, utils::response::LimitReadExt};
use futures::StreamExt;
use ruma::{OwnedRoomId, OwnedServerName, OwnedUserId};
use ruma::{MilliSecondsSinceUnixEpoch, OwnedRoomId, OwnedServerName, OwnedUserId};
use crate::get_room_info;
@@ -131,4 +131,30 @@ impl crate::Context<'_> {
self.write_str(&format!("Rooms {user_id} shares with us ({num}):\n```\n{body}\n```"))
.await
}
pub(super) async fn get_signing_keys(&self, server_name: OwnedServerName) -> Result {
let response = self
.services
.server_keys
.origin_request(server_name, MilliSecondsSinceUnixEpoch::now())
.await?;
self.write_str("| Key ID | Public Key | Expired (unix milliseconds) |\n")
.await?;
self.write_str("| ------ | ---------- | --------------------------- |\n")
.await?;
for (key_id, key) in &response.verify_keys {
self.write_str(&format!("| {key_id} | {} | |\n", key.key))
.await?;
}
for (key_id, old_key) in &response.old_verify_keys {
self.write_str(&format!(
"| {key_id} | {} | {:0<13} |\n",
old_key.key,
old_key.expired_ts.get().to_string()
))
.await?;
}
Ok(())
}
}
+5
View File
@@ -39,4 +39,9 @@ pub enum FederationCommand {
RemoteUserInRooms {
user_id: OwnedUserId,
},
/// Queries the remote server for their current signing keys.
GetSigningKeys {
server_name: OwnedServerName,
},
}
-1
View File
@@ -32,7 +32,6 @@ use crate::{admin, admin::AdminCommand, context::Context};
#[must_use]
pub(super) fn complete(line: &str) -> String { complete_command(AdminCommand::command(), line) }
#[must_use]
pub(super) fn dispatch(services: Arc<Services>, command: CommandInput) -> ProcessorFuture {
Box::pin(handle_command(services, command))
}
-14
View File
@@ -1,6 +1,5 @@
use clap::Subcommand;
use conduwuit::Result;
use ruma::OwnedServerName;
use crate::Context;
@@ -12,12 +11,6 @@ pub enum GlobalsCommand {
CurrentCount,
LastCheckForAnnouncementsId,
/// This returns an empty `Ok(BTreeMap<..>)` when there are no keys found
/// for the server.
SigningKeysFor {
origin: OwnedServerName,
},
}
/// All the getters and iterators from src/database/key_value/globals.rs
@@ -47,13 +40,6 @@ pub(super) async fn process(subcommand: GlobalsCommand, context: &Context<'_>) -
.await;
let query_time = timer.elapsed();
write!(context, "Query completed in {query_time:?}:\n\n```rs\n{results:#?}\n```")
},
| GlobalsCommand::SigningKeysFor { origin } => {
let timer = tokio::time::Instant::now();
let results = services.server_keys.verify_keys_for(&origin).await;
let query_time = timer.elapsed();
write!(context, "Query completed in {query_time:?}:\n\n```rs\n{results:#?}\n```")
},
}
+1 -1
View File
@@ -204,7 +204,7 @@ impl crate::Context<'_> {
},
};
room_ids.push(room_id.clone());
room_ids.push(room_id);
}
if room_alias_or_id.is_room_alias_id() {
+6
View File
@@ -13,6 +13,7 @@ use ruma::{
},
assign,
};
use serde_json::json;
use crate::Ruma;
@@ -55,5 +56,10 @@ pub(crate) async fn get_capabilities_route(
assign!(ProfileFieldsCapability::new(true), { disallowed: Some(services.oidc.restricted_profile_fields()) }),
);
capabilities.set(
"org.matrix.msc4174.webpush",
json!({ "enabled": true, "vapid": services.pusher.webpush_vapid_public_key() }),
)?;
Ok(get_capabilities::v3::Response::new(capabilities))
}
+13 -2
View File
@@ -1,6 +1,7 @@
use axum::extract::State;
use conduwuit::{
Err, Event, Result, at, debug_warn, err, ref_at,
result::LogErr,
utils::{
IterStream,
future::TryExtExt,
@@ -98,7 +99,12 @@ pub(crate) async fn get_context_route(
.pdus_rev(room_id, Some(base_count))
.ignore_err()
.then(async |mut pdu| {
pdu.1.set_unsigned(Some(sender_user));
let ctx = services
.rooms
.timeline
.get_unsigned_context(&pdu.1, Some(sender_user))
.await;
pdu.1.set_unsigned(Some(ctx)).log_err().ok();
if let Err(e) = services
.rooms
.pdu_metadata
@@ -121,7 +127,12 @@ pub(crate) async fn get_context_route(
.pdus(room_id, Some(base_count))
.ignore_err()
.then(async |mut pdu| {
pdu.1.set_unsigned(Some(sender_user));
let ctx = services
.rooms
.timeline
.get_unsigned_context(&pdu.1, Some(sender_user))
.await;
pdu.1.set_unsigned(Some(ctx)).log_err().ok();
if let Err(e) = services
.rooms
.pdu_metadata
+18 -3
View File
@@ -523,7 +523,22 @@ where
for (server, response) in futures {
match response {
| Ok(response) => {
for (user, master_key) in response.master_keys {
let is_wanted = |user_id: &UserId| {
user_id.server_name() == server && device_keys_input.contains_key(user_id)
};
let filtered_master_keys = response
.master_keys
.into_iter()
.filter(|(user_id, _)| is_wanted(user_id));
let filtered_self_signing_keys = response
.self_signing_keys
.into_iter()
.filter(|(user_id, _)| is_wanted(user_id));
let filtered_device_keys = response
.device_keys
.into_iter()
.filter(|(user_id, _)| is_wanted(user_id));
for (user, master_key) in filtered_master_keys {
let (master_key_id, mut master_key) = parse_master_key(&user, &master_key)?;
if let Ok(our_master_key) = services
@@ -549,8 +564,8 @@ where
}
}
self_signing_keys.extend(response.self_signing_keys);
device_keys.extend(response.device_keys);
self_signing_keys.extend(filtered_self_signing_keys);
device_keys.extend(filtered_device_keys);
},
| Err(e) => {
failures.insert(server.to_string(), json!({ "error": e.to_string() }));
+8 -2
View File
@@ -140,7 +140,12 @@ pub(crate) async fn get_message_events_route(
.wide_filter_map(|item| visibility_filter(&services, item, sender_user))
.take(limit)
.then(async |mut pdu| {
pdu.1.set_unsigned(Some(sender_user));
let ctx = services
.rooms
.timeline
.get_unsigned_context(&pdu.1, Some(sender_user))
.await;
pdu.1.set_unsigned(Some(ctx)).log_err().ok();
if let Err(e) = services
.rooms
.pdu_metadata
@@ -375,5 +380,6 @@ pub(crate) async fn is_ignored_invite(
services
.users
.invite_filter_level(&sender_user, recipient_user)
.await == FilterLevel::Ignore
.await
== FilterLevel::Ignore
}
+7 -7
View File
@@ -1,7 +1,7 @@
use axum::extract::State;
use conduwuit::{Err, Result};
use futures::StreamExt;
use ruma::{OwnedRoomId, api::client::membership::mutual_rooms};
use ruma::{OwnedRoomId, api::client::membership::get_mutual_rooms};
use crate::Ruma;
@@ -13,8 +13,8 @@ use crate::Ruma;
#[tracing::instrument(skip_all, name = "mutual_rooms", level = "info")]
pub(crate) async fn get_mutual_rooms_unstable_route(
State(services): State<crate::State>,
body: Ruma<mutual_rooms::unstable::Request>,
) -> Result<mutual_rooms::unstable::Response> {
body: Ruma<get_mutual_rooms::unstable::Request>,
) -> Result<get_mutual_rooms::unstable::Response> {
let sender_user = body.identity.expect_sender_user()?;
if sender_user == body.user_id {
@@ -28,7 +28,7 @@ pub(crate) async fn get_mutual_rooms_unstable_route(
.collect()
.await;
Ok(mutual_rooms::unstable::Response::new(mutual_rooms))
Ok(get_mutual_rooms::unstable::Response::new(mutual_rooms))
}
/// # `GET /_matrix/client/v1/mutual_rooms`
@@ -37,8 +37,8 @@ pub(crate) async fn get_mutual_rooms_unstable_route(
#[tracing::instrument(skip_all, name = "mutual_rooms", level = "info")]
pub(crate) async fn get_mutual_rooms_route(
State(services): State<crate::State>,
body: Ruma<mutual_rooms::v1::Request>,
) -> Result<mutual_rooms::v1::Response> {
body: Ruma<get_mutual_rooms::v1::Request>,
) -> Result<get_mutual_rooms::v1::Response> {
let sender_user = body.identity.expect_sender_user()?;
if sender_user == body.user_id {
@@ -52,7 +52,7 @@ pub(crate) async fn get_mutual_rooms_route(
.collect()
.await;
Ok(mutual_rooms::v1::Response::new(
Ok(get_mutual_rooms::v1::Response::new(
mutual_rooms
.len()
.try_into()
+77 -8
View File
@@ -1,13 +1,16 @@
use std::sync::LazyLock;
use axum::extract::State;
use conduwuit::{Err, Error, Result, err};
use conduwuit_service::Services;
use conduwuit::{Err, Error, Result, err, utils::IterStream};
use conduwuit_service::{Services, pusher::ActivationOutcome};
use futures::StreamExt;
use ruma::{
CanonicalJsonObject, CanonicalJsonValue,
api::{
client::push::{
delete_pushrule, get_pushers, get_pushrule, get_pushrule_actions,
get_pushrule_enabled, get_pushrules_all, get_pushrules_global_scope, set_pusher,
set_pushrule, set_pushrule_actions, set_pushrule_enabled,
delete_pushrule, get_pushrule, get_pushrule_actions, get_pushrule_enabled,
get_pushrules_all, get_pushrules_global_scope, set_pushrule, set_pushrule_actions,
set_pushrule_enabled,
},
error::ErrorKind,
},
@@ -20,6 +23,12 @@ use ruma::{
RemovePushRuleError, Ruleset,
},
};
use ruminuwuity::pushers::{
PusherKind,
ack::unstable as pushers_ack,
get_pushers::{self, v3::PusherWithActivation},
set_pusher,
};
use crate::Ruma;
@@ -460,7 +469,29 @@ pub(crate) async fn get_pushers_route(
) -> Result<get_pushers::v3::Response> {
let sender_user = body.identity.expect_sender_user()?;
Ok(get_pushers::v3::Response::new(services.pusher.get_pushers(sender_user).await))
let pushers = services
.pusher
.get_pushers(sender_user)
.await
.into_iter()
.stream()
.then(async |pusher| {
let activated = match pusher.kind {
| PusherKind::WebPush(_) => Some(
services
.pusher
.is_webpush_activated(sender_user, pusher.ids.pushkey.as_str())
.await,
),
| _ => None,
};
PusherWithActivation { pusher, activated }
})
.collect()
.await;
Ok(get_pushers::v3::Response::new(pushers))
}
/// # `POST /_matrix/client/r0/pushers/set`
@@ -475,12 +506,50 @@ pub(crate) async fn set_pushers_route(
let sender_user = body.identity.expect_sender_user()?;
let sender_device = body.identity.expect_sender_device()?;
services
let needs_activation = services
.pusher
.set_pusher(sender_user, sender_device, &body.action)
.await?;
Ok(set_pusher::v3::Response::new())
Ok(set_pusher::v3::Response::new(needs_activation))
}
pub(crate) async fn pushers_ack_route(
State(services): State<crate::State>,
body: Ruma<pushers_ack::Request>,
) -> Result<pushers_ack::Response> {
let sender_user = body.identity.expect_sender_user()?;
match services
.pusher
.activate_webpush_pusher(sender_user, &body.app_id, &body.ack_token)
.await?
{
| ActivationOutcome::Activated => Ok(pushers_ack::Response::new()),
| ActivationOutcome::NoSuchPusher => {
Err!(Request(NotFound("No pusher with this app ID exists.")))
},
| ActivationOutcome::UnknownToken => Err(Error::Request(
UNKNOWN_ACTIVATION_TOKEN.clone(),
"Unknown activation token.".into(),
http::StatusCode::BAD_REQUEST,
)),
| ActivationOutcome::Expired => Err(Error::Request(
EXPIRED_ACTIVATION_TOKEN.clone(),
"Activation token has expired.".into(),
http::StatusCode::BAD_REQUEST,
)),
}
}
static EXPIRED_ACTIVATION_TOKEN: LazyLock<ErrorKind> =
LazyLock::new(|| custom_error_kind("ORG.MATRIX.MSC4174_EXPIRED_ACTIVATION_TOKEN"));
static UNKNOWN_ACTIVATION_TOKEN: LazyLock<ErrorKind> =
LazyLock::new(|| custom_error_kind("ORG.MATRIX.MSC4174_UNKNOWN_ACTIVATION_TOKEN"));
fn custom_error_kind(errcode: &str) -> ErrorKind {
serde_json::from_value(serde_json::json!({ "errcode": errcode }))
.expect("errcode deserializes into an ErrorKind")
}
/// user somehow has bad push rules, these must always exist per spec.
+35 -10
View File
@@ -1,6 +1,5 @@
use axum::extract::State;
use conduwuit::{Err, Event, Result, debug_warn, err};
use futures::{FutureExt, TryFutureExt, future::try_join};
use ruma::api::client::room::get_room_event;
use crate::{Ruma, client::is_ignored_pdu};
@@ -16,19 +15,40 @@ pub(crate) async fn get_room_event_route(
let event_id = &body.event_id;
let room_id = &body.room_id;
let event = services
.rooms
.timeline
.get_remote_pdu(room_id, event_id)
.map_err(|_| err!(Request(NotFound("Event {} not found.", event_id))));
let mut event = match services.rooms.timeline.get_pdu(event_id).await {
| Ok(event) => event,
| Err(e) if e.is_not_found() => {
// Only fetch over federation for users who could see the event.
let (joined, world_readable) = tokio::join!(
services.rooms.state_cache.is_joined(sender_user, room_id),
services.rooms.state_accessor.is_world_readable(room_id)
);
if !joined && !world_readable {
return Err!(Request(NotFound("Event {event_id} not found.")));
}
services
.rooms
.timeline
.get_remote_pdu(room_id, event_id)
.await
.map_err(|_| err!(Request(NotFound("Event {event_id} not found."))))?
},
| Err(e) => return Err(e),
};
if !event.room_id_or_hash().eq(room_id) {
// Return not found instead of invalid param to avoid revealing the
// existence of events the user might not be allowed to know about.
return Err!(Request(NotFound("Event {event_id} not found.")));
}
// NOTE: checked after fetching, as visibility of an unknown event cannot
// be determined (user_can_see_event returns true for those).
let visible = services
.rooms
.state_accessor
.user_can_see_event(sender_user, room_id, event_id)
.map(Ok);
let (mut event, visible) = try_join(event, visible).await?;
.await;
if !visible || is_ignored_pdu(services, &event, sender_user).await? {
return Err!(Request(Forbidden("You don't have permission to view this event.")));
@@ -43,7 +63,12 @@ pub(crate) async fn get_room_event_route(
debug_warn!("Failed to add bundled aggregations to event: {e}");
}
event.set_unsigned(Some(body.identity.expect_sender_user()?));
let ctx = services
.rooms
.timeline
.get_unsigned_context(&event, Some(sender_user))
.await;
event.set_unsigned(Some(ctx))?;
Ok(get_room_event::v3::Response::new(event.into_format()))
}
+7 -1
View File
@@ -1,6 +1,7 @@
use axum::extract::State;
use conduwuit::{
Err, Event, Result, at, debug_warn,
result::LogErr,
utils::{BoolExt, stream::TryTools},
};
use futures::{FutureExt, TryStreamExt, future::try_join4};
@@ -53,7 +54,12 @@ pub(crate) async fn room_initial_sync_route(
.pdus_rev(room_id, None)
.try_take(limit)
.and_then(async |mut pdu| {
pdu.1.set_unsigned(Some(sender_user));
let ctx = services
.rooms
.timeline
.get_unsigned_context(&pdu.1, Some(sender_user))
.await;
pdu.1.set_unsigned(Some(ctx)).log_err().ok();
if let Err(e) = services
.rooms
.pdu_metadata
+20 -8
View File
@@ -6,8 +6,10 @@ use std::collections::VecDeque;
use conduwuit::{
Event, PduCount, Result, debug_warn, err,
matrix::pdu::PduEvent,
ref_at, trace,
utils::stream::{BroadbandExt, ReadyExt, TryIgnore},
ref_at,
result::LogErr,
trace,
utils::stream::{BroadbandExt, ReadyExt, TryIgnore, WidebandExt},
};
use conduwuit_service::Services;
use futures::StreamExt;
@@ -83,9 +85,14 @@ async fn load_timeline(
.pdus_rev(room_id, ending_count.map(|count| count.saturating_add(1)))
.ignore_err()
.ready_take_while(move |&(pducount, _)| pducount > starting_count)
.map(move |mut pdu| {
pdu.1.set_unsigned(Some(sender_user));
pdu
.wide_filter_map(async |mut pdu| {
let ctx = services
.rooms
.timeline
.get_unsigned_context(&pdu.1, Some(sender_user))
.await;
pdu.1.set_unsigned(Some(ctx)).log_err().ok();
Some(pdu)
})
.then(async move |mut pdu| {
if let Err(e) = services
@@ -108,9 +115,14 @@ async fn load_timeline(
.timeline
.pdus_rev(room_id, ending_count.map(|count| count.saturating_add(1)))
.ignore_err()
.map(move |mut pdu| {
pdu.1.set_unsigned(Some(sender_user));
pdu
.wide_filter_map(async |mut pdu| {
let ctx = services
.rooms
.timeline
.get_unsigned_context(&pdu.1, Some(sender_user))
.await;
pdu.1.set_unsigned(Some(ctx)).log_err().ok();
Some(pdu)
})
.then(async move |mut pdu| {
if let Err(e) = services
+7 -1
View File
@@ -6,6 +6,7 @@ use conduwuit::{
Event,
pdu::{PduCount, PduEvent, sticky},
},
result::LogErr,
trace,
utils::{
BoolExt, IterStream, ReadyExt, TryFutureExtExt,
@@ -322,7 +323,12 @@ async fn build_sticky_events(
continue;
}
pdu.set_unsigned(Some(syncing_user));
let ctx = services
.rooms
.timeline
.get_unsigned_context(&pdu, Some(sync_context.syncing_user))
.await;
pdu.set_unsigned(Some(ctx)).log_err().ok();
events.push(Event::into_format(pdu));
}
+13 -5
View File
@@ -11,6 +11,7 @@ use conduwuit::{
Event, TypeStateKey,
pdu::{PduCount, PduEvent, sticky},
},
result::LogErr,
trace,
utils::{
BoolExt, FutureBoolExt, IterStream, ReadyExt, TryFutureExtExt,
@@ -725,7 +726,8 @@ async fn stripped_state_event(
.get_field::<StateEventType>("type")
.ok()
.flatten()
.as_ref() == Some(event_type)
.as_ref()
== Some(event_type)
})
}
@@ -1188,12 +1190,12 @@ async fn collect_sticky_events(
// MSC4480 lets us ignore the limit for a newly visible room's backlog
for (room_id, pdu) in backlog {
push_sticky_event(&mut response, sender_user, room_id, pdu);
push_sticky_event(services, &mut response, sender_user, room_id, pdu).await;
}
let mut next_batch = None;
for (count, room_id, pdu) in stream.into_iter().take(limit) {
push_sticky_event(&mut response, sender_user, room_id, pdu);
push_sticky_event(services, &mut response, sender_user, room_id, pdu).await;
next_batch = Some(count);
}
@@ -1204,13 +1206,19 @@ async fn collect_sticky_events(
Ok(response)
}
fn push_sticky_event(
async fn push_sticky_event(
services: &Services,
response: &mut sync_events::v5::response::StickyEvents,
sender_user: &UserId,
room_id: OwnedRoomId,
mut pdu: PduEvent,
) {
pdu.set_unsigned(Some(sender_user));
let ctx = services
.rooms
.timeline
.get_unsigned_context(&pdu, Some(sender_user))
.await;
pdu.set_unsigned(Some(ctx)).log_err().ok();
response
.rooms
.entry(room_id)
+1 -4
View File
@@ -171,6 +171,7 @@ pub fn build(router: Router<State>, state: State) -> Router<State> {
.ruma_route(&client::get_key_changes_route)
.ruma_route(&client::get_pushers_route)
.ruma_route(&client::set_pushers_route)
.ruma_route(&client::pushers_ack_route)
.ruma_route(&client::upgrade_room_route)
.ruma_route(&client::get_threads_route)
.ruma_route(&client::get_relating_events_with_rel_type_and_event_type_route)
@@ -199,10 +200,6 @@ pub fn build(router: Router<State>, state: State) -> Router<State> {
router = router
.ruma_route(&server::get_server_version_route)
.route("/_matrix/key/v2/server", get(server::get_server_keys_route))
.route(
"/_matrix/key/v2/server/{key_id}",
get(server::get_server_keys_deprecated_route),
)
.ruma_route(&server::get_public_rooms_route)
.ruma_route(&server::get_public_rooms_filtered_route)
.ruma_route(&server::send_transaction_message_route)
+19 -8
View File
@@ -1,9 +1,13 @@
use std::any::{Any, TypeId};
use conduwuit::{Err, Error, Result, err, utils::IterStream};
use conduwuit::{
Err, Error, Result, err,
utils::{IterStream, millis_since_unix_epoch},
};
use http::StatusCode;
use ruma::{
DeviceId, OwnedDeviceId, OwnedServerName, OwnedUserId, UserId,
DeviceId, MilliSecondsSinceUnixEpoch, OwnedDeviceId, OwnedServerName, OwnedUserId, UInt,
UserId,
api::{
IncomingRequest,
auth_scheme::{
@@ -127,18 +131,24 @@ impl CheckAuth for ServerSignatures {
return Err!(Request(Unauthorized("Destination mismatch.")));
}
let min_valid_ts =
MilliSecondsSinceUnixEpoch(UInt::new_saturating(millis_since_unix_epoch()));
let key = services
.server_keys
.get_verify_key(&output.origin, &output.key)
.get_single_verify_key(&output.origin, &output.key, min_valid_ts)
.await
.map_err(|e| {
err!(Request(Unauthorized(warn!("Failed to fetch signing keys: {e}"))))
.ok_or_else(|| {
err!(Request(Unauthorized(debug_warn!(
origin=%output.origin,
"Unable to acquire your signing key (ID: {})",
output.key
))))
})?;
let keys: PubKeys = [(output.key.to_string(), key.key)].into();
let keys: PubKeyMap = [(output.origin.as_str().into(), keys)].into();
match output.verify_request(request, destination, &keys) {
match output.verify_http_request(request, destination, &keys) {
| Ok(()) => {
if services
.moderation
@@ -160,8 +170,9 @@ impl CheckAuth for ServerSignatures {
Ok(output.origin)
},
| Err(err) =>
Err!(Request(Unauthorized(warn!("Failed to verify X-Matrix header: {err}")))),
| Err(err) => Err!(Request(Unauthorized(debug_warn!(
"Failed to verify X-Matrix header: {err}"
)))),
}
}
}
-10
View File
@@ -3,7 +3,6 @@ use std::cmp;
use axum::extract::State;
use conduwuit::{
Err, Event, PduCount, Result, info,
result::LogErr,
utils::{IterStream, ReadyExt, stream::TryTools},
};
use futures::{FutureExt, StreamExt, TryStreamExt};
@@ -80,15 +79,6 @@ pub(crate) async fn get_backfill_route(
.await
.then_some(pdu))
})
.and_then(async |mut pdu| {
// Strip the transaction ID, as that is private
pdu.remove_transaction_id().log_err().ok();
// Add age, as this is specified
pdu.add_age().log_err().ok();
// It's not clear if we should strip or add any more data, leave as
// is. In particular: Redaction?
Ok(pdu)
})
.try_filter_map(|pdu| async move {
Ok(services
.rooms
+2 -2
View File
@@ -3,7 +3,7 @@ use std::collections::{HashSet, VecDeque};
use axum::extract::State;
use conduwuit::{Err, Event, Result, debug, info, trace, utils::to_canonical_object, warn};
use ruma::{OwnedEventId, api::federation::event::get_missing_events};
use serde_json::{json, value::RawValue};
use serde_json::value::RawValue;
use service::rooms::event_handler::GET_MISSING_EVENTS_MAX_BATCH_SIZE;
use super::AccessCheck;
@@ -91,7 +91,7 @@ pub(crate) async fn get_missing_events_route(
.await
{
debug!(%next_event_id, origin = %body.identity, "redacting event origin cannot see");
pdu.redact(&room_version, json!({}))?;
pdu.redact(&room_version, None)?;
}
trace!(
+1 -2
View File
@@ -324,7 +324,7 @@ async fn validate_invite_state(
services
.server_keys
.verify_event(&state_event_json, room_version_rules)
.verify_event_json(&state_event_json, room_version_rules)
.await
.map_err(|e| {
err!(Request(InvalidParam("Signature verification failed on invite event: {e}")))
@@ -401,7 +401,6 @@ async fn validate_invite_create_event(
if !state_res::auth_check(
room_version_rules,
pdu,
None,
|_, _| async {
unreachable!("No state should be fetched when processing a lone create event");
},
+48 -38
View File
@@ -1,10 +1,8 @@
use std::{
mem::take,
time::{Duration, SystemTime},
};
use std::{collections::BTreeMap, mem::take, time::Duration};
use axum::{Json, extract::State, response::IntoResponse};
use conduwuit::{Result, utils::timepoint_from_now};
use futures::StreamExt;
use ruma::{
MilliSecondsSinceUnixEpoch,
api::{
@@ -14,33 +12,62 @@ use ruma::{
assign,
serde::Raw,
};
use service::{server_keys, server_keys::in_one_week};
/// # `GET /_matrix/key/v2/server`
///
/// Gets the public signing keys of this server.
///
/// - Matrix does not support invalidating public keys, so the key returned by
/// this will be valid forever.
// Response type for this endpoint is Json because we need to calculate a
// signature for the response
pub(crate) async fn get_server_keys_route(
State(services): State<crate::State>,
) -> Result<impl IntoResponse> {
let server_name = services.globals.server_name();
let active_key_id = services.server_keys.active_key_id();
let mut all_keys = services.server_keys.verify_keys_for(server_name).await;
let verify_keys = BTreeMap::from([services.server_keys.active_verify_key()]);
let verify_keys = all_keys
.remove_entry(active_key_id)
.expect("active verify_key is missing");
let old_verify_keys = all_keys
.into_iter()
.map(|(id, key)| (id, OldVerifyKey::new(expires_ts(), key.key)))
.collect();
let old_verify_keys = if let Some(k) = &services.config.old_verify_keys {
k.to_owned()
} else {
// We need to figure out any old signing keys we've acquired from
// notaries.
let mut keys = services.server_keys.signing_keys_for(server_name);
let mut old_keys = BTreeMap::new();
while let Some(resp) = keys.next().await {
// We need to verify that this response is still trusted based on
// the current configuration. It's possible we got this response
// from a notary who we no longer trust.
if resp.valid_until_ts > in_one_week() {
continue;
}
let trusted = services.config.trusted_servers.iter().any(|notary| {
server_keys::Service::verify_server_keys_response(
&resp,
Some((notary.server_name(), notary.verify_keys())),
)
.is_ok()
});
if !trusted {
continue;
}
for (old_key_id, old_key) in resp.old_verify_keys {
if old_key.expired_ts > resp.valid_until_ts {
// This key expired in the future which is probably illegal
continue;
}
old_keys
.entry(old_key_id)
.and_modify(|current_old_key: &mut OldVerifyKey| {
// Use the lowest expiry timestamp
if current_old_key.expired_ts > old_key.expired_ts {
*current_old_key = old_key.clone();
}
})
.or_insert(old_key);
}
}
old_keys
};
let server_key = assign!(ServerSigningKeys::new(server_name.to_owned(), valid_until_ts()), {
verify_keys: [verify_keys].into(),
verify_keys,
old_verify_keys,
});
@@ -56,24 +83,7 @@ pub(crate) async fn get_server_keys_route(
}
fn valid_until_ts() -> MilliSecondsSinceUnixEpoch {
let dur = Duration::from_hours(168);
let dur = Duration::from_hours(12);
let timepoint = timepoint_from_now(dur).expect("SystemTime should not overflow");
MilliSecondsSinceUnixEpoch::from_system_time(timepoint).expect("UInt should not overflow")
}
fn expires_ts() -> MilliSecondsSinceUnixEpoch {
let timepoint = SystemTime::now();
MilliSecondsSinceUnixEpoch::from_system_time(timepoint).expect("UInt should not overflow")
}
/// # `GET /_matrix/key/v2/server/{keyId}`
///
/// Gets the public signing keys of this server.
///
/// - Matrix does not support invalidating public keys, so the key returned by
/// this will be valid forever.
pub(crate) async fn get_server_keys_deprecated_route(
State(services): State<crate::State>,
) -> impl IntoResponse {
get_server_keys_route(State(services)).await
}
+1 -1
View File
@@ -142,7 +142,7 @@ pub(crate) async fn validate_any_membership_event(
services
.server_keys
.verify_event(&pdu, room_version_rules)
.verify_event_json(&pdu, room_version_rules)
.await
.map_err(|e| {
err!(Request(InvalidParam("Signature verification failed on membership event: {e}")))
+29 -2
View File
@@ -1,8 +1,8 @@
use std::env::consts::OS;
use std::{collections::HashSet, env::consts::OS};
use either::Either;
use figment::Figment;
use ruma::events::room::server_acl::RoomServerAclEventContent;
use ruma::{OwnedServerName, events::room::server_acl::RoomServerAclEventContent};
use super::DEPRECATED_KEYS;
use crate::{Config, Err, Result, Server, debug, debug_info, debug_warn, error, warn};
@@ -318,6 +318,33 @@ pub fn check(config: &Config) -> Result {
);
}
if config
.trusted_servers
.iter()
.any(|t| t.server_name() == config.server_name)
&& !config.federation_loopback
{
return Err!(Config(
"trusted_servers",
"Your own `server_name` cannot appear in `trusted_servers`"
));
}
config
.trusted_servers
.iter()
.map(super::TrustedServer::server_name)
.try_fold(HashSet::<OwnedServerName>::new(), |mut servers, server_name| {
if !servers.insert(server_name.to_owned()) {
Err!(Config(
"trusted_servers",
"Duplicate `server_name` in `trusted_servers`: {}",
server_name.as_str()
))
} else {
Ok(servers)
}
})?;
Ok(())
}
+151 -41
View File
@@ -20,8 +20,12 @@ use lettre::message::Mailbox;
use openidconnect::{ClientId, ClientSecret, Scope};
use regex::RegexSet;
use ruma::{
OwnedRoomId, OwnedRoomOrAliasId, OwnedServerName, OwnedUserId, RoomVersionId,
api::client::{discovery::discover_support::ContactRole, rtc::RtcTransport},
OwnedRoomId, OwnedRoomOrAliasId, OwnedServerName, OwnedServerSigningKeyId, OwnedUserId,
RoomVersionId, ServerName,
api::{
client::{discovery::discover_support::ContactRole, rtc::RtcTransport},
federation::discovery::OldVerifyKey,
},
profile::ProfileFieldName,
serde::Base64,
};
@@ -1085,52 +1089,92 @@ pub struct Config {
pub proxy: ProxyConfig,
/// Servers listed here will be used to gather public keys of other servers
/// (notary trusted key servers).
/// (trusted key notary servers), as well as serve as trusted servers for
/// other operations like backfill and event fetching.
///
/// Currently, continuwuity doesn't support inbound batched key requests, so
/// Currently, continuwuity doesn't support inbound key requests, so
/// this list should only contain other Synapse servers.
///
/// example: ["matrix.org", "tchncs.de"]
/// example: ["matrix.org", "starstruck.systems"]
///
/// It is possible to restrict which signing keys trusted servers are
/// allowed to sign responses with. Without configuring this, all responses
/// from trusted servers will be implicitly trusted, which is fine for most
/// users. Strict verification is only necessary if TLS interception is part
/// of your threat model. Even without strict verification, signatures from
/// the originating server must be valid, regardless of whether the trusted
/// server's is valid or not.
///
/// To enable strict validation, you need to pass a TOML table containing
/// the trusted server name, and any public keys (not key IDs!) that you
/// expect to sign responses. You can fetch the current public keys a remote
/// server uses with `!admin federation get-signing-keys <server name>`.
/// Only add public keys that aren't expired, and make sure to remove any
/// that expire at a later date!
///
/// Example:
/// ```ignore
/// trusted_servers = [{server_name = "starstruck.systems", verify_keys=["Z+WZUaLLgkclXn1iQVzYIiJMjMEyltxg5LG8Dndu34A"]}]
/// ```
///
/// You can also mix-and-match strict verification and lax verification
/// trusted servers:
///
/// ```ignore
/// trusted_servers = [
/// {server_name = "starstruck.systems", verify_keys=["Z+WZUaLLgkclXn1iQVzYIiJMjMEyltxg5LG8Dndu34A"]},
/// "matrix.org"
/// ]
/// ```
///
/// Trusted servers are queried in parallel, so order does not matter. You
/// should not have more than five entries. You can set this value to an
/// empty list (`[]`) to operate without trusted server assistance, but this
/// is discouraged for performance and reliability reasons.
///
/// default: ["matrix.org"]
#[serde(default = "default_trusted_servers")]
pub trusted_servers: Vec<OwnedServerName>,
pub trusted_servers: Vec<TrustedServer>,
/// Whether to query the servers listed in trusted_servers first or query
/// the origin server first. For best security, querying the origin server
/// first is advised to minimize the exposure to a compromised trusted
/// server. For maximum federation/join performance this can be set to true,
/// however other options exist to query trusted servers first under
/// specific high-load circumstances and should be evaluated before setting
/// this to true.
/// Whether to prioritise trusted server lookups over origin server lookups.
///
/// By default, both notaries and origin servers are queried in parallel,
/// and handled on a first-come-first-serve basis, meaning the server always
/// uses the fastest response(s).
///
/// If you wish to prioritise handling responses from trusted servers
/// *before* handling responses from origin servers, set this to true.
#[serde(default)]
pub query_trusted_key_servers_first: bool,
/// Whether to query the servers listed in trusted_servers first
/// specifically on room joins. This option limits the exposure to a
/// compromised trusted server to room joins only. The join operation
/// requires gathering keys from many origin servers which can cause
/// significant delays. Therefor this defaults to true to mitigate
/// unexpected delays out-of-the-box. The security-paranoid or those willing
/// to tolerate delays are advised to set this to false. Note that setting
/// query_trusted_key_servers_first to true causes this option to be
/// ignored.
#[serde(default = "true_fn")]
pub query_trusted_key_servers_first_on_join: bool,
/// Only query trusted servers for keys and never the origin server. This is
/// intended for clusters or custom deployments using their trusted_servers
/// as forwarding-agents to cache and deduplicate requests. Notary servers
/// do not act as forwarding-agents by default, therefor do not enable this
/// unless you know exactly what you are doing.
#[serde(default)]
pub only_query_trusted_key_servers: bool,
/// Maximum number of keys to request in each trusted server batch query.
/// Any old signing keys that were used to sign events using this server
/// name.
///
/// default: 1024
#[serde(default = "default_trusted_server_batch_size")]
pub trusted_server_batch_size: usize,
/// You do not need to populate this unless you previously used a different
/// server software with the same server_name configured, and then have
/// migrated to continuwuity.
///
/// The expiry timestamp should be as exact as you can get it, as making it
/// too old may inadvertently invalidate historical events sent by your
/// server, and potentially make it impossible for you to use some rooms as
/// a result. If your key was compromised, and you set the expiry timestamp
/// too late, some forged events may continue to be allowed.
///
/// If not configured, continuwuity will discover its own historical keys
/// through trusted servers. For improved security, you can set this to an
/// empty map ({}) to disable this behaviour, if you know you have no old
/// signing keys.
///
/// Example:
/// {"ed25519:1"={expired_ts=1790506890859,key="d29vZndvb2Z3b29md29vZg"}}
///
/// The map key should be the key ID, expired_ts should be unix
/// milliseconds, and the actual key value should be the unpadded base64
/// representation of the public key component of the expired signing key.
///
/// default:
#[serde(default)]
pub old_verify_keys: Option<BTreeMap<OwnedServerSigningKeyId, OldVerifyKey>>,
/// Max log level for continuwuity. Allows debug, info, warn, or error.
///
@@ -2838,6 +2882,45 @@ pub enum OidcProfileKeyImportMode {
OnLogin,
}
#[derive(Clone, Debug, Deserialize)]
#[serde(untagged)]
pub enum TrustedServer {
Config(TrustedServerConfig),
Name(OwnedServerName),
}
impl TrustedServer {
#[must_use]
pub fn server_name(&self) -> &ServerName {
match self {
| Self::Name(server_name) => server_name,
| Self::Config(config) => &config.server_name,
}
}
#[must_use]
pub fn verify_keys(&self) -> &[Base64] {
let Self::Config(config) = self else {
return &[];
};
config.verify_keys.as_slice()
}
}
#[derive(Clone, Debug, Deserialize)]
pub struct TrustedServerConfig {
pub server_name: OwnedServerName,
#[serde(default)]
pub verify_keys: Vec<Base64>,
}
impl TrustedServerConfig {
#[must_use]
pub fn new(server_name: OwnedServerName) -> Self {
Self { server_name, verify_keys: Vec::new() }
}
}
const DEPRECATED_KEYS: &[&str] = &[
"cache_capacity",
"conduit_cache_capacity_modifier",
@@ -2853,6 +2936,26 @@ const DEPRECATED_KEYS: &[&str] = &[
];
impl Config {
/// Get the base domain to use for user-facing URLs.
#[must_use]
pub fn get_client_domain(&self) -> Url {
self.well_known.client.clone().unwrap_or_else(|| {
let host = self.server_name.host();
format!("https://{host}")
.as_str()
.try_into()
.expect("server name should be a valid host")
})
}
/// Contact for the VAPID `sub` claim.
#[must_use]
pub fn webpush_vapid_contact(&self) -> Url {
self.get_client_domain()
.join("/_continuwuity/about")
.expect("about page is a valid path")
}
/// Pre-initialize config
pub fn load(paths: &[PathBuf]) -> Result<Figment> {
let envs = [
@@ -3072,8 +3175,17 @@ fn default_otlp_protocol() -> String { "http".to_owned() }
fn default_tracing_flame_output_path() -> String { "./tracing.folded".to_owned() }
fn default_trusted_servers() -> Vec<OwnedServerName> {
vec![OwnedServerName::try_from("matrix.org").unwrap()]
fn default_trusted_servers() -> Vec<TrustedServer> {
// TODO(nex): Once we can be a notary, add maintainer(?) homeservers here.
// Rationale: Users are already running our code, arguably that's a higher
// level of trust than is assigned to notaries in the first place.
// We should still remind everyone that notaries are evil and out to get you
// and to replace this list with servers they actually trust.
let trusted_servers = vec![ruma::owned_server_name!("matrix.org")];
trusted_servers
.into_iter()
.map(TrustedServer::Name)
.collect()
}
/// do debug logging by default for debug builds
@@ -3210,8 +3322,6 @@ fn parallelism_scaled_u32(val: u32) -> u32 {
fn parallelism_scaled(val: usize) -> usize { val.saturating_mul(sys::available_parallelism()) }
fn default_trusted_server_batch_size() -> usize { 256 }
fn default_db_pool_workers() -> usize {
sys::available_parallelism()
.saturating_mul(4)
+1 -1
View File
@@ -100,7 +100,7 @@ macro_rules! err {
};
($variant:ident($($args:tt)+)) => {
$crate::error::Error::$variant($crate::format_maybe!($($args)+))
$crate::error::Error::$variant($crate::format_maybe!($($args)+), /* Cow<'static, str> */)
};
($level:ident!($($args:tt)+)) => {{
+1
View File
@@ -23,6 +23,7 @@ pub use self::{
id::{ShortId, *},
partial::PartialPdu,
raw_id::*,
unsigned::UnsignedContext,
};
use super::{Event, StateKey};
use crate::Result;
+11 -9
View File
@@ -1,10 +1,14 @@
use ruma::{RoomVersionId, canonical_json::redact_content_in_place};
use serde_json::{Value as JsonValue, json, value::to_raw_value};
use serde_json::{json, value::to_raw_value};
use crate::{Err, Result, err};
impl super::Pdu {
pub fn redact(&mut self, room_version_id: &RoomVersionId, reason: JsonValue) -> Result {
pub fn redact(
&mut self,
room_version_id: &RoomVersionId,
redacted_because: Option<&ruma::EventId>,
) -> Result {
let Some(rules) = room_version_id.rules() else {
return Err!("Cannot redact event for unknown room version {room_version_id}");
};
@@ -20,15 +24,13 @@ impl super::Pdu {
redact_content_in_place(&mut content, &rules.redaction, self.kind.to_string());
let reason = serde_json::to_value(reason).expect("Failed to preserialize reason");
let redacted_because = json!({
"redacted_because": reason,
});
self.unsigned = to_raw_value(&redacted_because)
if let Some(event_id) = redacted_because {
self.unsigned = to_raw_value(&json!({
"org.continuwuity.redacted_by": event_id,
}))
.expect("Failed to serialize unsigned")
.into();
}
self.content = to_raw_value(&content).expect("Failed to serialize content");
+4 -5
View File
@@ -64,7 +64,7 @@ fn nonsense_sticky_object_does_not_reject_the_pdu() {
#[test]
fn redaction_removes_stickiness() {
let mut pdu = sticky_pdu(&serde_json::json!({ "duration_ms": 300_000 }));
pdu.redact(&ruma::RoomVersionId::V11, serde_json::json!({}))
pdu.redact(&ruma::RoomVersionId::V11, None)
.expect("redaction succeeds");
assert!(pdu.sticky.is_none());
@@ -74,7 +74,7 @@ fn redaction_removes_stickiness() {
fn sticky_ttl_is_added_to_unsigned() {
let mut pdu = sticky_pdu(&serde_json::json!({ "duration_ms": 300_000 }));
pdu.origin_server_ts = ruma::MilliSecondsSinceUnixEpoch::now().get();
pdu.add_sticky_duration_ttl().expect("ttl is added");
pdu.set_unsigned(None).unwrap();
let unsigned: serde_json::Value =
serde_json::from_str(pdu.unsigned.as_deref().expect("unsigned is set").get())
@@ -88,8 +88,7 @@ fn sticky_ttl_is_added_to_unsigned() {
#[test]
fn no_ttl_for_events_that_are_not_sticky() {
let mut pdu = sticky_pdu(&serde_json::json!({ "duration_ms": "nonsense" }));
pdu.add_sticky_duration_ttl().expect("ttl is skipped");
let pdu = sticky_pdu(&serde_json::json!({ "duration_ms": "nonsense" }));
assert!(pdu.unsigned.is_none());
}
@@ -100,7 +99,7 @@ fn set_unsigned_adds_the_sticky_ttl() {
// section
let mut pdu = sticky_pdu(&serde_json::json!({ "duration_ms": 300_000 }));
pdu.origin_server_ts = ruma::MilliSecondsSinceUnixEpoch::now().get();
pdu.set_unsigned(None);
pdu.set_unsigned(None).unwrap();
let unsigned: serde_json::Value =
serde_json::from_str(pdu.unsigned.as_deref().expect("unsigned is set").get())
+78 -99
View File
@@ -1,112 +1,91 @@
use std::{borrow::Borrow, collections::BTreeMap};
use std::collections::BTreeMap;
use ruma::MilliSecondsSinceUnixEpoch;
use serde_json::value::{RawValue as RawJsonValue, Value as JsonValue, to_raw_value};
use ruma::{
MilliSecondsSinceUnixEpoch, UserId,
events::{AnyTimelineEvent, room::member::MembershipState},
serde::Raw,
};
use serde_json::value::{RawValue as RawJsonValue, to_raw_value};
use super::{Pdu, sticky};
use crate::{Result, err, result::LogErr};
use crate::Event;
type Unsigned = BTreeMap<String, Box<RawJsonValue>>;
#[derive(Default, Clone)]
pub struct UnsignedContext<'a> {
pub user_id: Option<&'a UserId>,
pub membership: Option<MembershipState>,
pub prev_content: Option<Box<serde_json::value::RawValue>>,
pub redacted_because: Option<Raw<AnyTimelineEvent>>,
}
impl Pdu {
/// Set the `unsigned` field of the PDU using only information in the PDU.
/// Some unsigned data is already set within the database (eg. prev events,
/// threads). Once this is done, other data must be calculated from the
/// database (eg. relations) This is for server-to-client events.
/// Backfill handles this itself.
pub fn set_unsigned(&mut self, user_id: Option<&ruma::UserId>) {
if Some(self.sender.borrow()) != user_id {
self.remove_transaction_id().log_err().ok();
/// Sets the `unsigned` field of the PDU using a mix of the provided
/// optional data, and information available in the PDU itself. The
/// `unsigned` field will always be inserted if it is not attached to the
/// PDU, however no operations will be performed if `unsigned` is present
/// but malformed and unparsable.
///
/// `age` is always inserted and is calculated based on the time of the
/// function call. `membership`, `prev_content`, and `redacted_by` are
/// inserted if not `None`. If `user_id` is `None`, `transaction_id` will be
/// removed. The sticky event TTL field is always inserted and calculated
/// (per `age`) if the PDU is sticky.
///
/// This function panics if any value cannot be serialised, which should not
/// happen provided `membership`, `prev_content`, and `redacted_because` are
/// well-formed.
pub fn set_unsigned(
&mut self,
context: Option<UnsignedContext<'_>>,
) -> crate::Result<Unsigned> {
// See: https://spec.matrix.org/v1.19/client-server-api/#definition-clientevent_unsigneddata
let mut unsigned = self
.unsigned()
.map_or_else(|| Ok(Unsigned::new()), |u| serde_json::from_str::<Unsigned>(u.get()))?;
let now = MilliSecondsSinceUnixEpoch::now().get();
let now_i: ruma::Int = now.into();
unsigned.insert(
"age".to_owned(),
to_raw_value(&now_i.saturating_sub(self.origin_server_ts.into()))?,
);
if let Some(context) = context {
if let Some(membership) = context.membership {
unsigned.insert("membership".to_owned(), to_raw_value(&membership)?);
}
if let Some(prev_content) = context.prev_content {
unsigned.insert("prev_content".to_owned(), to_raw_value(&prev_content)?);
}
if let Some(redacted_because) = context.redacted_because {
unsigned.remove("org.continuwuity.redacted_by");
unsigned.insert("redacted_because".to_owned(), to_raw_value(&redacted_because)?);
}
// replaces_state is embedded and always included.
if context.user_id.is_none_or(|u| u != self.sender()) {
unsigned.remove("transaction_id");
}
} else {
unsigned.remove("transaction_id");
}
self.add_age().log_err().ok();
self.add_sticky_duration_ttl().log_err().ok();
}
pub fn remove_transaction_id(&mut self) -> Result {
use BTreeMap as Map;
let Some(unsigned) = &self.unsigned else {
return Ok(());
};
let mut unsigned: Map<&str, Box<RawJsonValue>> = serde_json::from_str(unsigned.get())
.map_err(|e| err!(Database("Invalid unsigned in pdu event: {e}")))?;
unsigned.remove("transaction_id");
self.unsigned = to_raw_value(&unsigned)
.map(Some)
.expect("unsigned is valid");
Ok(())
}
pub fn add_age(&mut self) -> Result {
use BTreeMap as Map;
let mut unsigned: Map<&str, Box<RawJsonValue>> = self
.unsigned
.as_deref()
.map(RawJsonValue::get)
.map_or_else(|| Ok(Map::new()), serde_json::from_str)
.map_err(|e| err!(Database("Invalid unsigned in pdu event: {e}")))?;
// deliberately allowing for the possibility of negative age
let now: i128 = MilliSecondsSinceUnixEpoch::now().get().into();
let then: i128 = self.origin_server_ts.into();
let this_age = now.saturating_sub(then);
unsigned.insert("age", to_raw_value(&this_age)?);
self.unsigned = Some(to_raw_value(&unsigned)?);
Ok(())
}
pub fn add_sticky_duration_ttl(&mut self) -> Result {
use BTreeMap as Map;
let now = u64::from(MilliSecondsSinceUnixEpoch::now().get());
let Some(expires_at) = self
if let Some(expires_at) = self
.sticky
.as_deref()
.and_then(|sticky| sticky::expires_at(self.origin_server_ts, sticky, now))
else {
return Ok(());
};
let mut unsigned: Map<&str, Box<RawJsonValue>> = self
.unsigned
.as_deref()
.map(RawJsonValue::get)
.map_or_else(|| Ok(Map::new()), serde_json::from_str)
.map_err(|e| err!(Database("Invalid unsigned in pdu event: {e}")))?;
unsigned.insert(sticky::TTL_UNSIGNED_KEY, to_raw_value(&expires_at.saturating_sub(now))?);
self.unsigned = Some(to_raw_value(&unsigned)?);
Ok(())
}
pub fn add_relation(&mut self, name: &str, pdu: Option<&Self>) -> Result {
use serde_json::Map;
let mut unsigned: Map<String, JsonValue> = self
.unsigned
.as_deref()
.map(RawJsonValue::get)
.map_or_else(|| Ok(Map::new()), serde_json::from_str)
.map_err(|e| err!(Database("Invalid unsigned in pdu event: {e}")))?;
let pdu = pdu
.map(serde_json::to_value)
.transpose()?
.unwrap_or_else(|| JsonValue::Object(Map::new()));
unsigned
.entry("m.relations")
.or_insert(JsonValue::Object(Map::new()))
.as_object_mut()
.map(|object| object.insert(name.to_owned(), pdu));
.and_then(|sticky| sticky::expires_at(self.origin_server_ts, sticky, u64::from(now)))
{
unsigned.insert(
sticky::TTL_UNSIGNED_KEY.to_owned(),
to_raw_value(&expires_at.saturating_sub(u64::from(now)))?,
);
}
self.unsigned = Some(to_raw_value(&unsigned)?);
Ok(())
Ok(unsigned)
}
}
+250 -69
View File
@@ -5,17 +5,21 @@ use futures::{
future::{OptionFuture, join, join3},
};
use ruma::{
Int, OwnedUserId, RoomVersionId, UserId,
AnyKeyName, CanonicalJsonObject, Int, OwnedUserId, RoomVersionId, SigningKeyId, UserId,
events::room::{
join_rules::{JoinRule, RoomJoinRulesEventContent},
member::{MembershipState, ThirdPartyInvite},
member::MembershipState,
power_levels::{RoomPowerLevelsEventContent, UserPowerLevel},
third_party_invite::PublicKey,
},
int,
room_version_rules::{RoomIdFormatVersion, RoomVersionRules},
serde::Raw,
serde::{Raw, base64, base64::Standard},
signatures,
signatures::to_canonical_json_string_for_signing,
third_party_invite::IdentityServerBase64PublicKey,
};
use serde::Deserialize;
use serde::{Deserialize, Serialize};
use serde_json::{from_str as from_json_str, value::RawValue as RawJsonValue};
use super::{
@@ -25,7 +29,7 @@ use super::{
deserialize_power_levels_content_redact,
},
};
use crate::{debug, error, trace, warn};
use crate::{debug, err, error, trace, warn};
// FIXME: field extracting could be bundled for `content`
#[derive(Deserialize)]
@@ -48,6 +52,40 @@ struct RoomCreateContentFields {
federate: bool,
}
// Workaround for https://github.com/matrix-org/matrix-spec/issues/2471
#[derive(Deserialize, Debug)]
struct RoomThirdPartyInviteContentFields {
public_key: Option<IdentityServerBase64PublicKey>,
public_keys: Option<Vec<PublicKey>>,
}
// same here sort of (maintainers see https://forgejo.ellis.link/continuwuation/continuwuity-sec/issues/41)
#[derive(Deserialize, Debug)]
struct MemberThirdPartyInviteSignedContent {
signed: ThirdPartyInviteSignedObject,
}
#[derive(Serialize, Deserialize, Debug)]
struct ThirdPartyInviteSignedObject(CanonicalJsonObject);
impl ThirdPartyInviteSignedObject {
#[inline]
fn mxid(&self) -> Option<OwnedUserId> {
self.0
.get("mxid")?
.as_str()
.and_then(|raw| UserId::parse(raw).ok())
}
#[inline]
fn token(&self) -> Option<&str> { self.0.get("token")?.as_str() }
#[inline]
fn signatures(&self) -> Option<CanonicalJsonObject> {
self.0.get("signatures")?.as_object().map(ToOwned::to_owned)
}
}
/// For the given event `kind` what are the relevant auth events that are needed
/// to authenticate this `content`.
///
@@ -61,7 +99,7 @@ pub fn auth_types_for_event(
state_key: Option<&str>,
content: &RawJsonValue,
room_version: &RoomVersionRules,
) -> serde_json::Result<Vec<(StateEventType, StateKey)>> {
) -> crate::Result<Vec<(StateEventType, StateKey)>> {
if kind == &TimelineEventType::RoomCreate {
return Ok(vec![]);
}
@@ -84,7 +122,7 @@ pub fn auth_types_for_event(
#[allow(unused)]
struct RoomMemberContentFields {
membership: Option<Raw<MembershipState>>,
third_party_invite: Option<Raw<ThirdPartyInvite>>,
third_party_invite: Option<Raw<MemberThirdPartyInviteSignedContent>>,
join_authorised_via_users_server: Option<Raw<OwnedUserId>>,
}
@@ -116,16 +154,24 @@ pub fn auth_types_for_event(
auth_types.push(key);
}
// TODO: restore this once 3pid support isn't broken
// if membership == MembershipState::Invite {
// if let Some(Ok(t_id)) = content.third_party_invite.map(|t|
// t.deserialize()) { let key =
// (StateEventType::RoomThirdPartyInvite,
// t_id.signed.token.into()); if !auth_types.contains(&
// key) { auth_types.push(key);
// }
// }
// }
if membership == MembershipState::Invite {
if let Some(Ok(t_id)) = content.third_party_invite.map(|t| t.deserialize()) {
let key = (
StateEventType::RoomThirdPartyInvite,
t_id.signed
.token()
.ok_or_else(|| {
err!(Request(BadJson(
"Third party invite content missing a token"
)))
})?
.into(),
);
if !auth_types.contains(&key) {
auth_types.push(key);
}
}
}
}
}
}
@@ -179,12 +225,11 @@ fn get_user_power_level(
pub async fn auth_check<E, F, Fut>(
room_version: &RoomVersionRules,
incoming_event: &E,
current_third_party_invite: Option<&E>,
fetch_state: F,
create_event: &E,
) -> Result<bool, Error>
where
F: Fn(&StateEventType, &str) -> Fut + Send,
F: Fn(&StateEventType, &str) -> Fut + Send + Copy,
Fut: Future<Output = Option<E>> + Send,
E: Event + Send + Sync,
for<'a> &'a E: Event + Send,
@@ -444,13 +489,15 @@ where
sender,
sender_member_event.as_ref(),
incoming_event,
current_third_party_invite,
fetch_state,
power_levels_event.as_ref(),
join_rules_event.as_ref(),
user_for_join_auth.as_deref(),
&user_for_join_auth_membership,
&room_create_event,
)? {
)
.await?
{
return Ok(false);
}
@@ -639,14 +686,14 @@ where
/// event and the current State.
#[allow(clippy::too_many_arguments)]
#[allow(clippy::cognitive_complexity)]
fn valid_membership_change<E>(
async fn valid_membership_change<E, F, Fut>(
room_version: &RoomVersionRules,
target_user: &UserId,
target_user_membership_event: Option<&E>,
sender: &UserId,
sender_membership_event: Option<&E>,
current_event: &E,
current_third_party_invite: Option<&E>,
fetch_state: F,
power_levels_event: Option<&E>,
join_rules_event: Option<&E>,
user_for_join_auth: Option<&UserId>,
@@ -655,11 +702,13 @@ fn valid_membership_change<E>(
) -> Result<bool>
where
E: Event + Send + Sync,
F: Fn(&StateEventType, &str) -> Fut + Send + Copy,
Fut: Future<Output = Option<E>> + Send,
for<'a> &'a E: Event + Send,
{
#[derive(Deserialize)]
struct GetThirdPartyInvite {
third_party_invite: Option<Raw<ThirdPartyInvite>>,
third_party_invite: Option<Raw<MemberThirdPartyInviteSignedContent>>,
}
let create_content = from_json_str::<RoomCreateContentFields>(create_room.content().get())?;
let creator = if room_version.authorization.use_room_create_sender {
@@ -901,22 +950,18 @@ where
| MembershipState::Invite => {
// If content has third_party_invite key
trace!("starting target_membership=invite check");
match third_party_invite.and_then(|i| i.deserialize().ok()) {
| Some(tp_id) =>
match third_party_invite {
| Some(raw_tp_id) =>
if target_user_current_membership == MembershipState::Ban {
warn!(?target_user_membership_event_id, "Can't invite banned user");
false
} else {
let allow = verify_third_party_invite(
Some(target_user),
sender,
&tp_id,
current_third_party_invite,
);
if !allow {
warn!("Third party invite invalid");
}
allow
let Ok(tp_id) = raw_tp_id.deserialize() else {
warn!("Invite third party content is invalid");
return Ok(false);
};
verify_third_party_invite(Some(target_user), sender, &tp_id, fetch_state)
.await
},
| _ =>
if !sender_is_joined {
@@ -1120,9 +1165,6 @@ fn can_send_event(
ple: Option<&impl Event>,
user_level: UserPowerLevel,
) -> bool {
if user_level == UserPowerLevel::Infinite {
return true;
}
let event_type_power_level = get_send_level(event.event_type(), event.state_key(), ple);
debug!(
@@ -1137,8 +1179,9 @@ fn can_send_event(
return false;
}
if event.state_key().is_some_and(|k| k.starts_with('@'))
&& event.state_key() != Some(event.sender().as_str())
if event
.state_key()
.is_some_and(|k| k.starts_with('@') && k != event.sender().as_str())
{
warn!(
?user_level,
@@ -1437,14 +1480,133 @@ fn get_send_level(
.unwrap_or_else(|| if state_key.is_some() { int!(50) } else { int!(0) })
}
fn verify_third_party_invite(
_target_user: Option<&UserId>,
_sender: &UserId,
_tp_id: &ThirdPartyInvite,
_current_third_party_invite: Option<&impl Event>,
) -> bool {
// TODO: implement proper verification here
true
async fn verify_third_party_invite<F, Fut, E>(
target_user: Option<&UserId>,
sender: &UserId,
tp_id: &MemberThirdPartyInviteSignedContent,
fetch_state: F,
) -> bool
where
F: Fn(&StateEventType, &str) -> Fut + Send,
Fut: Future<Output = Option<E>> + Send,
E: Event + Send + Sync,
{
// If content.third_party_invite does not have a signed property, reject.
// ^ This would be a deserialization error before this function call, which
// equates to a rejection
let signed = &tp_id.signed;
// If signed does not have mxid and token properties, reject.
let Some(mxid) = signed.mxid() else {
warn!("Invite third party signed content is missing or has invalid `mxid` property");
return false;
};
let Some(token) = signed.token() else {
warn!("Invite third party signed content is missing or has invalid `token` property");
return false;
};
if let Err(e) = mxid.validate_historical() {
warn!(error=?e, ?mxid, "Invite third party signed content has invalid `mxid` property");
return false;
}
// If mxid does not match state_key, reject.
// N.B. target_user is the state_key
if target_user.is_none_or(|u| u != mxid) {
warn!(?target_user, %mxid, "Invite third party mxid does not match membership state key");
return false;
}
// If there is no m.room.third_party_invite event in the current room state
// with state_key matching token, reject.
let Some(current_third_party_invite) =
fetch_state(&StateEventType::RoomThirdPartyInvite, token).await
else {
warn!("No third party invite associated with the invite's third party invite found");
return false;
};
let Ok(tpi_content) = current_third_party_invite
.get_content::<RoomThirdPartyInviteContentFields>()
.inspect_err(
|e| warn!(error=?e, "Third party invite event content is invalid (cannot deserialize)"),
)
else {
return false;
};
if current_third_party_invite.state_key() != Some(token) {
warn!(
state_key=?current_third_party_invite.state_key(),
%token,
"Third party invite state key mismatch with signed invite token"
);
return false;
}
// If sender does not match sender of the m.room.third_party_invite, reject.
if sender != current_third_party_invite.sender() {
warn!(
?sender,
third_party_invite_sender=%current_third_party_invite.sender(),
"Third party invite sender does not match invite membership sender"
);
return false;
}
// If any signature in signed matches any public key in the
// m.room.third_party_invite event, allow. The public keys are in content of
// m.room.third_party_invite as:
//
// A single public key in the public_key property.
// A list of public keys in the public_keys property.
// NOTE: Technically the two below operations shouldn't be fallible at this
// point in execution, but if we can't de/serialize them now, we won't be
// able to in the future, so rejecting the event is the right way to go.
// We log an error though since this is unexpected.
let Ok(signed_json) = to_canonical_json_string_for_signing(&signed.0).inspect_err(
|e| error!(error=?e, "Canonical signed object cannot be cast to a signing string"),
) else {
return false;
};
let signatures = signed
.signatures()
.unwrap_or_default() // Empty sigs object will never pass so this is fine
.values()
.filter_map(|signatures| signatures.as_object())
.flat_map(|signatures| {
signatures.iter().map(|(key, sig)| {
let key_id = SigningKeyId::<AnyKeyName>::parse(key).ok()?;
base64::Base64::<Standard>::parse(sig.as_str()?)
.map(|parsed| (key_id.algorithm(), parsed))
.ok()
})
})
.flatten()
.collect::<Vec<_>>();
for public_key_bytes in tpi_content
.public_key
.into_iter()
.chain(
tpi_content
.public_keys
.unwrap_or_default()
.iter()
.map(|pk| pk.public_key.clone()),
)
.filter_map(|pk| pk.decode().ok())
{
for (algorithm, signature) in &signatures {
if signatures::verify_canonical_json_bytes(
algorithm,
&public_key_bytes,
signature.as_bytes(),
signed_json.as_bytes(),
)
.is_ok()
{
return true;
}
}
}
warn!("Invite via third party was not signed by any associated public keys");
false
}
#[cfg(test)]
@@ -1463,7 +1625,7 @@ mod tests {
use serde_json::value::to_raw_value as to_raw_json_value;
use crate::{
matrix::{Event, EventTypeExt, Pdu as PduEvent},
matrix::{Event, EventTypeExt},
state_res::{
StateMap,
event_auth::valid_membership_change,
@@ -1474,8 +1636,8 @@ mod tests {
},
};
#[test]
fn test_ban_pass() {
#[tokio::test]
async fn test_ban_pass() {
let _ = tracing::subscriber::set_default(
tracing_subscriber::fmt().with_test_writer().finish(),
);
@@ -1497,6 +1659,8 @@ mod tests {
);
let fetch_state = |ty, key| auth_events.get(&(ty, key)).cloned();
let async_fetch_state =
|t: &StateEventType, k: &str| std::future::ready(fetch_state(t.to_owned(), k.into()));
let target_user = charlie();
let sender = alice();
@@ -1508,19 +1672,20 @@ mod tests {
sender,
fetch_state(StateEventType::RoomMember, sender.as_str().into()).as_ref(),
&requester,
None::<&PduEvent>,
&async_fetch_state,
fetch_state(StateEventType::RoomPowerLevels, "".into()).as_ref(),
fetch_state(StateEventType::RoomJoinRules, "".into()).as_ref(),
None,
&MembershipState::Leave,
&fetch_state(StateEventType::RoomCreate, "".into()).unwrap(),
)
.await
.unwrap()
);
}
#[test]
fn test_join_non_creator() {
#[tokio::test]
async fn test_join_non_creator() {
let _ = tracing::subscriber::set_default(
tracing_subscriber::fmt().with_test_writer().finish(),
);
@@ -1542,6 +1707,8 @@ mod tests {
);
let fetch_state = |ty, key| auth_events.get(&(ty, key)).cloned();
let async_fetch_state =
|t: &StateEventType, k: &str| std::future::ready(fetch_state(t.to_owned(), k.into()));
let target_user = charlie();
let sender = charlie();
@@ -1553,19 +1720,20 @@ mod tests {
sender,
fetch_state(StateEventType::RoomMember, sender.as_str().into()).as_ref(),
&requester,
None::<&PduEvent>,
&async_fetch_state,
fetch_state(StateEventType::RoomPowerLevels, "".into()).as_ref(),
fetch_state(StateEventType::RoomJoinRules, "".into()).as_ref(),
None,
&MembershipState::Leave,
&fetch_state(StateEventType::RoomCreate, "".into()).unwrap(),
)
.await
.unwrap()
);
}
#[test]
fn test_join_creator() {
#[tokio::test]
async fn test_join_creator() {
let _ = tracing::subscriber::set_default(
tracing_subscriber::fmt().with_test_writer().finish(),
);
@@ -1587,6 +1755,8 @@ mod tests {
);
let fetch_state = |ty, key| auth_events.get(&(ty, key)).cloned();
let async_fetch_state =
|t: &StateEventType, k: &str| std::future::ready(fetch_state(t.to_owned(), k.into()));
let target_user = alice();
let sender = alice();
@@ -1598,19 +1768,20 @@ mod tests {
sender,
fetch_state(StateEventType::RoomMember, sender.as_str().into()).as_ref(),
&requester,
None::<&PduEvent>,
&async_fetch_state,
fetch_state(StateEventType::RoomPowerLevels, "".into()).as_ref(),
fetch_state(StateEventType::RoomJoinRules, "".into()).as_ref(),
None,
&MembershipState::Leave,
&fetch_state(StateEventType::RoomCreate, "".into()).unwrap(),
)
.await
.unwrap()
);
}
#[test]
fn test_ban_fail() {
#[tokio::test]
async fn test_ban_fail() {
let _ = tracing::subscriber::set_default(
tracing_subscriber::fmt().with_test_writer().finish(),
);
@@ -1632,6 +1803,8 @@ mod tests {
);
let fetch_state = |ty, key| auth_events.get(&(ty, key)).cloned();
let async_fetch_state =
|t: &StateEventType, k: &str| std::future::ready(fetch_state(t.to_owned(), k.into()));
let target_user = alice();
let sender = charlie();
@@ -1643,19 +1816,20 @@ mod tests {
sender,
fetch_state(StateEventType::RoomMember, sender.as_str().into()).as_ref(),
&requester,
None::<&PduEvent>,
&async_fetch_state,
fetch_state(StateEventType::RoomPowerLevels, "".into()).as_ref(),
fetch_state(StateEventType::RoomJoinRules, "".into()).as_ref(),
None,
&MembershipState::Leave,
&fetch_state(StateEventType::RoomCreate, "".into()).unwrap(),
)
.await
.unwrap()
);
}
#[test]
fn test_restricted_join_rule() {
#[tokio::test]
async fn test_restricted_join_rule() {
let _ = tracing::subscriber::set_default(
tracing_subscriber::fmt().with_test_writer().finish(),
);
@@ -1694,6 +1868,8 @@ mod tests {
);
let fetch_state = |ty, key| auth_events.get(&(ty, key)).cloned();
let async_fetch_state =
|t: &StateEventType, k: &str| std::future::ready(fetch_state(t.to_owned(), k.into()));
let target_user = ella();
let sender = ella();
@@ -1705,13 +1881,14 @@ mod tests {
sender,
fetch_state(StateEventType::RoomMember, sender.as_str().into()).as_ref(),
&requester,
None::<&PduEvent>,
&async_fetch_state,
fetch_state(StateEventType::RoomPowerLevels, "".into()).as_ref(),
fetch_state(StateEventType::RoomJoinRules, "".into()).as_ref(),
Some(alice()),
&MembershipState::Join,
&fetch_state(StateEventType::RoomCreate, "".into()).unwrap(),
)
.await
.unwrap()
);
@@ -1723,19 +1900,20 @@ mod tests {
sender,
fetch_state(StateEventType::RoomMember, sender.as_str().into()).as_ref(),
&requester,
None::<&PduEvent>,
&async_fetch_state,
fetch_state(StateEventType::RoomPowerLevels, "".into()).as_ref(),
fetch_state(StateEventType::RoomJoinRules, "".into()).as_ref(),
Some(ella()),
&MembershipState::Leave,
&fetch_state(StateEventType::RoomCreate, "".into()).unwrap(),
)
.await
.unwrap()
);
}
#[test]
fn test_knock() {
#[tokio::test]
async fn test_knock() {
let _ = tracing::subscriber::set_default(
tracing_subscriber::fmt().with_test_writer().finish(),
);
@@ -1766,6 +1944,8 @@ mod tests {
);
let fetch_state = |ty, key| auth_events.get(&(ty, key)).cloned();
let async_fetch_state =
|t: &StateEventType, k: &str| std::future::ready(fetch_state(t.to_owned(), k.into()));
let target_user = ella();
let sender = ella();
@@ -1777,13 +1957,14 @@ mod tests {
sender,
fetch_state(StateEventType::RoomMember, sender.as_str().into()).as_ref(),
&requester,
None::<&PduEvent>,
&async_fetch_state,
fetch_state(StateEventType::RoomPowerLevels, "".into()).as_ref(),
fetch_state(StateEventType::RoomJoinRules, "".into()).as_ref(),
None,
&MembershipState::Leave,
&fetch_state(StateEventType::RoomCreate, "".into()).unwrap(),
)
.await
.unwrap()
);
}
+12 -12
View File
@@ -757,7 +757,8 @@ where
Some(state_key),
event.content(),
room_version,
)?;
)
.map_err(|e| Error::InvalidPdu(e.to_string()))?;
trace!(list = ?auth_types, event_id = event.event_id().as_str(), "auth types for event");
let mut auth_state = StateMap::new();
@@ -809,11 +810,17 @@ where
.await;
trace!(map = ?auth_state.keys().collect::<Vec<_>>(), event_id = event.event_id().as_str(), "auth state for event");
// If this is not a create event, we must have the room create event in
// the auth state by now.
if *event.event_type() != StateEventType::RoomCreate.into()
|| event.state_key().is_none_or(|k| !k.is_empty())
if *event.event_type() == StateEventType::RoomCreate.into()
&& event.state_key().is_some_and(str::is_empty)
{
// Inject the create event so that the call to auth_check doesn't
// explode. It's not used otherwise.
auth_state
.entry(StateEventType::RoomCreate.with_state_key(""))
.or_insert_with(|| event.clone());
} else {
// If this is not a create event, we must have the room create event
// in the auth state by now.
if !auth_state.contains_key(&StateEventType::RoomCreate.with_state_key("")) {
return Err(Error::NotFound(format!(
"m.room.create did not appear in the auth state for {}",
@@ -824,12 +831,6 @@ where
debug!(event_id = event.event_id().as_str(), "Running auth checks");
// The key for this is (eventType + a state_key of the signed token not
// sender) so search for it
let current_third_party = auth_state.iter().find_map(|(_, pdu)| {
(*pdu.event_type() == TimelineEventType::RoomThirdPartyInvite).then_some(pdu)
});
let fetch_state = |ty: &StateEventType, key: &str| {
future::ready(
auth_state
@@ -841,7 +842,6 @@ where
let auth_result = auth_check(
room_version,
&event,
current_third_party,
fetch_state,
&fetch_state(&StateEventType::RoomCreate, "")
.await
+2
View File
@@ -44,5 +44,7 @@ pub fn unstable_features() -> BTreeMap<String, bool> {
("org.matrix.msc4155".to_owned(), true),
// profile change propagation (https://github.com/matrix-org/matrix-spec-proposals/pull/4466)
("computer.gingershaped.msc4466".to_owned(), true),
// web push (https://github.com/matrix-org/matrix-spec-proposals/pull/4174)
("org.matrix.msc4174".to_owned(), true),
])
}
+9 -1
View File
@@ -267,7 +267,15 @@ pub(super) static MAPS: &[Descriptor] = &[
..descriptor::RANDOM_SMALL
},
Descriptor {
name: "server_signingkeys",
name: "senderkey_webpushbackoff",
..descriptor::RANDOM_SMALL
},
Descriptor {
name: "senderkey_webpushstate",
..descriptor::RANDOM_SMALL
},
Descriptor {
name: "servernamekeyid_response",
..descriptor::RANDOM
},
Descriptor {
+1
View File
@@ -21,6 +21,7 @@ unstable-msc4203 = []
[dependencies]
assign.workspace = true
http.workspace = true
ruma.workspace = true
serde = { workspace = true }
serde_json = { workspace = true }
+1
View File
@@ -5,3 +5,4 @@ pub mod draupnir_antispam;
pub mod invite_permission_config;
pub mod meowlnir_antispam;
pub mod policy;
pub mod pushers;
+36
View File
@@ -0,0 +1,36 @@
pub mod unstable {
use ruma::{
api::{auth_scheme::AccessToken, error::Error as MatrixError, request, response},
metadata,
};
metadata! {
method: POST,
rate_limited: false,
authentication: AccessToken,
history: {
unstable => "/_matrix/client/unstable/org.matrix.msc4174/pushers/ack",
}
}
#[request(error = MatrixError)]
pub struct Request {
pub app_id: String,
pub ack_token: String,
}
#[response(error = MatrixError)]
#[derive(Default)]
#[allow(clippy::empty_structs_with_brackets, reason = "required by #[response]")]
pub struct Response {}
impl Request {
#[must_use]
pub fn new(app_id: String, ack_token: String) -> Self { Self { app_id, ack_token } }
}
impl Response {
#[must_use]
pub fn new() -> Self { Self {} }
}
}
+69
View File
@@ -0,0 +1,69 @@
pub mod v3 {
use ruma::{
api::{auth_scheme::AccessToken, error::Error as MatrixError, request, response},
metadata,
serde::from_raw_json_value,
};
use serde::{Deserialize, Serialize, de};
use serde_json::value::RawValue as RawJsonValue;
use crate::pushers::Pusher;
metadata! {
method: GET,
rate_limited: false,
authentication: AccessToken,
history: {
1.0 => "/_matrix/client/r0/pushers",
1.1 => "/_matrix/client/v3/pushers",
}
}
#[request(error = MatrixError)]
#[derive(Default)]
#[allow(clippy::empty_structs_with_brackets, reason = "required by #[request]")]
pub struct Request {}
#[response(error = MatrixError)]
pub struct Response {
pub pushers: Vec<PusherWithActivation>,
}
#[derive(Clone, Debug, Serialize)]
pub struct PusherWithActivation {
#[serde(flatten)]
pub pusher: Pusher,
#[serde(skip_serializing_if = "Option::is_none")]
pub activated: Option<bool>,
}
#[derive(Deserialize)]
struct ActivationDeHelper {
activated: Option<bool>,
}
impl<'de> Deserialize<'de> for PusherWithActivation {
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: de::Deserializer<'de>,
{
let json = Box::<RawJsonValue>::deserialize(deserializer)?;
let ActivationDeHelper { activated } = from_raw_json_value(&json)?;
let pusher = from_raw_json_value(&json)?;
Ok(Self { pusher, activated })
}
}
impl Request {
#[must_use]
pub fn new() -> Self { Self {} }
}
impl Response {
#[must_use]
pub fn new(pushers: Vec<PusherWithActivation>) -> Self { Self { pushers } }
}
}
+87
View File
@@ -0,0 +1,87 @@
pub mod ack;
pub mod get_pushers;
mod pusher_serde;
pub mod set_pusher;
use std::fmt;
pub use ruma::api::client::push::{EmailPusherData, PusherIds};
use ruma::{
push::{HttpPusherData, PushFormat},
serde::JsonObject,
};
use serde::{Deserialize, Serialize};
#[derive(Clone, Debug, Serialize)]
pub struct Pusher {
#[serde(flatten)]
pub ids: PusherIds,
#[serde(flatten)]
pub kind: PusherKind,
pub app_display_name: String,
pub device_display_name: String,
#[serde(skip_serializing_if = "Option::is_none")]
pub profile_tag: Option<String>,
pub lang: String,
}
#[derive(Clone, Debug)]
pub enum PusherKind {
Http(HttpPusherData),
Email(EmailPusherData),
WebPush(WebPushPusherData),
Custom {
kind: String,
data: JsonObject,
},
}
#[derive(Clone, Serialize, Deserialize)]
pub struct WebPushPusherData {
pub url: String,
pub auth: String,
#[serde(skip_serializing_if = "Option::is_none")]
pub format: Option<PushFormat>,
#[serde(flatten, default, skip_serializing_if = "JsonObject::is_empty")]
pub data: JsonObject,
}
impl fmt::Debug for WebPushPusherData {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.debug_struct("WebPushPusherData")
.field("url", &self.url)
.field("auth", &"<redacted>")
.field("format", &self.format)
.field("data", &self.data)
.finish()
}
}
impl PusherKind {
#[must_use]
pub fn as_str(&self) -> &str {
match self {
| Self::Http(_) => "http",
| Self::Email(_) => "email",
| Self::WebPush(_) => "org.matrix.msc4174.webpush",
| Self::Custom { kind, .. } => kind,
}
}
#[must_use]
pub fn as_webpush(&self) -> Option<&WebPushPusherData> {
match self {
| Self::WebPush(data) => Some(data),
| _ => None,
}
}
}
+196
View File
@@ -0,0 +1,196 @@
use ruma::serde::from_raw_json_value;
use serde::{Deserialize, Serialize, de, ser::SerializeStruct};
use serde_json::value::RawValue as RawJsonValue;
use super::{Pusher, PusherIds, PusherKind};
#[derive(Debug, Deserialize)]
struct PusherDeHelper {
#[serde(flatten)]
ids: PusherIds,
app_display_name: String,
device_display_name: String,
profile_tag: Option<String>,
lang: String,
}
impl<'de> Deserialize<'de> for Pusher {
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: de::Deserializer<'de>,
{
let json = Box::<RawJsonValue>::deserialize(deserializer)?;
let PusherDeHelper {
ids,
app_display_name,
device_display_name,
profile_tag,
lang,
} = from_raw_json_value(&json)?;
let kind = from_raw_json_value(&json)?;
Ok(Self {
ids,
kind,
app_display_name,
device_display_name,
profile_tag,
lang,
})
}
}
impl Serialize for PusherKind {
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where
S: serde::Serializer,
{
let mut st = serializer.serialize_struct("PusherKind", 2)?;
st.serialize_field("kind", self.as_str())?;
match self {
| Self::Http(data) => st.serialize_field("data", data)?,
| Self::Email(data) => st.serialize_field("data", data)?,
| Self::WebPush(data) => st.serialize_field("data", data)?,
| Self::Custom { data, .. } => st.serialize_field("data", data)?,
}
st.end()
}
}
#[derive(Debug, Deserialize)]
struct PusherKindDeHelper {
kind: String,
data: Box<RawJsonValue>,
}
impl<'de> Deserialize<'de> for PusherKind {
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: de::Deserializer<'de>,
{
let json = Box::<RawJsonValue>::deserialize(deserializer)?;
let PusherKindDeHelper { kind, data } = from_raw_json_value(&json)?;
match kind.as_str() {
| "http" => from_raw_json_value(&data).map(Self::Http),
| "email" => from_raw_json_value(&data).map(Self::Email),
| "org.matrix.msc4174.webpush" => from_raw_json_value(&data).map(Self::WebPush),
| _ => Ok(Self::Custom { kind, data: from_raw_json_value(&data)? }),
}
}
}
#[cfg(test)]
mod tests {
use serde_json::{from_value as from_json_value, json, to_value as to_json_value};
use super::{Pusher, PusherKind};
#[test]
fn roundtrip_ruma_http_pusher() {
let stored = json!({
"pushkey": "abcdef",
"app_id": "my.matrix.app",
"kind": "http",
"data": {
"url": "https://push.example.org/_matrix/push/v1/notify",
"format": "event_id_only",
"default_payload": { "session_id": "abc" },
},
"app_display_name": "My Matrix App",
"device_display_name": "My Phone",
"profile_tag": "tag",
"lang": "en",
});
let pusher: Pusher = from_json_value(stored.clone()).unwrap();
assert!(matches!(pusher.kind, PusherKind::Http(_)));
assert_eq!(to_json_value(&pusher).unwrap(), stored);
}
#[test]
fn deserialize_legacy_append_field() {
let stored = json!({
"pushkey": "abcdef",
"app_id": "my.matrix.app",
"kind": "email",
"data": {},
"append": true,
"app_display_name": "My Matrix App",
"device_display_name": "My Phone",
"lang": "en",
});
let pusher: Pusher = from_json_value(stored).unwrap();
assert!(matches!(pusher.kind, PusherKind::Email(_)));
assert_eq!(pusher.profile_tag, None);
}
#[test]
fn roundtrip_webpush_pusher() {
let stored = json!({
"pushkey": "BLn9b-VR0ca83knDNZ32dCHGyjJp",
"app_id": "m.webpush",
"kind": "org.matrix.msc4174.webpush",
"data": {
"url": "https://updates.push.services.mozilla.com/wpush/v2/abc",
"auth": "_ordMnz7uTCmrpBTeUV4Bw",
"format": "event_id_only",
"default_payload": { "session_id": "abc" },
},
"app_display_name": "WebPush",
"device_display_name": "Firefox",
"lang": "en",
});
let pusher: Pusher = from_json_value(stored.clone()).unwrap();
let data = pusher.kind.as_webpush().unwrap();
assert_eq!(data.auth, "_ordMnz7uTCmrpBTeUV4Bw");
assert_eq!(data.data.len(), 1);
assert_eq!(to_json_value(&pusher).unwrap(), stored);
}
#[test]
fn roundtrip_pusher_with_activation() {
use crate::pushers::get_pushers::v3::PusherWithActivation;
let wire = json!({
"pushkey": "BLn9b-VR0ca83knDNZ32dCHGyjJp",
"app_id": "m.webpush",
"kind": "org.matrix.msc4174.webpush",
"data": {
"url": "https://updates.push.services.mozilla.com/wpush/v2/abc",
"auth": "_ordMnz7uTCmrpBTeUV4Bw",
},
"app_display_name": "WebPush",
"device_display_name": "Firefox",
"lang": "en",
"activated": true,
});
let pusher: PusherWithActivation = from_json_value(wire.clone()).unwrap();
assert_eq!(pusher.activated, Some(true));
assert_eq!(pusher.pusher.kind.as_str(), "org.matrix.msc4174.webpush");
assert_eq!(to_json_value(&pusher).unwrap(), wire);
}
#[test]
fn roundtrip_unknown_kind() {
let stored = json!({
"pushkey": "abcdef",
"app_id": "my.matrix.app",
"kind": "my.custom.kind",
"data": { "whatever": 1 },
"app_display_name": "App",
"device_display_name": "Device",
"lang": "en",
});
let pusher: Pusher = from_json_value(stored.clone()).unwrap();
assert_eq!(pusher.kind.as_str(), "my.custom.kind");
assert_eq!(to_json_value(&pusher).unwrap(), stored);
}
}
+129
View File
@@ -0,0 +1,129 @@
pub mod v3 {
use ruma::{
JsOption,
api::{
EmptyBody, IncomingResponse, OutgoingResponse,
auth_scheme::AccessToken,
error::{DeserializationError, Error as MatrixError, IntoHttpError},
request,
},
metadata,
serde::from_raw_json_value,
};
use serde::{Deserialize, Serialize, de};
use serde_json::value::RawValue as RawJsonValue;
use crate::pushers::{Pusher, PusherIds};
metadata! {
method: POST,
rate_limited: true,
authentication: AccessToken,
history: {
1.0 => "/_matrix/client/r0/pushers/set",
1.1 => "/_matrix/client/v3/pushers/set",
}
}
#[request(error = MatrixError)]
pub struct Request {
#[ruma_api(body)]
pub action: PusherAction,
}
#[derive(Clone, Debug, Default)]
pub struct Response {
pub needs_activation: bool,
}
impl Request {
#[must_use]
pub fn new(action: PusherAction) -> Self { Self { action } }
}
impl Response {
#[must_use]
pub fn new(needs_activation: bool) -> Self { Self { needs_activation } }
}
#[cfg(feature = "server")]
impl OutgoingResponse for Response {
type Body = EmptyBody<false>;
fn try_into_http_response_inner(
self,
) -> Result<http::Response<Self::Body>, IntoHttpError> {
let status = if self.needs_activation {
http::StatusCode::CREATED
} else {
http::StatusCode::OK
};
http::Response::builder()
.status(status)
.header(http::header::CONTENT_TYPE, ruma::http_headers::APPLICATION_JSON)
.body(EmptyBody)
.map_err(Into::into)
}
}
#[cfg(feature = "client")]
impl IncomingResponse for Response {
type EndpointError = MatrixError;
fn try_from_http_response_inner(
response: http::Response<&[u8]>,
) -> Result<Self, DeserializationError> {
Ok(Self {
needs_activation: response.status() == http::StatusCode::CREATED,
})
}
}
#[derive(Clone, Debug)]
pub enum PusherAction {
Post(Box<Pusher>),
Delete(PusherIds),
}
#[derive(Debug, Deserialize)]
struct PusherActionDeHelper {
kind: JsOption<String>,
}
impl<'de> Deserialize<'de> for PusherAction {
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: de::Deserializer<'de>,
{
let json = Box::<RawJsonValue>::deserialize(deserializer)?;
let PusherActionDeHelper { kind } = from_raw_json_value(&json)?;
match kind {
| JsOption::Some(_) => Ok(Self::Post(from_raw_json_value(&json)?)),
| JsOption::Null => Ok(Self::Delete(from_raw_json_value(&json)?)),
| JsOption::Undefined => Err(de::Error::missing_field("kind")),
}
}
}
impl Serialize for PusherAction {
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where
S: serde::Serializer,
{
match self {
| Self::Post(pusher) => pusher.serialize(serializer),
| Self::Delete(ids) => {
use serde::ser::SerializeStruct;
let mut st = serializer.serialize_struct("PusherAction", 3)?;
st.serialize_field("pushkey", &ids.pushkey)?;
st.serialize_field("app_id", &ids.app_id)?;
st.serialize_field("kind", &None::<&str>)?;
st.end()
},
}
}
}
}
+3
View File
@@ -88,6 +88,7 @@ futures.workspace = true
governor.workspace = true
hickory-resolver.workspace = true
http.workspace = true
httpdate.workspace = true
image.workspace = true
image.optional = true
ipaddress.workspace = true
@@ -96,6 +97,7 @@ log.workspace = true
loole.workspace = true
lru-cache.workspace = true
nonzero_ext.workspace = true
p256.workspace = true
regex.workspace = true
reqwest.workspace = true
assign.workspace = true
@@ -112,6 +114,7 @@ termimad.optional = true
tokio.workspace = true
tracing.workspace = true
url.workspace = true
web-push-native.workspace = true
webpage.workspace = true
webpage.optional = true
recaptcha-verify = { version = "0.2.0", default-features = false }
+1 -1
View File
@@ -399,7 +399,7 @@ impl Service {
let mut stream = admin_users;
while let Some(user_id) = stream.next().await {
generated_admin_list.push(user_id.clone());
generated_admin_list.push(user_id);
}
}
}
+11
View File
@@ -24,6 +24,7 @@ pub struct Service {
pub sender: reqwest::Client,
pub appservice: reqwest::Client,
pub pusher: reqwest::Client,
pub webpush: reqwest::Client,
pub cidr_range_denylist: Vec<IPAddress>,
}
@@ -141,6 +142,16 @@ impl crate::Service for Service {
.redirect(redirect::Policy::limited(2))
.build()?,
// MSC4174 recommends against following redirects, since the destination
// is only checked before the request is sent.
webpush: base(config)?
.connect_timeout(Duration::from_secs(config.pusher_conn_timeout))
.timeout(Duration::from_secs(config.pusher_timeout))
.pool_max_idle_per_host(1)
.pool_idle_timeout(Duration::from_secs(config.pusher_idle_timeout))
.redirect(redirect::Policy::none())
.build()?,
cidr_range_denylist: config
.ip_range_denylist
.iter()
-13
View File
@@ -6,7 +6,6 @@ use conduwuit::{
config::{Config, check},
error,
};
use url::Url;
use crate::registration_tokens::{ValidToken, ValidTokenSource};
@@ -53,18 +52,6 @@ impl Service {
.map(|token| ValidToken { token, source: ValidTokenSource::Config })
}
/// Get the base domain to use for user-facing URLs.
#[must_use]
pub fn get_client_domain(&self) -> Url {
self.well_known.client.clone().unwrap_or_else(|| {
let host = self.server_name.host();
format!("https://{host}")
.as_str()
.try_into()
.expect("server name should be a valid host")
})
}
fn handle_reload(&self) -> Result {
if self.server.config.config_reload_signal {
#[cfg(all(feature = "systemd", target_os = "linux"))]
+2 -2
View File
@@ -14,7 +14,7 @@ use ruma::{
SupportedVersions,
auth_scheme::{AuthScheme, NoAuthentication},
error::Error as RumaError,
federation::authentication::{ServerSignatures, ServerSignaturesInput},
federation::authentication::{ServerSignatures, XMatrixSigningInput},
path_builder::PathBuilder,
},
};
@@ -92,7 +92,7 @@ impl super::Service {
PathBuilder: PathBuilder<Input<'i>: FederationPathBuilderInput>,
> + Send,
{
let authentication = ServerSignaturesInput::new(
let authentication = XMatrixSigningInput::new(
self.services.server.name.clone(),
dest.to_owned(),
self.services.server_keys.keypair(),
+1 -3
View File
@@ -6,7 +6,7 @@ use async_trait::async_trait;
use conduwuit::{Result, Server, SyncRwLock, error, utils::bytes::pretty};
use data::Data;
use regex::RegexSet;
use ruma::{OwnedEventId, OwnedRoomAliasId, OwnedServerName, OwnedUserId, ServerName, UserId};
use ruma::{OwnedEventId, OwnedRoomAliasId, OwnedUserId, ServerName, UserId};
use crate::service;
@@ -106,8 +106,6 @@ impl Service {
self.server.config.allow_announcements_check
}
pub fn trusted_servers(&self) -> &[OwnedServerName] { &self.server.config.trusted_servers }
pub fn turn_password(&self) -> &String { &self.server.config.turn_password }
pub fn turn_ttl(&self) -> u64 { self.server.config.turn_ttl }
+110 -8
View File
@@ -1,30 +1,38 @@
use std::{cmp, collections::HashMap, future::ready};
use std::{
cmp,
collections::{BTreeMap, HashMap},
future::ready,
sync::Arc,
time::Instant,
};
use conduwuit::{
Err, Event, Pdu, Result, debug, debug_info, debug_warn, err, error, info,
Err, Error, Event, Pdu, Result, debug, debug_info, debug_warn, err, error, info,
result::NotFound,
trace,
utils::{
IterStream, ReadyExt,
stream::{TryExpect, TryIgnore},
to_canonical_object,
},
warn,
};
use database::Json;
use database::{Database, Json};
use futures::{FutureExt, StreamExt, TryStreamExt};
use itertools::Itertools;
use ruma::{
OwnedRoomId, OwnedUserId, UserId,
OwnedEventId, OwnedRoomId, OwnedUserId, UserId,
events::{
AnyStrippedStateEvent, GlobalAccountDataEventType, StateEventType,
push_rules::PushRulesEvent,
room::member::{MembershipState, RoomMemberEventContent},
},
push::Ruleset,
serde::Raw,
serde::{Raw, from_raw_json_value},
};
use serde_json::value::to_raw_value;
use crate::{Services, media, rooms::short::ShortStateHash};
use crate::{Services, media, rooms, rooms::short::ShortStateHash};
/// The current schema version.
/// - If database is opened at greater version we reject with error. The
@@ -73,6 +81,8 @@ async fn fresh(services: &Services) -> Result<()> {
db["global"].insert(b"fix_local_invite_state", []);
db["global"].insert(SPLIT_USERID_PASSWORD, []);
db["global"].insert(DROP_ROOMSYNCTOKEN_SHORTSTATEHASH, []);
db["global"].insert(UNEMBED_UNSIGNED_INFO, []);
db["global"].insert(DROP_OLD_SIGNING_KEYS_STORAGE, []);
// Create the admin room and server user on first run
info!("Creating admin room and server user");
@@ -118,7 +128,7 @@ async fn migrate(services: &Services) -> Result<()> {
.map_err(|e| err!("Failed to run SHA256 media migration: {e}"))?;
} else if config.media_startup_check {
info!("Starting media startup integrity check.");
let now = std::time::Instant::now();
let now = Instant::now();
media::migrations::checkup_sha256_media(services)
.await
.map_err(|e| err!("Failed to verify media integrity: {e}"))?;
@@ -255,6 +265,27 @@ async fn migrate(services: &Services) -> Result<()> {
})?;
}
if db["global"].get(UNEMBED_UNSIGNED_INFO).await.is_not_found() {
services.server.runtime().spawn(unembed_unsigned_info(
services.db.clone(),
services.rooms.metadata.clone(),
services.rooms.timeline.clone(),
));
}
if db["global"]
.get(DROP_OLD_SIGNING_KEYS_STORAGE)
.await
.is_not_found()
{
services
.db
.db
.drop_column("server_signingkeys")
.inspect(|()| services.db["global"].insert(DROP_OLD_SIGNING_KEYS_STORAGE, []))
.map_err(|e| err!("Failed to drop server_signingkeys: {e:?}"))?;
}
assert_eq!(
services.globals.db.database_version().await,
DATABASE_VERSION,
@@ -810,7 +841,7 @@ async fn fix_local_invite_state(services: &Services) -> Result {
// if they're a local user on this homeserver
.try_filter(|((user_id, _), _): &KeyVal| ready(services.globals.user_is_local(user_id)))
.and_then(async |((user_id, room_id), stripped_state): KeyVal| Ok::<_,
conduwuit::Error>((user_id.clone(), room_id.clone(), stripped_state.deserialize
Error>((user_id.clone(), room_id.clone(), stripped_state.deserialize
().unwrap_or_else(|e| {
trace!("Failed to deserialize: {:?}", stripped_state.json());
warn!(
@@ -901,3 +932,74 @@ async fn obliterate_roomsynctoken_shortstatehash_with_extreme_prejudice(
Ok(())
}
const UNEMBED_UNSIGNED_INFO: &str = "unembed_unsigned_info";
#[tracing::instrument(name = "unembed_unsigned_info", skip_all)]
async fn unembed_unsigned_info(
db: Arc<Database>,
metadata: Arc<rooms::metadata::Service>,
timeline: Arc<rooms::timeline::Service>,
) -> Result {
type Map = BTreeMap<String, Box<serde_json::value::RawValue>>;
info!(
"Starting background migration. This may take a long time, and may cause your server to \
lag. Please avoid restarting until the migration is complete."
);
let start = Instant::now();
let mut total_migrated = 0_usize;
let mut all_rooms = metadata.iter_ids();
while let Some(room_id) = all_rooms.next().await {
info!("Migrating PDUs in {room_id}");
let mut pdus = std::pin::pin!(timeline.all_pdus(&room_id));
let mut migrated = 0_usize;
while let Some((_, pdu)) = pdus.next().await {
let mut pdu = pdu.clone();
let mut unsigned: Map = pdu
.unsigned()
.and_then(|u| from_raw_json_value::<Map, Error>(u).ok())
.unwrap_or_default();
unsigned.remove("age");
unsigned.remove("membership");
unsigned.remove("prev_content");
unsigned.remove("prev_sender");
if let Some(redacted_because) = unsigned
.remove("redacted_because")
.and_then(|r| from_raw_json_value::<Map, Error>(&r).ok())
{
match redacted_because
.get("event_id")
.and_then(|e| from_raw_json_value::<OwnedEventId, Error>(e).ok())
{
| None => {},
| Some(event_id) => {
unsigned.insert(
"org.continuwuity.redacted_by".to_owned(),
to_raw_value(event_id.as_str()).expect("str must be raw value"),
);
},
}
}
pdu.unsigned = Some(to_raw_value(&unsigned).expect("unsigned must be valid JSON"));
let pdu_id = timeline
.get_pdu_id(pdu.event_id())
.await
.expect("PDU must have an id");
timeline
.replace_pdu(&pdu_id, &to_canonical_object(pdu).expect("PDU must be valid JSON"))
.await
.expect("must be able to replace PDU after migrating unsigned data");
migrated = migrated.saturating_add(1);
}
total_migrated = total_migrated.saturating_add(migrated);
info!(elapsed=?start.elapsed(), "Migrated {migrated} PDUs in {room_id}");
}
db["global"].insert(UNEMBED_UNSIGNED_INFO, []);
info!(elapsed=?start.elapsed(), total_migrated_pdus=total_migrated, "Finished migration.");
Ok(())
}
const DROP_OLD_SIGNING_KEYS_STORAGE: &str = "drop_server_signingkeys";
+292 -226
View File
@@ -1,9 +1,11 @@
mod webpush;
use std::{fmt::Debug, mem, sync::Arc};
use bytes::BytesMut;
use conduwuit::utils::response::LimitReadExt;
use conduwuit_core::{
Err, Event, Result, debug_warn, err, trace,
Err, Event, Result, Server, debug_warn, err, trace,
utils::{stream::TryIgnore, string_from_bytes},
warn,
};
@@ -15,7 +17,6 @@ use ruma::{
api::{
IncomingResponseExt, OutgoingRequest, OutgoingRequestExt,
auth_scheme::NoAuthentication,
client::push::{Pusher, PusherKind, set_pusher},
path_builder::SinglePath,
push_gateway::send_event_notification::{
self,
@@ -27,18 +28,33 @@ use ruma::{
Action, HighlightTweakValue, PushConditionPowerLevelsCtx, PushConditionRoomCtx,
PushFormat, Ruleset, Tweak,
},
serde::Raw,
serde::{JsonObject, Raw},
uint,
};
use ruminuwuity::pushers::{Pusher, PusherKind, set_pusher::v3::PusherAction};
pub use self::webpush::ActivationOutcome;
use self::webpush::Vapid;
use crate::{Dep, client, config, globals, rooms, sending, users};
const PUSH_GATEWAY_SCHEMES: &[&str] = &["http", "https"];
const WEBPUSH_SCHEMES: &[&str] = &["https"];
/// Whether the pusher opted out of badge counts (MSC4076).
pub(crate) fn badge_counts_disabled(data: &JsonObject) -> bool {
data.contains_key("org.matrix.msc4076.disable_badge_count")
|| data.contains_key("disable_badge_count")
}
pub struct Service {
db: Data,
services: Services,
me: std::sync::Weak<Self>,
vapid: Vapid,
}
struct Services {
server: Arc<Server>,
globals: Dep<globals::Service>,
config: Dep<config::Service>,
client: Dep<client::Service>,
@@ -50,17 +66,24 @@ struct Services {
struct Data {
senderkey_pusher: Arc<Map>,
senderkey_webpushbackoff: Arc<Map>,
senderkey_webpushstate: Arc<Map>,
pushkey_deviceid: Arc<Map>,
}
impl crate::Service for Service {
fn build(args: crate::Args<'_>) -> Result<Arc<Self>> {
Ok(Arc::new(Self {
let vapid = Vapid::init(args.db, args.server.config.webpush_vapid_contact().to_string())?;
Ok(Arc::new_cyclic(|me| Self {
db: Data {
senderkey_pusher: args.db["senderkey_pusher"].clone(),
senderkey_webpushbackoff: args.db["senderkey_webpushbackoff"].clone(),
senderkey_webpushstate: args.db["senderkey_webpushstate"].clone(),
pushkey_deviceid: args.db["pushkey_deviceid"].clone(),
},
services: Services {
server: args.server.clone(),
globals: args.depend::<globals::Service>("globals"),
client: args.depend::<client::Service>("client"),
config: args.depend::<config::Service>("config"),
@@ -70,6 +93,8 @@ impl crate::Service for Service {
users: args.depend::<users::Service>("users"),
sending: args.depend::<sending::Service>("sending"),
},
me: me.clone(),
vapid,
}))
}
@@ -81,71 +106,87 @@ impl Service {
&self,
sender: &UserId,
sender_device: &DeviceId,
pusher: &set_pusher::v3::PusherAction,
) -> Result {
match pusher {
| set_pusher::v3::PusherAction::Post(data) => {
let pushkey = data.pusher.ids.pushkey.as_str();
if pushkey.len() > 512 {
return Err!(Request(InvalidParam(
"Push key length cannot be greater than 512 bytes."
)));
}
if data.pusher.ids.app_id.as_str().len() > 64 {
return Err!(Request(InvalidParam(
"App ID length cannot be greater than 64 bytes."
)));
}
// add some validation to the pusher URL
let pusher_kind = &data.pusher.kind;
if let PusherKind::Http(http) = pusher_kind {
let url = &http.url;
let url = url::Url::parse(&http.url).map_err(|e| {
err!(Request(InvalidParam(
warn!(%url, "HTTP pusher URL is not a valid URL: {e}")
)))
})?;
if ["http", "https"]
.iter()
.all(|&scheme| scheme != url.scheme().to_lowercase())
{
return Err!(Request(InvalidParam(
warn!(%url, "HTTP pusher URL is not a valid HTTP/HTTPS URL")
)));
}
if let Ok(ip) =
IPAddress::parse(url.host_str().expect("URL previously validated"))
{
if !self.services.client.valid_cidr_range(&ip) {
return Err!(Request(InvalidParam(
warn!(%url, "HTTP pusher URL is a forbidden remote address")
)));
}
}
}
let pushkey = data.pusher.ids.pushkey.as_str();
let key = (sender, pushkey);
self.db.senderkey_pusher.put(key, Json(pusher));
self.db.pushkey_deviceid.insert(pushkey, sender_device);
},
| set_pusher::v3::PusherAction::Delete(ids) => {
action: &PusherAction,
) -> Result<bool> {
let pusher = match action {
| PusherAction::Post(pusher) => pusher,
| PusherAction::Delete(ids) => {
self.delete_pusher(sender, ids.pushkey.as_str()).await;
return Ok(false);
},
| _ => return Err!(Request(InvalidParam("Unknown pusher action"))),
};
let pushkey = pusher.ids.pushkey.as_str();
if pushkey.len() > 512 {
return Err!(Request(InvalidParam(
"Push key length cannot be greater than 512 bytes."
)));
}
Ok(())
if pusher.ids.app_id.as_str().len() > 64 {
return Err!(Request(InvalidParam("App ID length cannot be greater than 64 bytes.")));
}
match &pusher.kind {
| PusherKind::Http(http) => {
self.validate_push_url(&http.url, PUSH_GATEWAY_SCHEMES)?;
},
| PusherKind::WebPush(data) => {
self.validate_push_url(&data.url, WEBPUSH_SCHEMES)?;
webpush::decode_pushkey(pushkey)?;
webpush::decode_auth(&data.auth)?;
if let Some(default_payload) = data.data.get("default_payload") {
if serde_json::to_vec(default_payload)?.len() > webpush::MAX_DEFAULT_PAYLOAD {
return Err!(Request(TooLarge(
"Default payload cannot be greater than {} bytes.",
webpush::MAX_DEFAULT_PAYLOAD
)));
}
}
},
| PusherKind::Email(_) | PusherKind::Custom { .. } => (),
}
let key = (sender, pushkey);
let previous: Option<Pusher> = self
.db
.senderkey_pusher
.qry(&key)
.await
.deserialized()
.map_err(|e| {
if !e.is_not_found() {
warn!(%pushkey, "Replacing a stored pusher that could not be read: {e}");
}
})
.ok();
let needs_activation = self
.webpush_needs_activation(sender, pusher, previous.as_ref())
.await;
if !needs_activation && pusher.kind.as_webpush().is_none() {
self.db.senderkey_webpushstate.del(key);
self.db.senderkey_webpushbackoff.del(key);
}
self.db.senderkey_pusher.put(key, Json(pusher));
self.db.pushkey_deviceid.insert(pushkey, sender_device);
// Must run after the put: it delivers straight away, and a rejection
// from the push service deletes the pusher it is validating.
if needs_activation {
self.begin_webpush_activation(sender, pusher);
}
Ok(needs_activation)
}
pub async fn delete_pusher(&self, sender: &UserId, pushkey: &str) {
let key = (sender, pushkey);
self.db.senderkey_pusher.del(key);
self.db.senderkey_webpushbackoff.del(key);
self.db.senderkey_webpushstate.del(key);
self.db.pushkey_deviceid.remove(pushkey);
self.services
@@ -192,6 +233,35 @@ impl Service {
.map(ToOwned::to_owned)
}
/// Checks that a pusher URL is well-formed and not pointing at a forbidden
/// address.
fn validate_push_url(&self, url: &str, allowed: &[&str]) -> Result<url::Url> {
let parsed = url::Url::parse(url).map_err(|e| {
err!(Request(InvalidParam(warn!(%url, "Pusher URL is not a valid URL: {e}"))))
})?;
let scheme = parsed.scheme().to_lowercase();
if !allowed.contains(&scheme.as_str()) {
return Err!(Request(InvalidParam(warn!(
%url,
"Pusher URL scheme {scheme} is not allowed"
))));
}
if let Some(host) = parsed.host_str() {
if let Ok(ip) = IPAddress::parse(host) {
if !self.services.client.valid_cidr_range(&ip) {
return Err!(Request(InvalidParam(warn!(
%url,
"Pusher URL is a forbidden remote address"
))));
}
}
}
Ok(parsed)
}
#[tracing::instrument(skip(self, dest, request))]
pub async fn send_request<T>(&self, dest: &str, request: T) -> Result<T::IncomingResponse>
where
@@ -211,9 +281,41 @@ impl Service {
})?
.map(BytesMut::freeze);
let reqwest_request = reqwest::Request::try_from(http_request)?;
let response = self
.execute_pusher_request(
&self.services.client.pusher,
reqwest::Request::try_from(http_request)?,
)
.await?;
if let Some(url_host) = reqwest_request.url().host_str() {
let status = response.status();
if !status.is_success() {
debug_warn!("Push gateway response body: {:?}", string_from_bytes(response.body()));
return Err!(BadServerResponse(warn!(
"Push gateway {dest} returned unsuccessful HTTP response: {status}"
)));
}
let (parts, body) = response.into_parts();
T::IncomingResponse::try_from_http_response(http::Response::from_parts(
parts,
body.as_ref(),
))
.map_err(|e| {
err!(BadServerResponse(warn!("Push gateway {dest} returned invalid response: {e}")))
})
}
/// Executes a request, checking the destination address on the way out and
/// on the way in. The status is returned as-is; callers decide which codes
/// are failures.
async fn execute_pusher_request(
&self,
client: &reqwest::Client,
request: reqwest::Request,
) -> Result<http::Response<Vec<u8>>> {
if let Some(url_host) = request.url().host_str() {
trace!("Checking request URL for IP");
if let Ok(ip) = IPAddress::parse(url_host) {
if !self.services.client.valid_cidr_range(&ip) {
@@ -222,69 +324,36 @@ impl Service {
}
}
let response = self.services.client.pusher.execute(reqwest_request).await;
let dest = request.url().clone();
let mut response = client.execute(request).await.map_err(|e| {
err!(BadServerResponse(warn!(%dest, "Could not send request to pusher: {e}")))
})?;
match response {
| Ok(mut response) => {
// reqwest::Response -> http::Response conversion
trace!("Checking response destination's IP");
if let Some(remote_addr) = response.remote_addr() {
if let Ok(ip) = IPAddress::parse(remote_addr.ip().to_string()) {
if !self.services.client.valid_cidr_range(&ip) {
return Err!(BadServerResponse(
"Not allowed to send requests to this IP"
));
}
}
trace!("Checking response destination's IP");
if let Some(remote_addr) = response.remote_addr() {
if let Ok(ip) = IPAddress::parse(remote_addr.ip().to_string()) {
if !self.services.client.valid_cidr_range(&ip) {
return Err!(BadServerResponse("Not allowed to send requests to this IP"));
}
let status = response.status();
let mut http_response_builder = http::Response::builder()
.status(status)
.version(response.version());
mem::swap(
response.headers_mut(),
http_response_builder
.headers_mut()
.expect("http::response::Builder is usable"),
);
let body = response
.limit_read(
self.services
.config
.max_request_size
.try_into()
.expect("usize fits into u64"),
)
.await?;
if !status.is_success() {
debug_warn!("Push gateway response body: {:?}", string_from_bytes(&body));
return Err!(BadServerResponse(warn!(
"Push gateway {dest} returned unsuccessful HTTP response: {status}"
)));
}
let (parts, body) = http_response_builder
.body(body)
.expect("reqwest body is valid http body")
.into_parts();
let response = T::IncomingResponse::try_from_http_response(
http::Response::from_parts(parts, body.as_ref()),
);
response.map_err(|e| {
err!(BadServerResponse(warn!(
"Push gateway {dest} returned invalid response: {e}"
)))
})
},
| Err(e) => {
warn!("Could not send request to pusher {dest}: {e}");
Err(e.into())
},
}
}
let mut result = http::Response::new(Vec::new());
*result.status_mut() = response.status();
*result.version_mut() = response.version();
mem::swap(response.headers_mut(), result.headers_mut());
*result.body_mut() = response
.limit_read(
self.services
.config
.max_request_size
.try_into()
.expect("usize fits into u64"),
)
.await?;
Ok(result)
}
#[tracing::instrument(skip(self, user, unread, pusher, ruleset, event))]
@@ -315,13 +384,7 @@ impl Service {
let serialized = event.to_format();
for action in self
.get_actions(
user,
&ruleset,
power_levels.clone(),
&serialized,
event.room_id().unwrap(),
)
.get_actions(user, &ruleset, power_levels.clone(), &serialized, room_id)
.await
{
let n = match action {
@@ -343,7 +406,8 @@ impl Service {
}
if notify == Some(true) {
self.send_notice(unread, pusher, tweaks, event).await?;
self.send_notice(user, unread, pusher, tweaks, event)
.await?;
}
// Else the event triggered no actions
@@ -406,6 +470,7 @@ impl Service {
#[tracing::instrument(skip(self, unread, pusher, tweaks, event))]
async fn send_notice<E>(
&self,
user: &UserId,
unread: UInt,
pusher: &Pusher,
tweaks: Vec<Tweak>,
@@ -417,109 +482,110 @@ impl Service {
// TODO: email
match &pusher.kind {
| PusherKind::Http(http) => {
let url = &http.url;
let url = url::Url::parse(&http.url).map_err(|e| {
err!(Request(InvalidParam(
warn!(%url, "HTTP pusher URL is not a valid URL: {e}")
)))
})?;
self.validate_push_url(&http.url, PUSH_GATEWAY_SCHEMES)?;
if ["http", "https"]
.iter()
.all(|&scheme| scheme != url.scheme().to_lowercase())
{
return Err!(Request(InvalidParam(
warn!(%url, "HTTP pusher URL is not a valid HTTP/HTTPS URL")
)));
}
if let Ok(ip) =
IPAddress::parse(url.host_str().expect("URL previously validated"))
{
if !self.services.client.valid_cidr_range(&ip) {
return Err!(Request(InvalidParam(
warn!(%url, "HTTP pusher URL is a forbidden remote address")
)));
}
}
// TODO (timo): can pusher/devices have conflicting formats
let event_id_only = http.format == Some(PushFormat::EventIdOnly);
let mut device =
Device::new(pusher.ids.app_id.clone(), pusher.ids.pushkey.clone());
device.data.data.clone_from(&http.data);
device.data.format.clone_from(&http.format);
// Tweaks are only added if the format is NOT event_id_only
if !event_id_only {
device.tweaks.clone_from(&tweaks);
}
let d = vec![device];
let mut notify = Notification::new(d);
notify.event_id = Some(event.event_id().to_owned());
notify.room_id = Some(event.room_id().unwrap().to_owned());
if http
.data
.get("org.matrix.msc4076.disable_badge_count")
.is_none() && http.data.get("disable_badge_count").is_none()
{
notify.counts = NotificationCounts::new(unread, uint!(0));
} else {
// counts will not be serialised if it's the default (0, 0)
// skip_serializing_if = "NotificationCounts::is_default"
notify.counts = NotificationCounts::default();
}
if !event_id_only {
if *event.kind() == TimelineEventType::RoomEncrypted
|| tweaks.iter().any(|t| {
matches!(
t,
Tweak::Highlight(HighlightTweakValue::Yes) | Tweak::Sound(_)
)
}) {
notify.prio = NotificationPriority::High;
} else {
notify.prio = NotificationPriority::Low;
}
notify.sender = Some(event.sender().to_owned());
notify.event_type = Some(event.kind().to_owned());
notify.content = serde_json::value::to_raw_value(event.content()).ok();
if *event.kind() == TimelineEventType::RoomMember {
notify.user_is_target =
event.state_key() == Some(event.sender().as_str());
}
notify.sender_display_name =
self.services.users.displayname(event.sender()).await.ok();
notify.room_name = self
.services
.state_accessor
.get_name(event.room_id().unwrap())
.await
.ok();
notify.room_alias = self
.services
.state_accessor
.get_canonical_alias(event.room_id().unwrap())
.await
.ok();
}
let notify = self
.build_notification(
pusher,
&http.data,
http.format.as_ref(),
&tweaks,
event,
unread,
)
.await;
self.send_request(&http.url, send_event_notification::v1::Request::new(notify))
.await?;
Ok(())
},
// TODO: Handle email
//PusherKind::Email(_) => Ok(()),
| _ => Ok(()),
| PusherKind::WebPush(data) => {
let notify = self
.build_notification(
pusher,
&data.data,
data.format.as_ref(),
&tweaks,
event,
unread,
)
.await;
self.send_webpush_notice(user, pusher, data, notify).await
},
| PusherKind::Email(_) | PusherKind::Custom { .. } => Ok(()),
}
}
async fn build_notification<E>(
&self,
pusher: &Pusher,
data: &JsonObject,
format: Option<&PushFormat>,
tweaks: &[Tweak],
event: &E,
unread: UInt,
) -> Notification
where
E: Event + Send + Sync,
{
// TODO (timo): can pusher/devices have conflicting formats
let event_id_only = format == Some(&PushFormat::EventIdOnly);
let mut device = Device::new(pusher.ids.app_id.clone(), pusher.ids.pushkey.clone());
device.data.data.clone_from(data);
device.data.format = format.cloned();
// Tweaks are only added if the format is NOT event_id_only
if !event_id_only {
device.tweaks = tweaks.to_owned();
}
let mut notify = Notification::new(vec![device]);
notify.event_id = Some(event.event_id().to_owned());
notify.room_id = event.room_id().map(ToOwned::to_owned);
if badge_counts_disabled(data) {
// counts will not be serialised if it's the default (0, 0)
// skip_serializing_if = "NotificationCounts::is_default"
notify.counts = NotificationCounts::default();
} else {
notify.counts = NotificationCounts::new(unread, uint!(0));
}
if !event_id_only {
if *event.kind() == TimelineEventType::RoomEncrypted
|| tweaks.iter().any(|t| {
matches!(t, Tweak::Highlight(HighlightTweakValue::Yes) | Tweak::Sound(_))
}) {
notify.prio = NotificationPriority::High;
} else {
notify.prio = NotificationPriority::Low;
}
notify.sender = Some(event.sender().to_owned());
notify.event_type = Some(event.kind().to_owned());
notify.content = serde_json::value::to_raw_value(event.content()).ok();
if *event.kind() == TimelineEventType::RoomMember {
notify.user_is_target = event.state_key() == Some(event.sender().as_str());
}
notify.sender_display_name =
self.services.users.displayname(event.sender()).await.ok();
if let Some(room_id) = event.room_id() {
notify.room_name = self.services.state_accessor.get_name(room_id).await.ok();
notify.room_alias = self
.services
.state_accessor
.get_canonical_alias(room_id)
.await
.ok();
}
}
notify
}
}
+208
View File
@@ -0,0 +1,208 @@
//! Web Push key validation, token comparison, and payload shaping helpers.
use base64::{Engine, engine::general_purpose::URL_SAFE_NO_PAD};
use conduwuit_core::{Err, Result, err};
use ruma::{api::push_gateway::send_event_notification::v1::Notification, serde::JsonObject};
use serde_json::Value as JsonValue;
use web_push_native::{Auth, p256::PublicKey};
use crate::pusher::badge_counts_disabled;
const MAX_PAYLOAD: usize = 3900;
const MAX_BODY_LENGTH: usize = 1000;
const MAX_CIPHERTEXT_LENGTH: usize = 2000;
pub(crate) fn decode_pushkey(pushkey: &str) -> Result<PublicKey> {
let bytes = URL_SAFE_NO_PAD
.decode(pushkey)
.map_err(|e| err!(Request(InvalidParam("Push key is not URL-safe base64: {e}"))))?;
if bytes.len() != 65 || bytes.first() != Some(&0x04) {
return Err!(Request(InvalidParam(
"Push key must be a P-256 public key in the uncompressed form"
)));
}
PublicKey::from_sec1_bytes(&bytes)
.map_err(|e| err!(Request(InvalidParam("Push key is not a valid P-256 public key: {e}"))))
}
pub(crate) fn decode_auth(auth: &str) -> Result<Auth> {
let bytes = URL_SAFE_NO_PAD
.decode(auth)
.map_err(|e| err!(Request(InvalidParam("Auth secret is not URL-safe base64: {e}"))))?;
let bytes: [u8; 16] = bytes
.try_into()
.map_err(|_| err!(Request(InvalidParam("Auth secret must be 16 bytes"))))?;
Ok(Auth::from(bytes))
}
pub(super) fn constant_time_eq(a: &str, b: &str) -> bool {
a.len() == b.len()
&& a.bytes()
.zip(b.bytes())
.fold(0_u8, |acc, (x, y)| acc | (x ^ y))
== 0
}
pub(super) fn build_payload(notify: &Notification, data: &JsonObject) -> Result<Vec<u8>> {
let JsonValue::Object(mut payload) = serde_json::to_value(notify)? else {
return Err!("Notification did not serialize to an object");
};
payload.remove("devices");
if let Some(JsonValue::Object(counts)) = payload.remove("counts") {
for key in ["unread", "missed_calls"] {
if let Some(count) = counts.get(key) {
payload.insert(key.to_owned(), count.clone());
}
}
} else if !badge_counts_disabled(data) {
payload.insert("unread".to_owned(), 0.into());
}
if let Some(JsonValue::Object(default_payload)) = data.get("default_payload") {
for (key, value) in default_payload {
payload.insert(key.clone(), value.clone());
}
}
if let Some(JsonValue::Object(content)) = payload.get_mut("content") {
if let Some(JsonValue::String(body)) = content.get("body")
&& body.len() > MAX_BODY_LENGTH
{
content.insert(
"body".to_owned(),
format!("{}…", truncate_bytes(body, MAX_BODY_LENGTH)).into(),
);
}
if content
.get("ciphertext")
.and_then(JsonValue::as_str)
.is_some_and(|ciphertext| ciphertext.len() > MAX_CIPHERTEXT_LENGTH)
{
content.remove("ciphertext");
}
}
let mut encoded = serde_json::to_vec(&payload)?;
if encoded.len() > MAX_PAYLOAD {
payload.remove("content");
encoded = serde_json::to_vec(&payload)?;
}
if encoded.len() > MAX_PAYLOAD {
return Err!(Request(TooLarge("Web push payload is too large to deliver")));
}
Ok(encoded)
}
fn truncate_bytes(text: &str, max: usize) -> String {
let mut truncated = String::with_capacity(max);
for ch in text.chars() {
if truncated.len().saturating_add(ch.len_utf8()) > max {
break;
}
truncated.push(ch);
}
truncated
}
#[cfg(test)]
mod tests {
use ruma::{
api::push_gateway::send_event_notification::v1::{Notification, NotificationCounts},
push::PushFormat,
serde::JsonObject,
uint,
};
use serde_json::{Value as JsonValue, json};
use super::{MAX_BODY_LENGTH, build_payload, constant_time_eq};
fn payload_of(notify: &Notification, data: &JsonObject) -> JsonObject {
let encoded = build_payload(notify, data).unwrap();
match serde_json::from_slice(&encoded).unwrap() {
| JsonValue::Object(payload) => payload,
| other => panic!("expected an object, got {other}"),
}
}
fn notification_with_body(body: &str) -> Notification {
let mut notify = Notification::new(vec![]);
notify.content = Some(serde_json::value::to_raw_value(&json!({ "body": body })).unwrap());
notify
}
#[test]
fn flattens_counts_and_drops_devices() {
let mut notify = Notification::new(vec![]);
notify.counts = NotificationCounts::new(uint!(3), uint!(1));
let payload = payload_of(&notify, &JsonObject::new());
assert!(!payload.contains_key("devices"));
assert!(!payload.contains_key("counts"));
assert_eq!(payload.get("unread"), Some(&json!(3)));
assert_eq!(payload.get("missed_calls"), Some(&json!(1)));
}
#[test]
fn keeps_zero_unread_count() {
let payload = payload_of(&Notification::new(vec![]), &JsonObject::new());
assert_eq!(payload.get("unread"), Some(&json!(0)));
}
#[test]
fn omits_count_when_badges_are_disabled() {
let mut data = JsonObject::new();
data.insert("org.matrix.msc4076.disable_badge_count".to_owned(), json!(true));
let payload = payload_of(&Notification::new(vec![]), &data);
assert!(!payload.contains_key("unread"));
}
#[test]
fn merges_default_payload() {
let mut data = JsonObject::new();
data.insert("default_payload".to_owned(), json!({ "session_id": "abc" }));
data.insert("format".to_owned(), json!(PushFormat::EventIdOnly));
let payload = payload_of(&Notification::new(vec![]), &data);
assert_eq!(payload.get("session_id"), Some(&json!("abc")));
assert!(!payload.contains_key("format"));
}
#[test]
fn truncates_long_body() {
let payload = payload_of(&notification_with_body(&"a".repeat(2000)), &JsonObject::new());
let body = payload["content"]["body"].as_str().unwrap();
assert!(body.len() <= MAX_BODY_LENGTH + '…'.len_utf8());
assert!(body.ends_with('…'));
}
#[test]
fn truncates_multibyte_body_by_bytes() {
let payload = payload_of(&notification_with_body(&"é".repeat(2000)), &JsonObject::new());
let body = payload["content"]["body"].as_str().unwrap();
assert!(body.len() <= MAX_BODY_LENGTH + '…'.len_utf8());
assert!(body.chars().count() < 2000);
}
#[test]
fn rejects_payload_that_cannot_be_shrunk() {
let mut data = JsonObject::new();
data.insert("default_payload".to_owned(), json!({ "pad": "x".repeat(4000) }));
build_payload(&Notification::new(vec![]), &data).unwrap_err();
}
#[test]
fn token_comparison() {
assert!(constant_time_eq("abc", "abc"));
assert!(!constant_time_eq("abc", "abz"));
assert!(!constant_time_eq("abc", "abcd"));
}
}
+432
View File
@@ -0,0 +1,432 @@
mod helpers;
mod vapid;
use std::time::{Duration, SystemTime};
use conduwuit_core::{Err, Result, debug, debug_warn, err, utils, warn};
use conduwuit_database::{Deserialized, Json};
use futures::StreamExt;
pub(super) use helpers::{decode_auth, decode_pushkey};
use http::StatusCode;
use ruma::{
UserId,
api::push_gateway::send_event_notification::v1::{Notification, NotificationPriority},
push::PushFormat,
};
use ruminuwuity::pushers::{Pusher, WebPushPusherData};
use serde::{Deserialize, Serialize};
use serde_json::json;
use tokio::time::sleep;
pub(super) use vapid::Vapid;
use web_push_native::WebPushBuilder;
use self::helpers::{build_payload, constant_time_eq};
use super::{Service, WEBPUSH_SCHEMES};
const TTL: Duration = Duration::from_hours(48);
const ACK_TOKEN_TTL_MS: u64 = 5 * 60 * 1000;
const ACK_TOKEN_LENGTH: usize = 32;
pub(super) const MAX_DEFAULT_PAYLOAD: usize = 1024;
const DELIVERY_ATTEMPTS: u32 = 3;
const RETRY_BASE: Duration = Duration::from_secs(2);
const RETRY_MAX: Duration = Duration::from_secs(20);
const BACKOFF_BASE: Duration = Duration::from_mins(1);
const BACKOFF_MAX: Duration = Duration::from_hours(24);
#[derive(Clone, Copy, Debug)]
pub enum ActivationOutcome {
Activated,
NoSuchPusher,
UnknownToken,
Expired,
}
enum Attempt {
Delivered,
/// The subscription is gone; delete the pusher.
Rejected,
/// Transient, carrying the delay the push service asked for.
Failed(Option<Duration>),
}
#[derive(Clone, Default, Deserialize, Serialize)]
pub(super) struct WebPushState {
pub(super) activated: bool,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub(super) ack_token: Option<String>,
#[serde(default)]
pub(super) token_expires_at: u64,
}
#[derive(Clone, Default, Deserialize, Serialize)]
struct WebPushBackoff {
#[serde(default)]
next_attempt_at: u64,
#[serde(default)]
failure_count: u32,
}
impl Service {
pub fn webpush_vapid_public_key(&self) -> &str { self.vapid.public_key() }
/// Reports whether a Web Push pusher has completed its activation
/// handshake.
pub async fn is_webpush_activated(&self, sender: &UserId, pushkey: &str) -> bool {
self.webpush_state(sender, pushkey).await.activated
}
pub(super) async fn webpush_state(&self, sender: &UserId, pushkey: &str) -> WebPushState {
self.db
.senderkey_webpushstate
.qry(&(sender, pushkey))
.await
.deserialized()
.unwrap_or_default()
}
fn put_webpush_state(&self, sender: &UserId, pushkey: &str, state: &WebPushState) {
self.db
.senderkey_webpushstate
.put((sender, pushkey), Json(state));
}
pub(super) async fn webpush_needs_activation(
&self,
sender: &UserId,
pusher: &Pusher,
previous: Option<&Pusher>,
) -> bool {
let Some(data) = pusher.kind.as_webpush() else {
return false;
};
let Some(old) = previous.and_then(|previous| previous.kind.as_webpush()) else {
return true;
};
old.url != data.url
|| old.auth != data.auth
|| !self
.webpush_state(sender, pusher.ids.pushkey.as_str())
.await
.activated
}
pub(super) fn begin_webpush_activation(&self, sender: &UserId, pusher: &Pusher) {
// Send a one-time encrypted token before allowing notification
// delivery.
let token = utils::rand::string(ACK_TOKEN_LENGTH);
let state = WebPushState {
activated: false,
ack_token: Some(token.clone()),
token_expires_at: utils::millis_since_unix_epoch().saturating_add(ACK_TOKEN_TTL_MS),
};
self.put_webpush_state(sender, pusher.ids.pushkey.as_str(), &state);
self.db
.senderkey_webpushbackoff
.del((sender, pusher.ids.pushkey.as_str()));
let Some(service) = self.me.upgrade() else {
return;
};
let sender = sender.to_owned();
let pusher = pusher.clone();
let payload = json!({ "app_id": pusher.ids.app_id, "ack_token": token });
self.services.server.runtime().spawn(async move {
let Some(data) = pusher.kind.as_webpush() else {
return;
};
if let Err(e) = service
.deliver_webpush(
&sender,
&pusher,
data,
payload.to_string().into_bytes(),
NotificationPriority::High,
)
.await
{
debug_warn!(
%sender, app_id = %pusher.ids.app_id,
"Failed to send web push validation token: {e}"
);
}
});
}
#[tracing::instrument(skip(self, ack_token), level = "debug")]
/// Activates the pusher whose pending token matches the client
/// acknowledgement.
pub async fn activate_webpush_pusher(
&self,
sender: &UserId,
app_id: &str,
ack_token: &str,
) -> Result<ActivationOutcome> {
let candidates: Vec<String> = self
.get_pushkeys(sender)
.filter_map(async |pushkey| {
let pusher = self.get_pusher(sender, &pushkey).await.ok()?;
(pusher.kind.as_webpush().is_some() && pusher.ids.app_id == app_id)
.then_some(pushkey)
})
.collect()
.await;
if candidates.is_empty() {
return Ok(ActivationOutcome::NoSuchPusher);
}
for pushkey in candidates {
let mut state = self.webpush_state(sender, &pushkey).await;
if !state
.ack_token
.as_deref()
.is_some_and(|token| constant_time_eq(token, ack_token))
{
continue;
}
state.ack_token = None;
if state.token_expires_at < utils::millis_since_unix_epoch() {
self.put_webpush_state(sender, &pushkey, &state);
return Ok(ActivationOutcome::Expired);
}
state.activated = true;
self.put_webpush_state(sender, &pushkey, &state);
self.db.senderkey_webpushbackoff.del((sender, &pushkey));
return Ok(ActivationOutcome::Activated);
}
Ok(ActivationOutcome::UnknownToken)
}
#[tracing::instrument(skip(self, pusher, data, notify), level = "debug")]
/// Delivers a notification only after the pusher has completed activation.
pub(super) async fn send_webpush_notice(
&self,
user: &UserId,
pusher: &Pusher,
data: &WebPushPusherData,
notify: Notification,
) -> Result {
let pushkey = pusher.ids.pushkey.as_str();
if !self.webpush_state(user, pushkey).await.activated {
debug!(%user, %pushkey, "Web push pusher is not activated yet, skipping");
return Ok(());
}
let backoff: WebPushBackoff = self
.db
.senderkey_webpushbackoff
.qry(&(user, pushkey))
.await
.deserialized()
.unwrap_or_default();
if utils::millis_since_unix_epoch() < backoff.next_attempt_at {
debug!(%user, %pushkey, "Web push pusher is backing off, dropping notification");
return Ok(());
}
let prio = if data.format.as_ref() == Some(&PushFormat::EventIdOnly) {
NotificationPriority::Low
} else {
notify.prio.clone()
};
let payload = build_payload(&notify, &data.data)?;
self.deliver_webpush(user, pusher, data, payload, prio)
.await
}
#[tracing::instrument(skip(self, pusher, data, payload), level = "debug")]
/// Encrypts and sends a Web Push request, retrying transient failures and
/// deleting rejected subscriptions.
async fn deliver_webpush(
&self,
user: &UserId,
pusher: &Pusher,
data: &WebPushPusherData,
payload: Vec<u8>,
prio: NotificationPriority,
) -> Result {
let endpoint = self.validate_push_url(&data.url, WEBPUSH_SCHEMES)?;
let authorization = self.vapid.authorization(&endpoint)?;
let pushkey = pusher.ids.pushkey.as_str();
let mut attempt = 1_u32;
loop {
match self
.attempt_webpush(user, pusher, data, &payload, &authorization, &prio)
.await?
{
| Attempt::Delivered => {
self.db.senderkey_webpushbackoff.del((user, pushkey));
return Ok(());
},
| Attempt::Rejected => {
self.delete_pusher(user, pushkey).await;
return Ok(());
},
| Attempt::Failed(retry_after) => {
if attempt >= DELIVERY_ATTEMPTS {
self.back_off_webpush(user, pushkey, retry_after).await;
return Err!(BadServerResponse(warn!(
%user, %pushkey,
"Push service did not accept the notification after {attempt} attempts"
)));
}
let delay = retry_after
.unwrap_or_else(|| RETRY_BASE.saturating_mul(attempt))
.min(RETRY_MAX);
sleep(delay).await;
attempt = attempt.saturating_add(1);
},
}
}
}
/// The message is re-encrypted per attempt, since each needs its own
/// ephemeral key.
async fn attempt_webpush(
&self,
user: &UserId,
pusher: &Pusher,
data: &WebPushPusherData,
payload: &[u8],
authorization: &str,
prio: &NotificationPriority,
) -> Result<Attempt> {
let pushkey = pusher.ids.pushkey.as_str();
let mut request = WebPushBuilder::new(
data.url
.parse()
.map_err(|e| err!(Request(InvalidParam("Pusher URL is not a valid URI: {e}"))))?,
decode_pushkey(pushkey)?,
decode_auth(&data.auth)?,
)
.with_valid_duration(TTL)
.build(payload.to_vec())
.map_err(|e| err!("Failed to encrypt web push message: {e}"))?;
let headers = request.headers_mut();
headers.insert(http::header::AUTHORIZATION, authorization.parse()?);
headers.insert(
"urgency",
if *prio == NotificationPriority::High {
"high"
} else {
"normal"
}
.parse()?,
);
let response = match self
.execute_pusher_request(
&self.services.client.webpush,
reqwest::Request::try_from(request)?,
)
.await
{
| Ok(response) => response,
| Err(e) => {
debug_warn!(%user, %pushkey, "Could not reach the push service: {e}");
return Ok(Attempt::Failed(None));
},
};
let status = response.status();
if status.is_success() {
return Ok(Attempt::Delivered);
}
if matches!(status, StatusCode::FORBIDDEN | StatusCode::NOT_FOUND | StatusCode::GONE) {
warn!(
%user, %pushkey,
"Push service rejected the subscription ({status}), deleting pusher"
);
return Ok(Attempt::Rejected);
}
if status.is_redirection() {
warn!(%user, %pushkey, "Push service redirected ({status}); not following it");
}
debug_warn!(%user, %pushkey, "Push service returned unsuccessful HTTP response: {status}");
Ok(Attempt::Failed(
(status == StatusCode::TOO_MANY_REQUESTS)
.then(|| retry_after(response.headers()))
.flatten(),
))
}
async fn back_off_webpush(
&self,
user: &UserId,
pushkey: &str,
retry_after: Option<Duration>,
) {
let key = (user, pushkey);
let mut backoff: WebPushBackoff = self
.db
.senderkey_webpushbackoff
.qry(&key)
.await
.deserialized()
.unwrap_or_default();
backoff.failure_count = backoff.failure_count.saturating_add(1);
let delay = retry_after.map_or_else(
|| {
BACKOFF_BASE
.saturating_mul(1_u32 << backoff.failure_count.min(10))
.min(BACKOFF_MAX)
},
|retry_after| retry_after.min(BACKOFF_MAX),
);
backoff.next_attempt_at = utils::millis_since_unix_epoch()
.saturating_add(delay.as_millis().try_into().unwrap_or(u64::MAX));
self.db.senderkey_webpushbackoff.put(key, Json(&backoff));
}
}
/// RFC 9110 allows `Retry-After` to be either a delay in seconds or an HTTP
/// date.
fn retry_after(headers: &http::HeaderMap) -> Option<Duration> {
let value = headers.get(http::header::RETRY_AFTER)?.to_str().ok()?;
if let Ok(secs) = value.parse::<u64>() {
return Some(Duration::from_secs(secs));
}
httpdate::parse_http_date(value)
.ok()?
.duration_since(SystemTime::now())
.ok()
}
+97
View File
@@ -0,0 +1,97 @@
//! VAPID key management and authorization-header generation for Web Push.
use std::{collections::HashMap, sync::Arc};
use base64::{Engine, engine::general_purpose::URL_SAFE_NO_PAD};
use conduwuit_core::{Result, SyncRwLock, debug_info, err, utils};
use conduwuit_database::Database;
use p256::{
SecretKey,
ecdsa::{Signature, SigningKey, signature::Signer},
elliptic_curve::sec1::ToEncodedPoint,
};
use serde_json::json;
const VAPID_TTL_SECS: u64 = 12 * 60 * 60;
const VAPID_REFRESH_MARGIN_SECS: u64 = 60;
/// The server's VAPID keypair and cached authorization headers.
pub(crate) struct Vapid {
secret: SecretKey,
public_key: String,
contact: String,
authorizations: SyncRwLock<HashMap<String, (String, u64)>>,
}
impl Vapid {
/// Loads the persistent VAPID keypair, generating one on first use.
pub(crate) fn init(db: &Arc<Database>, contact: String) -> Result<Self> {
let secret = match db["global"].get_blocking(b"vapid_secret_key") {
| Ok(ref val) => SecretKey::from_slice(val).map_err(|e| {
err!(Database(error!(
"Stored VAPID key is invalid: {e}. Refusing to replace it, as that would \
invalidate every existing web push subscription."
)))
})?,
| Err(e) if !e.is_not_found() => return Err(e),
| Err(_) => {
let secret = loop {
if let Ok(secret) = SecretKey::from_slice(&rand::random::<[u8; 32]>()) {
break secret;
}
};
db["global"].raw_put(b"vapid_secret_key", &*secret.to_bytes());
debug_info!("Generated new VAPID keypair for web push");
secret
},
};
let public_key =
URL_SAFE_NO_PAD.encode(secret.public_key().to_encoded_point(false).as_bytes());
Ok(Self {
secret,
public_key,
contact,
authorizations: SyncRwLock::new(HashMap::new()),
})
}
pub(super) fn public_key(&self) -> &str { &self.public_key }
/// Returns a cached or freshly signed VAPID authorization for an endpoint.
pub(super) fn authorization(&self, endpoint: &url::Url) -> Result<String> {
let audience = endpoint.origin().ascii_serialization();
let now = utils::millis_since_unix_epoch().saturating_div(1000);
if let Some((header, expires_at)) = self.authorizations.read().get(&audience)
&& now.saturating_add(VAPID_REFRESH_MARGIN_SECS) < *expires_at
{
return Ok(header.clone());
}
let expires_at = now.saturating_add(VAPID_TTL_SECS);
let header = self.sign(&audience, expires_at)?;
let mut authorizations = self.authorizations.write();
authorizations.retain(|_, (_, expires_at)| now < *expires_at);
authorizations.insert(audience, (header.clone(), expires_at));
Ok(header)
}
fn sign(&self, audience: &str, expires_at: u64) -> Result<String> {
let header =
URL_SAFE_NO_PAD.encode(serde_json::to_vec(&json!({ "typ": "JWT", "alg": "ES256" }))?);
let claims = URL_SAFE_NO_PAD.encode(serde_json::to_vec(&json!({
"aud": audience,
"exp": expires_at,
"sub": self.contact,
}))?);
let signing_input = format!("{header}.{claims}");
let signature: Signature = SigningKey::from(&self.secret).sign(signing_input.as_bytes());
let signature = URL_SAFE_NO_PAD.encode(signature.to_bytes());
Ok(format!("vapid t={signing_input}.{signature},k={}", self.public_key))
}
}
@@ -461,6 +461,8 @@ impl super::Service {
assert!(!candidates.is_empty(), "no candidates to fetch missing events from");
let mut discovered_events =
HashMap::with_capacity(events.len().saturating_add(events.len().saturating_mul(3)));
// Events we fetched, but can never resolve (e.g. they are malformed).
let mut unusable_events: HashSet<OwnedEventId> = HashSet::new();
trace!(
elapsed=?start.elapsed(),
"Fetching {} unknown PDUs on demand from {} candidates",
@@ -468,13 +470,27 @@ impl super::Service {
candidates.len()
);
// For each event in the apex set,
for apex_event_id in &events {
// Seed the search queue with the apex event (which itself may be
// missing).
let mut todo: VecDeque<OwnedEventId> = [apex_event_id.to_owned()].into();
// For each event in the queue,
while let Some(target_id) = todo.pop_front() {
if discovered_events.contains_key(&target_id) {
// If we already discovered this event, or know it to be
// unusable, skip
if discovered_events.contains_key(&target_id)
|| unusable_events.contains(&target_id)
{
continue;
}
// If we already have this event locally (even as an outlier),
// discover it locally and continue.
// If this function returns a value, we already have all the
// auth events for the event (since we were able to persist it
// at all), so there's no need to add its dependencies to the
// queue.
if let Ok(local_pdu) = self.services.timeline.get_pdu(&target_id).await {
trace!(elapsed=?start.elapsed(), "Found {target_id} in db");
let mut obj = local_pdu.into_canonical_object();
@@ -483,6 +499,8 @@ impl super::Service {
continue;
}
// Are we allowed to try to pull this event? If not, propagate
// the error.
self.ensure_can_pull_event(&target_id).inspect_err(|e| {
debug_warn!(
error=?e,
@@ -496,19 +514,30 @@ impl super::Service {
.fetch_event_vias(candidates.iter(), &target_id, room_version_rules)
.await
{
| Ok((_, x)) => {
| Ok((_, value)) => {
self.clear_failed_pdu(&target_id);
x
value
},
| Err(e) => {
// The remote server returned an error for this event.
// Back off on the event, and fail the fetch attempt
// (since even one missing auth event makes the entire
// call unusable).
self.hit_failed_pdu_pull(target_id.clone());
// TODO(nex): We should still persist the events we
// already fetched to avoid another event being received
// and immediately triggering the same walk we're
// aborting now.
return Err!(Request(NotFound(warn!(
elapsed=?start.elapsed(),
%apex_event_id,
"failed to fetch missing auth event {target_id} from any candidate: {e}"
"failed to fetch missing auth event {target_id} from any candidate: \
{e}"
))));
},
};
// Find out what this event's auth events might be missing
// locally.
let auth_events =
match expect_event_id_array(&value, "auth_events").map_err(|e| {
err!(Request(BadJson(warn!(
@@ -519,46 +548,33 @@ impl super::Service {
}) {
| Ok(auth_events) => auth_events,
| Err(e) => {
// This event is not well-formed and cannot be used.
// Add it to the unusable events set to avoid trying
// it again.
// We don't reject here because the server we pulled
// from might just be lying.
warn!(
elapsed=?start.elapsed(),
?e,
"event {target_id} is malformed (bad auth_events), skipping"
);
self.hit_failed_pdu_pull(target_id.clone());
unusable_events.insert(target_id);
continue;
},
};
let mut have_all_auth = true;
for auth_event_id in auth_events {
if let Ok(local_pdu) = self.services.timeline.get_pdu(&auth_event_id).await {
trace!(elapsed=?start.elapsed(),"Found auth event {auth_event_id} in db");
let mut obj = local_pdu.into_canonical_object();
obj.remove("event_id");
discovered_events.insert(auth_event_id.clone(), obj);
continue;
}
if discovered_events.contains_key(&auth_event_id) {
trace!(elapsed=?start.elapsed(), %auth_event_id, "Already found auth event");
continue;
}
debug!(elapsed=?start.elapsed(), "Missing auth event {auth_event_id} for event {target_id}");
todo.push_back(auth_event_id);
have_all_auth = false;
}
// Insert this PDU back at the end of the queue so that it will
// be resolved once all of its auth events have been
// fetched.
if have_all_auth {
debug!(elapsed=?start.elapsed(),%target_id, "Have all auth events");
discovered_events.insert(target_id, value);
} else {
debug_warn!(elapsed=?start.elapsed(),
"Fetched {target_id} but missing some auth events, will have to re-fetch."
);
todo.push_back(target_id);
}
// Queue this event's auth events. They are discovered by
// this same loop, and the whole result is topologically sorted
// below, so there's no need to hold this event back until they
// arrive.
todo.extend(auth_events);
discovered_events.insert(target_id, value);
}
}
// For each discovered event, handle it as an outlier, returning the
// parsed PDU map.
let seeded_ordered = build_local_dag(&discovered_events, DagBuilderTree::AuthEvents)
.await
.expect("failed to build local DAG");
+11 -1
View File
@@ -1,7 +1,7 @@
use std::{collections::HashMap, time::Instant};
use conduwuit::{
Event, PduEvent, debug, debug_info, debug_warn, trace,
Err, Event, PduEvent, debug, debug_info, debug_warn, trace,
utils::{BoolExt, IterStream, stream::BroadbandExt},
};
use futures::StreamExt;
@@ -45,6 +45,16 @@ impl super::Service {
.get_forward_extremities(room_id)
.collect::<Vec<_>>()
.await;
if tail.is_empty() {
// Without any forward extremities, we don't know what the latest
// events are in the room, and consequently have no idea what we
// might be missing.
//
// See also: https://forgejo.ellis.link/continuwuation/continuwuity/issues/2259
return Err!(
"No forward extremities found in {room_id}, unable to determine horizon"
);
}
let mut gapfilled = self
.get_missing_events(
+4 -12
View File
@@ -123,7 +123,7 @@ impl super::Service {
match self
.services
.server_keys
.verify_event(&pdu_json, room_version_rules)
.verify_event_json(&pdu_json, room_version_rules)
.await
{
| Ok(ruma::signatures::Verified::All) => {
@@ -160,15 +160,9 @@ impl super::Service {
ready(auth_events_by_key.get(&key).map(ToOwned::to_owned))
};
state_res::event_auth::auth_check(
room_version_rules,
pdu,
None, // TODO: third party invite
state_fetch,
create_event,
)
.await
.map_err(|e| err!("Event self-authentication failed: {e:?}"))
state_res::event_auth::auth_check(room_version_rules, pdu, state_fetch, create_event)
.await
.map_err(|e| err!("Event self-authentication failed: {e:?}"))
}
/// Checks that the event passes PDU check 5, which ensures that the event
@@ -242,7 +236,6 @@ impl super::Service {
let auth_check = state_res::event_auth::auth_check(
room_version_rules,
incoming_pdu,
None, // TODO: third party invite
|ty, sk| state_fetch(ty.clone(), sk.into()),
create_event.as_pdu(),
)
@@ -292,7 +285,6 @@ impl super::Service {
state_res::event_auth::auth_check(
room_version_rules,
incoming_pdu,
None, // third-party invite
state_fetch,
create_event.as_pdu(),
)
+3 -4
View File
@@ -492,7 +492,7 @@ impl Service {
let resp_auth = &resp_events.auth_chain;
self.services
.server_keys
.acquire_events_pubkeys(resp_auth.iter().chain(resp_state.iter()))
.get_events_keys(resp_auth.iter().chain(resp_state.iter()), &room_version_rules)
.await;
info!("Going through send_join response room_state");
@@ -505,7 +505,7 @@ impl Service {
.then(|pdu| {
self.services
.server_keys
.validate_and_add_event_id_no_fetch(pdu, &room_version_rules)
.verify_event_json_no_fetch_add_event_id(pdu, &room_version_rules)
.inspect_err(|e| {
debug_warn!(
"Could not validate send_join response room_state event: {e:?}"
@@ -558,7 +558,7 @@ impl Service {
.then(|pdu| {
self.services
.server_keys
.validate_and_add_event_id_no_fetch(pdu, &room_version_rules)
.verify_event_json_no_fetch_add_event_id(pdu, &room_version_rules)
})
.ready_filter_map(Result::ok)
.ready_for_each(|(event_id, value)| {
@@ -582,7 +582,6 @@ impl Service {
let auth_check = state_res::event_auth::auth_check(
&room_version.rules().unwrap(),
&parsed_join_pdu,
None, // TODO: third party invite
|k, s| state_fetch(k.clone(), s.into()),
&state_fetch(StateEventType::RoomCreate, "".into())
.await
+8 -2
View File
@@ -2,7 +2,8 @@ use std::{mem::size_of, sync::Arc};
use conduwuit::{
arrayvec::ArrayVec,
matrix::{Event, PduCount},
matrix::PduCount,
result::LogErr,
utils::{
ReadyExt,
stream::{TryIgnore, WidebandExt},
@@ -94,7 +95,12 @@ impl Data {
let mut pdu = self.services.timeline.get_pdu_from_id(&pdu_id).await.ok()?;
pdu.as_mut_pdu().set_unsigned(Some(user_id));
let ctx = self
.services
.timeline
.get_unsigned_context(&pdu, Some(user_id))
.await;
pdu.set_unsigned(Some(ctx)).log_err().ok();
Some((shorteventid, pdu))
})
+9 -3
View File
@@ -5,6 +5,7 @@ use conduwuit::{
arrayvec::ArrayVec,
debug_warn,
matrix::event::{Event, Matches},
result::LogErr,
utils::{
ArrayVecExt, IterStream, ReadyExt, set,
stream::{TryIgnore, WidebandExt},
@@ -136,10 +137,15 @@ impl Service {
})
.skip(query.skip)
.take(query.limit)
.map(move |mut pdu| {
pdu.set_unsigned(query.user_id);
.wide_filter_map(async move |mut pdu| {
let ctx = self
.services
.timeline
.get_unsigned_context(&pdu, Some(sender_user))
.await;
pdu.set_unsigned(Some(ctx)).log_err().ok();
pdu
Some(pdu)
})
.then(async move |mut pdu| {
if let Err(e) = self
+12 -4
View File
@@ -77,12 +77,20 @@ impl super::Service {
room_id: &RoomId,
event_id: &EventId,
) -> bool {
let Ok(shortstatehash) = self.pdu_shortstatehash(event_id).await else {
return true;
};
let currently_member = self.services.state_cache.is_joined(user_id, room_id).await;
let Ok(shortstatehash) = self.pdu_shortstatehash(event_id).await else {
// We haven't calculated the state at this event, so we can't
// properly history visibility check it. This happens for outliers
// and backfilled events. Since continuwuity is primarily targeting
// single-user/trusted deployments, an acceptable compromise for now
// is to ensure that the user is currently joined to the room.
// Federation fails closed here, but if that was replicated for the
// check used for client-server, nobody would be allowed to see
// events before their server started participating in the room.
return currently_member;
};
let history_visibility = self
.state_get_content(shortstatehash, &StateEventType::RoomHistoryVisibility, "")
.await
+6 -5
View File
@@ -550,11 +550,12 @@ impl Service {
.services
.state_cache
.is_joined(querying_user, &summary.room_id)
.await || self
.services
.state_cache
.is_invited(querying_user, &summary.room_id)
.await)
.await
|| self
.services
.state_cache
.is_invited(querying_user, &summary.room_id)
.await)
{
return true;
}
+124 -35
View File
@@ -1,8 +1,12 @@
use std::{collections::BTreeMap, sync::Arc};
use conduwuit::result::LogErr;
use conduwuit_core::{
Event, Result, err,
matrix::pdu::{PduCount, PduEvent, PduId, RawPduId},
matrix::{
event::RelationTypeEqual,
pdu::{PduCount, PduEvent, PduId, RawPduId},
},
utils::{
ReadyExt,
stream::{TryIgnore, WidebandExt},
@@ -11,9 +15,14 @@ use conduwuit_core::{
use conduwuit_database::{Deserialized, Map};
use futures::{Stream, StreamExt};
use ruma::{
CanonicalJsonValue, EventId, OwnedUserId, RoomId, UserId,
api::client::threads::get_threads::v1::IncludeThreads, events::relation::BundledThread,
serde::Raw, uint,
CanonicalJsonObject, CanonicalJsonValue, EventId, OwnedUserId, RoomId, UInt, UserId,
api::{Direction, client::threads::get_threads::v1::IncludeThreads},
events::{
AnySyncMessageLikeEvent, AnySyncTimelineEvent,
relation::{BundledThread, RelationType},
},
serde::Raw,
uint,
};
use serde_json::json;
@@ -26,6 +35,7 @@ pub struct Service {
struct Services {
short: Dep<rooms::short::Service>,
pdu_metadata: Dep<rooms::pdu_metadata::Service>,
timeline: Dep<rooms::timeline::Service>,
}
@@ -41,6 +51,7 @@ impl crate::Service for Service {
},
services: Services {
short: args.depend::<rooms::short::Service>("rooms::short"),
pdu_metadata: args.depend::<rooms::pdu_metadata::Service>("rooms::pdu_metadata"),
timeline: args.depend::<rooms::timeline::Service>("rooms::timeline"),
},
}))
@@ -50,6 +61,88 @@ impl crate::Service for Service {
}
impl Service {
/// Serializes an event for the `latest_event` of a thread's aggregation,
/// stripping `unsigned` as we don't want to send e.g. txnid to clients.
fn latest_event<E>(event: &E) -> Raw<AnySyncMessageLikeEvent>
where
E: Event,
{
let mut latest_pdu = event.clone().into_pdu();
latest_pdu.unsigned = None;
latest_pdu
.into_format::<Raw<AnySyncTimelineEvent>>()
.cast_unchecked()
}
fn set_thread(unsigned: &mut CanonicalJsonObject, thread: BundledThread) {
let content = serde_json::to_value(thread).expect("to_value always works");
unsigned.insert(
"m.relations".to_owned(),
json!({ "m.thread": content })
.try_into()
.expect("thread is valid json"),
);
}
/// Remove `event` from the aggregation of the thread matching
/// `root_event_id`
pub async fn remove_from_thread<E>(&self, root_event_id: &EventId, event: &E) -> Result
where
E: Event + Send + Sync,
{
let Ok(root_id) = self.services.timeline.get_pdu_id(root_event_id).await else {
return Ok(());
};
let mut root_pdu_json = self
.services
.timeline
.get_pdu_json_from_id(&root_id)
.await
.map_err(|e| err!(Database("Thread root pdu not found: {e:?}")))?;
let Some(CanonicalJsonValue::Object(unsigned)) = root_pdu_json.get_mut("unsigned") else {
return Ok(());
};
let thread: Vec<_> = self
.services
.pdu_metadata
.get_relations(
event.sender(),
&event.room_id_or_hash(),
root_event_id,
PduCount::max(),
usize::MAX,
0,
Direction::Backward,
)
.await
.into_iter()
.filter(|(_, pdu)| RelationType::Thread.relation_type_equal(pdu))
.collect();
match thread.first() {
| Some((_, latest)) => Self::set_thread(
unsigned,
BundledThread::new(
Self::latest_event(latest),
thread.len().try_into().unwrap_or(UInt::MAX),
true,
),
),
| None => {
// No events are left in the thread, or the root is backfilled
unsigned.remove("m.relations");
},
}
self.services
.timeline
.replace_pdu(&root_id, &root_pdu_json)
.await
}
pub async fn add_to_thread<E>(&self, root_event_id: &EventId, event: &E) -> Result
where
E: Event + Send + Sync,
@@ -77,11 +170,20 @@ impl Service {
.await
.map_err(|e| err!(Request(InvalidParam("Thread root pdu not found: {e:?}"))))?;
if let CanonicalJsonValue::Object(unsigned) = root_pdu_json
.entry("unsigned".to_owned())
.or_insert_with(|| CanonicalJsonValue::Object(BTreeMap::default()))
// get_relations doesn't support backfilled targets, so redactions
// couldn't update the aggregation of a backfilled root
//
// TODO: Support backfilled threads. This needs
// `src/service/rooms/pdu_metadata/mod.rs`'s
// (TODO: Relations with backfilled pdus) to be implemented,
// and then this check can be removed.
if matches!(root_id.pdu_count(), PduCount::Normal(_))
&& let CanonicalJsonValue::Object(unsigned) = root_pdu_json
.entry("unsigned".to_owned())
.or_insert_with(|| CanonicalJsonValue::Object(BTreeMap::default()))
{
if let Some(mut relations) = unsigned
let latest_event = Self::latest_event(event);
let thread = match unsigned
.get("m.relations")
.and_then(|r| r.as_object())
.and_then(|r| r.get("m.thread"))
@@ -89,34 +191,16 @@ impl Service {
serde_json::from_value::<BundledThread>(relations.clone().into()).ok()
}) {
// Thread already existed
relations.count = relations.count.saturating_add(uint!(1));
relations.latest_event = Raw::from_json(event.content().to_owned());
let content = serde_json::to_value(relations).expect("to_value always works");
unsigned.insert(
"m.relations".to_owned(),
json!({ "m.thread": content })
.try_into()
.expect("thread is valid json"),
);
} else {
| Some(mut thread) => {
thread.count = thread.count.saturating_add(uint!(1));
thread.latest_event = latest_event;
thread
},
// New thread
let relations = BundledThread::new(
Raw::from_json(event.content().to_owned()),
uint!(1),
true,
);
| None => BundledThread::new(latest_event, uint!(1), true),
};
let content = serde_json::to_value(relations).expect("to_value always works");
unsigned.insert(
"m.relations".to_owned(),
json!({ "m.thread": content })
.try_into()
.expect("thread is valid json"),
);
}
Self::set_thread(unsigned, thread);
self.services
.timeline
@@ -164,7 +248,12 @@ impl Service {
let mut pdu = self.services.timeline.get_pdu_from_id(&pdu_id).await.ok()?;
let pdu_id: PduId = pdu_id.into();
pdu.as_mut_pdu().set_unsigned(Some(user_id));
let ctx = self
.services
.timeline
.get_unsigned_context(&pdu, Some(user_id))
.await;
pdu.set_unsigned(Some(ctx)).log_err().ok();
Some((pdu_id.shorteventid, pdu))
});
-11
View File
@@ -336,17 +336,6 @@ impl super::Service {
else {
return;
};
unsigned.insert(
"prev_content".to_owned(),
CanonicalJsonValue::Object(
utils::to_canonical_object(prev_state.get_content_as_value())
.expect("Failed to convert prev_content into canonical JSON object"),
),
);
unsigned.insert(
String::from("prev_sender"),
CanonicalJsonValue::String(prev_state.sender().to_string()),
);
unsigned.insert(
String::from("replaces_state"),
CanonicalJsonValue::String(prev_state.event_id().to_string()),
+25 -3
View File
@@ -120,6 +120,13 @@ impl super::Service {
return Err!(Request(NotFound("No one can backfill this PDU, room is empty.")));
}
self.services
.event_handler
.ensure_can_pull_event(event_id)
.inspect_err(|e| {
debug_warn!(%event_id, error=?e, "Not attempting to fetch remote PDU");
})?;
let servers = self.candidate_backfill_servers(room_id).await;
for backfill_server in servers {
@@ -145,7 +152,12 @@ impl super::Service {
.event_handler
.handle_incoming_pdu(&backfill_server, room_id, event_id, value, true)
.boxed()
.await?;
.await
.inspect_err(|_| {
self.services
.event_handler
.hit_failed_pdu_pull(event_id.to_owned());
})?;
debug!("Successfully backfilled {event_id} from {backfill_server}");
Some(self.get_pdu(event_id).await)
},
@@ -156,10 +168,14 @@ impl super::Service {
};
if let Some(pdu) = pdu {
debug!("Fetched {event_id} from {backfill_server}");
self.services.event_handler.clear_failed_pdu(event_id);
return pdu;
}
}
self.services
.event_handler
.hit_failed_pdu_pull(event_id.to_owned());
Err!("No servers could be used to fetch {} in {}.", room_id, event_id)
}
@@ -272,8 +288,14 @@ impl super::Service {
}
// Insert all trusted servers in the config
candidate_backfill_servers
.extend(self.services.server.config.trusted_servers.iter().cloned());
candidate_backfill_servers.extend(
self.services
.server
.config
.trusted_servers
.iter()
.map(|n| n.server_name().to_owned()),
);
// Remove our own name, we can't request backfill from ourselves
candidate_backfill_servers.remove(self.services.globals.server_name());
+4 -9
View File
@@ -252,15 +252,10 @@ impl super::Service {
| _ => create_pdu.as_ref().unwrap().as_pdu(),
};
let auth_check = state_res::auth_check(
&room_version_rules,
&pdu,
None, // TODO: third_party_invite
auth_fetch,
create_event,
)
.await
.map_err(|e| err!(Request(Forbidden(warn!("Auth check failed: {e:?}")))))?;
let auth_check =
state_res::auth_check(&room_version_rules, &pdu, auth_fetch, create_event)
.await
.map_err(|e| err!(Request(Forbidden(warn!("Auth check failed: {e:?}")))))?;
if !auth_check {
return Err!(Request(Forbidden("Event is not authorized.")));
+93 -4
View File
@@ -1,13 +1,14 @@
//! Helpers for submitting events with the right checks performed
use conduwuit::{
Err, Result, err,
matrix::pdu::{PartialPdu, sticky},
pub(crate) use conduwuit::{
Err, Event, Result, err,
matrix::pdu::{PartialPdu, UnsignedContext, sticky},
trace,
};
use ruma::{
MilliSecondsSinceUnixEpoch, OwnedEventId, RoomId, UserId,
events::{
AnyStateEventContent, StateEventType,
AnyStateEventContent, AnyTimelineEvent, StateEventType,
room::{
canonical_alias::RoomCanonicalAliasEventContent,
history_visibility::{HistoryVisibility, RoomHistoryVisibilityEventContent},
@@ -325,4 +326,92 @@ impl super::Service {
Ok(())
}
/// Fetches contextual unsigned information, like prev content, membership,
/// and redacted because.
#[tracing::instrument(skip_all, fields(event_id=%event.event_id(), ?user_id))]
pub async fn get_unsigned_context<'a, E>(
&self,
event: E,
user_id: Option<&'a UserId>,
) -> UnsignedContext<'a>
where
E: Event,
{
let ssh = self
.services
.state_accessor
.pdu_shortstatehash(event.event_id())
.await;
let (membership, prev_content) = if let Some(sender_user) = user_id
&& let Ok(ssh) = ssh
{
trace!("Fetching membership");
let membership = Some(
self.services
.state_accessor
.user_membership(ssh, sender_user)
.await,
);
let prev_content = match event.get_unsigned_property::<OwnedEventId>("replaces_state")
{
| Ok(event_id) => {
// Only include prev content if the sender is allowed to see
// the event
let can_see = self
.services
.state_accessor
.user_can_see_event(sender_user, &event.room_id_or_hash(), &event_id)
.await;
if can_see {
trace!(prev_event_id=%event_id, "Fetching prev content");
self.get_pdu(&event_id).await.map(|p| p.content).ok()
} else {
trace!(
prev_event_id=%event_id,
"Not fetching prev content as sender cannot see the event"
);
None
}
},
| _ => None,
};
(membership, prev_content)
} else {
trace!(?ssh, "No shortstatehash, cannot fetch prev content or membership");
(None, None)
};
// N.B. Theoretically, the redaction event should be included in a
// redacted capacity if the requesting user isn't allowed to see the
// redaction event. However, the cost of looking up that information
// is extraordinary compared to the value it would provide, so is
// deliberately not done.
let redacted_because = match event
.get_unsigned_property::<OwnedEventId>("org.continuwuity.redacted_by")
{
| Ok(event_id) => self
.get_pdu(&event_id)
.await
.map(|mut e| {
// We don't include unsigned here since the other option is
// to recursively generate unsigned data, which is
// expensive.
// Synapse seems to do that, but I don't see a use for it.
e.unsigned = None;
e
})
.map(Event::into_format::<Raw<AnyTimelineEvent>>)
.ok()
.inspect(|_| trace!(redacted_because_id=%event_id, "Fetched redacted_because")),
| _ => None,
};
UnsignedContext {
user_id,
membership,
prev_content,
redacted_because,
}
}
}
+26 -4
View File
@@ -2,10 +2,11 @@ use conduwuit_core::{
Result, err,
matrix::event::Event,
utils::{self},
warn,
};
use ruma::EventId;
use ruma::{EventId, events::room::encrypted::Relation};
use super::ExtractBody;
use super::{ExtractBody, ExtractRelatesTo};
use crate::rooms::short::ShortRoomId;
impl super::Service {
@@ -31,6 +32,12 @@ impl super::Service {
err!(Database(error!(?pdu_id, %event_id, ?e, "PDU ID points to invalid PDU.")))
})?;
let thread_root = match pdu.get_content::<ExtractRelatesTo>() {
| Ok(ExtractRelatesTo { relates_to: Relation::Thread(thread) }) =>
Some(thread.event_id),
| _ => None,
};
if let Ok(content) = pdu.get_content::<ExtractBody>() {
if let Some(body) = content.body {
self.services
@@ -45,12 +52,27 @@ impl super::Service {
.get_room_version(&pdu.room_id_or_hash())
.await?;
pdu.redact(&room_version_id, reason.to_value())?;
pdu.redact(&room_version_id, Some(reason.event_id()))?;
let obj = utils::to_canonical_object(&pdu).map_err(|e| {
err!(Database(error!(%event_id, ?e, "Failed to convert PDU to canonical JSON")))
})?;
self.replace_pdu(&pdu_id, &obj).await
self.replace_pdu(&pdu_id, &obj).await?;
// So the thread's root doesn't keep the original content in its
// `latest_event` aggregation
if let Some(thread_root) = thread_root {
self.services
.threads
.remove_from_thread(&thread_root, &pdu)
.await
.inspect_err(|e| {
warn!("Failed to remove redacted event from thread {thread_root}: {e:?}");
})
.ok();
}
Ok(())
}
}
+3 -9
View File
@@ -317,7 +317,7 @@ impl Service {
for (dest, events) in txns {
if self.server.config.startup_netburst && !events.is_empty() {
statuses.insert(dest.clone(), TransactionStatus::Running);
futures.push(self.send_events(dest.clone(), events));
futures.push(self.send_events(dest, events));
}
}
}
@@ -661,7 +661,7 @@ impl Service {
.remove(&user_id)
.expect("our read receipts always have the user here");
let receipt_data = ReceiptData::new(receipt, vec![event_id.clone()]);
let receipt_data = ReceiptData::new(receipt, vec![event_id]);
if read.insert(user_id, receipt_data).is_none() {
*num = num.saturating_add(1);
@@ -1006,16 +1006,10 @@ impl Service {
&self,
mut pdu_json: CanonicalJsonObject,
) -> impl Future<Output = Box<RawJsonValue>> {
if let Some(unsigned) = pdu_json
.get_mut("unsigned")
.and_then(|val| val.as_object_mut())
{
// TODO: remove all unsigned data over federation.
unsigned.remove("transaction_id");
}
// We don't support any room versions that have the event_id in the PDU
// JSON, so we can safely remove it here.
pdu_json.remove("event_id");
pdu_json.remove("unsigned");
std::future::ready(
to_raw_value(&pdu_json).expect("CanonicalJson is valid serde_json::Value"),
-248
View File
@@ -1,248 +0,0 @@
use std::{
borrow::Borrow,
collections::{BTreeMap, BTreeSet},
time::Duration,
};
use conduwuit::{debug, debug_error, debug_warn, error, info, result::FlatOk, trace, warn};
use futures::{StreamExt, stream::FuturesUnordered};
use ruma::{
CanonicalJsonObject, OwnedServerName, OwnedServerSigningKeyId, ServerName,
ServerSigningKeyId, api::federation::discovery::ServerSigningKeys, serde::Raw,
};
use serde_json::value::RawValue as RawJsonValue;
use tokio::time::{Instant, timeout_at};
use super::key_exists;
type Batch = BTreeMap<OwnedServerName, Vec<OwnedServerSigningKeyId>>;
impl super::Service {
/// Returns all verify keys required to verify the incoming events.
pub async fn acquire_events_pubkeys<'a, I>(&self, events: I)
where
I: Iterator<Item = &'a Box<RawJsonValue>> + Send,
{
type Batch = BTreeMap<OwnedServerName, BTreeSet<OwnedServerSigningKeyId>>;
type Signatures = BTreeMap<OwnedServerName, BTreeMap<OwnedServerSigningKeyId, String>>;
let mut batch = Batch::new();
events
.cloned()
.map(Raw::<CanonicalJsonObject>::from_json)
.map(|event| event.get_field::<Signatures>("signatures"))
.filter_map(FlatOk::flat_ok)
.flat_map(IntoIterator::into_iter)
.for_each(|(server, sigs)| {
batch.entry(server).or_default().extend(sigs.into_keys());
});
let batch = batch
.iter()
.map(|(server, keys)| (server.borrow(), keys.iter().map(Borrow::borrow)));
self.acquire_pubkeys(batch).await;
}
/// Acquires all the verify keys required in the provided batch, using
/// notaries and direct-fetch where applicable.
pub async fn acquire_pubkeys<'a, S, K>(&self, batch: S)
where
S: Iterator<Item = (&'a ServerName, K)> + Send + Clone,
K: Iterator<Item = &'a ServerSigningKeyId> + Send + Clone,
{
let notary_only = self.services.server.config.only_query_trusted_key_servers;
let notary_first_always = self.services.server.config.query_trusted_key_servers_first;
let notary_first_on_join = self
.services
.server
.config
.query_trusted_key_servers_first_on_join;
let requested_servers = batch.clone().count();
let requested_keys = batch.clone().flat_map(|(_, key_ids)| key_ids).count();
debug!("acquire {requested_keys} keys from {requested_servers}");
let mut missing = self.acquire_locals(batch).await;
let mut missing_keys = keys_count(&missing);
let mut missing_servers = missing.len();
if missing_servers == 0 {
return;
}
info!("{missing_keys} keys for {missing_servers} servers will be acquired");
if notary_first_always || notary_first_on_join {
missing = self.acquire_notary(missing.into_iter()).await;
missing_keys = keys_count(&missing);
missing_servers = missing.len();
if missing_keys == 0 {
return;
}
warn!(
"missing {missing_keys} keys for {missing_servers} servers from all notaries \
first"
);
}
if !notary_only {
missing = self.acquire_origins(missing.into_iter()).await;
missing_keys = keys_count(&missing);
missing_servers = missing.len();
if missing_keys == 0 {
return;
}
debug_warn!("missing {missing_keys} keys for {missing_servers} servers unreachable");
}
if !notary_first_always && !notary_first_on_join {
missing = self.acquire_notary(missing.into_iter()).await;
missing_keys = keys_count(&missing);
missing_servers = missing.len();
if missing_keys == 0 {
return;
}
debug_warn!(
"still missing {missing_keys} keys for {missing_servers} servers from all \
notaries."
);
}
if missing_keys > 0 {
warn!(
"did not obtain {missing_keys} keys for {missing_servers} servers out of \
{requested_keys} total keys for {requested_servers} total servers."
);
}
for (server, key_ids) in missing {
debug_warn!(%server, ?key_ids, "missing");
}
}
/// Filters the batch to return only keys which we do not already have
/// locally.
async fn acquire_locals<'a, S, K>(&self, batch: S) -> Batch
where
S: Iterator<Item = (&'a ServerName, K)> + Send,
K: Iterator<Item = &'a ServerSigningKeyId> + Send,
{
let mut missing = Batch::new();
for (server, key_ids) in batch {
for key_id in key_ids {
if !self.verify_key_exists(server, key_id).await {
missing
.entry(server.into())
.or_default()
.push(key_id.into());
}
}
}
missing
}
/// Fetches verification keys from the origin servers directly, returning
/// any keys that could not be fetched.
async fn acquire_origins<I>(&self, batch: I) -> Batch
where
I: Iterator<Item = (OwnedServerName, Vec<OwnedServerSigningKeyId>)> + Send,
{
let timeout = Instant::now()
.checked_add(Duration::from_secs(45))
.expect("timeout overflows");
let mut requests: FuturesUnordered<_> = batch
.map(|(origin, key_ids)| self.acquire_origin(origin, key_ids, timeout))
.collect();
let mut missing = Batch::new();
while let Some((origin, key_ids)) = requests.next().await {
if !key_ids.is_empty() {
missing.insert(origin, key_ids);
}
}
missing
}
/// Fetches verification keys from an origin directly, returning any keys
/// that were fetched.
async fn acquire_origin(
&self,
origin: OwnedServerName,
mut key_ids: Vec<OwnedServerSigningKeyId>,
timeout: Instant,
) -> (OwnedServerName, Vec<OwnedServerSigningKeyId>) {
match timeout_at(timeout, self.server_request(&origin)).await {
| Err(e) => debug_warn!(%origin, "timed out: {e}"),
| Ok(Err(e)) => debug_error!(%origin, "{e}"),
| Ok(Ok(server_keys)) => {
trace!(
%origin,
?key_ids,
?server_keys,
"received server_keys"
);
self.add_signing_keys(server_keys.clone()).await;
key_ids.retain(|key_id| !key_exists(&server_keys, key_id));
},
}
(origin, key_ids)
}
/// Fetches signing keys for the given batch from the configured notary
/// servers, returning any keys that could not be fetched.
async fn acquire_notary<I>(&self, batch: I) -> Batch
where
I: Iterator<Item = (OwnedServerName, Vec<OwnedServerSigningKeyId>)> + Send,
{
let mut missing: Batch = batch.collect();
for notary in self.services.globals.trusted_servers() {
let missing_keys = keys_count(&missing);
let missing_servers = missing.len();
debug!(
"Asking notary {notary} for {missing_keys} missing keys from {missing_servers} \
servers"
);
let batch = missing
.iter()
.map(|(server, keys)| (server.borrow(), keys.iter().map(Borrow::borrow)));
match self.batch_notary_request(notary, batch).await {
| Err(e) => error!("Failed to contact notary {notary:?}: {e}"),
| Ok(results) =>
for server_keys in results {
self.acquire_notary_result(&mut missing, server_keys).await;
},
}
}
missing
}
/// Contacts a notary and fetches signing keys for the given batch, removing
/// keys from the missing batch when they are found.
async fn acquire_notary_result(&self, missing: &mut Batch, server_keys: ServerSigningKeys) {
let server = &server_keys.server_name;
self.add_signing_keys(server_keys.clone()).await;
if let Some(key_ids) = missing.get_mut(server) {
key_ids.retain(|key_id| !key_exists(&server_keys, key_id));
if key_ids.is_empty() {
missing.remove(server);
}
}
}
}
fn keys_count(batch: &Batch) -> usize {
batch.values().flat_map(|key_ids| key_ids.iter()).count()
}
+491 -111
View File
@@ -1,146 +1,526 @@
use std::borrow::Borrow;
/// get.rs handles actually acquiring keys from remote servers.
use std::collections::BTreeMap;
use conduwuit::{Err, Result, trace};
use assign::assign;
use conduwuit::{Result, debug, debug_error, debug_info, debug_warn, err, info, trace};
use futures::{StreamExt, stream::FuturesUnordered};
use ruma::{
CanonicalJsonObject, ServerName, ServerSigningKeyId, api::federation::discovery::VerifyKey,
CanonicalJsonObject, CanonicalJsonValue, MilliSecondsSinceUnixEpoch, ServerName,
ServerSigningKeyId, UInt,
api::federation::discovery::{
ServerSigningKeys, VerifyKey, get_remote_server_keys_batch::v2::QueryCriteria,
},
room_version_rules::RoomVersionRules,
serde::Base64,
uint,
};
use serde_json::value::RawValue;
use tokio::select;
use super::{PubKeyMap, PubKeys, extract_key};
use crate::server_keys::util::required_keys;
use super::{PubKeyMap, in_one_week};
use crate::server_keys::{request::ServerKeysQuery, util::required_keys};
impl super::Service {
/// Extracts the minimum timestamp within which a signing key must be valid
/// for it to be able to verify this event.
pub(super) fn min_valid_ts_from_event(
object: &CanonicalJsonObject,
) -> Result<MilliSecondsSinceUnixEpoch> {
object
.get("origin_server_ts")
.and_then(CanonicalJsonValue::as_integer)
.map(i64::from)
.map(|ts| {
MilliSecondsSinceUnixEpoch(UInt::new_saturating(
u64::try_from(ts).unwrap_or_default(),
))
})
.ok_or_else(|| err!(Request(BadJson("Missing origin_server_ts in event"))))
}
/// Fetches the verify keys required to verify the incoming event.
///
/// Only returns an error if the `min_valid_ts` cannot be inferred from the
/// object, or if calculating which keys are required fails (malformed
/// event).
///
/// Keys which cannot be fetched are not included in the resulting map, but
/// will not cause an error.
pub async fn get_event_keys(
&self,
object: &CanonicalJsonObject,
room_version_rules: &RoomVersionRules,
) -> Result<PubKeyMap> {
let required = match required_keys(object, &room_version_rules.signatures) {
| Ok(required) => required,
| Err(e) => {
return Err!(BadServerResponse(
"Failed to determine keys required to verify event: {e}"
));
},
};
let min_valid_ts = Self::min_valid_ts_from_event(object)?;
let required = required_keys(object, &room_version_rules.signatures).map_err(|e| {
err!(BadServerResponse("Failed to determine keys required to verify event: {e}"))
})?;
let batch = required
.iter()
.map(|(s, ids)| (s.borrow(), ids.iter().map(Borrow::borrow)));
.into_iter()
.map(|(s, ids)| {
(
s,
ids.into_iter()
.map(|key_id| (key_id, new_query_criteria(min_valid_ts)))
.collect::<BTreeMap<_, _>>(),
)
})
.collect::<ServerKeysQuery>();
Ok(self.get_pubkeys(batch).await)
Ok(self.fetch_verify_keys(batch).await)
}
/// Fetches all verify keys for the given batch. Any keys which could not be
/// fetched are elided from the resulting map.
pub async fn get_pubkeys<'a, S, K>(&self, batch: S) -> PubKeyMap
where
S: Iterator<Item = (&'a ServerName, K)> + Send,
K: Iterator<Item = &'a ServerSigningKeyId> + Send,
{
let mut keys = PubKeyMap::new();
for (server, key_ids) in batch {
let pubkeys = self.get_pubkeys_for(server, key_ids).await;
keys.insert(server.into(), pubkeys);
}
keys
}
/// Gets verify keys for the given server name and a set of key IDs. Any
/// keys that cannot be found are elided from the resulting set.
pub async fn get_pubkeys_for<'a, I>(&self, origin: &ServerName, key_ids: I) -> PubKeys
where
I: Iterator<Item = &'a ServerSigningKeyId> + Send,
{
let mut keys = PubKeys::new();
for key_id in key_ids {
if let Ok(verify_key) = self.get_verify_key(origin, key_id).await {
keys.insert(key_id.into(), verify_key.key);
}
}
keys
}
/// Fetches a single verify key via origin and notaries.
pub async fn get_verify_key(
/// Fetches the verify keys required to verify all the incoming events.
pub async fn get_events_keys(
&self,
origin: &ServerName,
key_id: &ServerSigningKeyId,
) -> Result<VerifyKey> {
let notary_first = self.services.server.config.query_trusted_key_servers_first;
let notary_only = self.services.server.config.only_query_trusted_key_servers;
if let Some(result) = self.verify_keys_for(origin).await.remove(key_id) {
trace!("Found key in cache");
return Ok(result);
}
if notary_first {
if let Ok(result) = self.get_verify_key_from_notaries(origin, key_id).await {
return Ok(result);
}
}
if !notary_only {
if let Ok(result) = self.get_verify_key_from_origin(origin, key_id).await {
return Ok(result);
}
}
if !notary_first {
if let Ok(result) = self.get_verify_key_from_notaries(origin, key_id).await {
return Ok(result);
}
}
Err!(BadServerResponse(debug_error!(
%key_id,
%origin,
"Failed to fetch federation signing-key"
)))
}
/// Fetches verify keys from notary servers, adding them to the cache, and
/// returning the requested key if found.
async fn get_verify_key_from_notaries(
&self,
origin: &ServerName,
key_id: &ServerSigningKeyId,
) -> Result<VerifyKey> {
for notary in self.services.globals.trusted_servers() {
if let Ok(server_keys) = self.notary_request(notary, origin).await {
for server_key in server_keys.clone() {
self.add_signing_keys(server_key).await;
raw_objects: impl Iterator<Item = &Box<RawValue>>,
room_version_rules: &RoomVersionRules,
) -> PubKeyMap {
let mut batch = ServerKeysQuery::new();
for raw_object in raw_objects {
let object: CanonicalJsonObject = match serde_json::from_str(raw_object.get()) {
| Ok(object) => object,
| Err(e) => {
debug_error!(?raw_object, error=?e, "failed to parse incoming event");
continue;
},
};
let min_valid_ts = match Self::min_valid_ts_from_event(&object) {
| Ok(ts) => ts,
| Err(e) => {
debug!(?object, error=?e, "incoming event has no `origin_server_ts` field, skipping");
continue;
},
};
let required = match required_keys(&object, &room_version_rules.signatures) {
| Ok(required) => required,
| Err(e) => {
debug_warn!(
?object,
error=?e,
"failed to determine required verify keys for incoming event, skipping"
);
continue;
},
};
for (server_name, signing_key_ids) in required {
let batch_entry = batch.entry(server_name).or_default();
for key_id in signing_key_ids {
batch_entry
.entry(key_id)
.and_modify(|criteria| {
// We want to use the lower of the two minimum valid
// timestamps if the key is already existing;
// otherwise, just insert this one.
criteria.minimum_valid_until_ts = Some(
criteria
.minimum_valid_until_ts
.expect("We never ask for no min valid ts")
.min(min_valid_ts),
);
})
.or_insert_with(|| new_query_criteria(min_valid_ts));
}
}
}
for server_key in server_keys {
if let Some(result) = extract_key(server_key, key_id) {
return Ok(result);
self.fetch_verify_keys(batch).await
}
/// Fetches verify keys matching the required batch parameters.
///
/// First attempts to fulfil any requirements from the local server cache,
/// falling back to asking origins and notaries.
///
/// Origins and notaries are always asked in parallel, and will complement
/// each other if notaries are not prioritised, using a
/// first-come-first-serve stream consumption approach.
///
/// If notaries are prioritised, this function will instead drain all notary
/// futures until all keys are found, or there are no more notary responses.
/// In that case, the origin future(s) will be consumed.
///
/// If there are still unacquired keys, a debug info log will be emitted
/// containing further details, but an error will not be returned.
async fn fetch_verify_keys(&self, batch: ServerKeysQuery) -> PubKeyMap {
let mut still_missing = batch.clone();
let mut result = PubKeyMap::new();
trace!(?batch, "Preparing to fetch verify keys");
// First, filter out any keys we already have locally.
for (origin, keys) in &batch {
for (key_id, criteria) in keys {
let min_valid_ts = criteria
.minimum_valid_until_ts
.unwrap_or_else(|| MilliSecondsSinceUnixEpoch(uint!(0)));
if let Some(server_keys) = self.get_signing_key(origin, key_id).await
&& let Some(key) =
verify_key_from_response(key_id, &server_keys, min_valid_ts)
{
trace!(%origin, %key_id, ?min_valid_ts, "Verify key exists locally");
Self::track_one_result(
&mut result,
&mut still_missing,
origin,
key_id,
key.key,
);
continue;
}
trace!(%origin, %key_id, ?min_valid_ts, "Did not find verify key locally");
}
}
if still_missing.is_empty() {
trace!("Not missing any more keys!");
return result;
}
trace!(?still_missing, "Still missing some keys after local population");
// Now we need to fetch keys en masse.
// For each server in the set, we'll ping off a request to the origin to
// fetch their signing keys. Then, in parallel, we'll ask every notary
// we know for the full batch. If the server is configured to ask the
// origin first, we'll simply await the origin request first, failing
// back to the notary futures. Otherwise, we can work through each
// response as they come back until we either get all the keys we
// need, or we run out of servers to ask.
let mut origin_futures: FuturesUnordered<_> = FuturesUnordered::new();
let mut notary_futures: FuturesUnordered<_> = FuturesUnordered::new();
for origin in still_missing.keys() {
let min_valid_ts_for_this_remote = batch
.get(origin)
.and_then(|keys| {
keys.values()
.filter_map(|criteria| criteria.minimum_valid_until_ts)
.max()
})
.unwrap_or_else(|| MilliSecondsSinceUnixEpoch(uint!(0)));
trace!(
%origin,
?min_valid_ts_for_this_remote,
"Spawning future to ask origin for their verify keys"
);
origin_futures
.push(self.origin_request(origin.to_owned(), min_valid_ts_for_this_remote));
}
self.services
.server
.config
.trusted_servers
.iter()
.for_each(|notary| {
trace!(notary=%notary.server_name(), "Asking notary for still missing keys");
notary_futures
.push(self.notary_request(notary.server_name(), still_missing.clone()));
});
let prioritise_notaries = !self.services.server.config.trusted_servers.is_empty()
&& self.services.server.config.query_trusted_key_servers_first;
// TODO: In the future, an option to establish a quorum between notaries
// may be desired. In such a case, the `still_missing { break }`
// blocks should be skipped in order to acquire all notary responses for
// later comparison.
if !prioritise_notaries {
// If we aren't prioritising notaries, we can just select futures on
// a first-come-first-serve basis.
loop {
select! {
origin_response = origin_futures.next(), if !origin_futures.is_empty() => {
let origin_response = match origin_response {
Some(Ok(v)) => v,
None => continue,
Some(Err(e)) => {
debug_warn!(err=?e, "Got error response from origin");
continue;
}
};
trace!(?origin_response, "Got signing key response from origin");
self.add_signing_keys(&origin_response);
Self::track_result(&mut result, &mut still_missing, &origin_response);
if still_missing.is_empty() {
break;
}
},
notary_response = notary_futures.next(), if !notary_futures.is_empty() => {
let notary_response = match notary_response {
Some(Ok(v)) => v,
None => continue,
Some(Err(e)) => {
debug_warn!(err=?e, "Got error response from notary");
continue;
}
};
trace!(
chunks=%notary_response.len(),
"Got signing keys response from a notary"
);
for server_keys in notary_response {
trace!(?server_keys, "Adding response from notary");
self.add_signing_keys(&server_keys);
Self::track_result(&mut result, &mut still_missing, &server_keys);
if still_missing.is_empty() {
break;
}
}
}
else => {
trace!("Origin or notary future stream(s) finished.");
break
}
}
}
} else {
// Drain notary responses first, and then sift through origins.
trace!(count=%notary_futures.len(), "Waiting for notary responses...");
while let Some(res) = notary_futures.next().await {
let Ok(res) = res else {
debug_warn!(err=?res.err(), "Got error response from notary");
continue;
};
for server_keys in res {
trace!(?server_keys, "Adding response from notary");
self.add_signing_keys(&server_keys);
Self::track_result(&mut result, &mut still_missing, &server_keys);
}
if still_missing.is_empty() {
break;
}
}
if !still_missing.is_empty() {
trace!("Still missing some keys, waiting for origin responses...");
while let Some(res) = origin_futures.next().await {
let Ok(res) = res else {
debug_warn!(err=?res.err(), "Got error response from origin");
continue;
};
trace!(response=?res, "Got signing key response from origin");
self.add_signing_keys(&res);
Self::track_result(&mut result, &mut still_missing, &res);
if still_missing.is_empty() {
break;
}
}
}
}
Err!(Request(NotFound("Failed to fetch signing-key from notaries")))
if !still_missing.is_empty() {
let missing_servers = still_missing.len();
let missing_keys = still_missing
.values()
.map(BTreeMap::len)
.fold(0_usize, usize::saturating_add);
let fetched_servers = result.len();
let fetched_keys = result
.values()
.map(BTreeMap::len)
.fold(0_usize, usize::saturating_add);
debug_info!(
%fetched_servers,
%fetched_keys,
%missing_servers,
%missing_keys,
"Could not acquire all requested signing keys"
);
trace!(?still_missing, "Could not acquire all requested signing keys");
}
result
}
/// Fetches verify keys from the origin server, adding them to the cache,
/// and returning the requested key if found.
async fn get_verify_key_from_origin(
/// Tracks the result of a signing keys response by removing each of the
/// returned verify keys and old verify keys from the missing map, and
/// adding them to the results map.
fn track_result(
results: &mut PubKeyMap,
missing: &mut ServerKeysQuery,
batch: &ServerSigningKeys,
) {
batch.verify_keys.iter().for_each(|(key_id, key)| {
Self::track_one_result(results, missing, &batch.server_name, key_id, key.key.clone());
});
batch.old_verify_keys.iter().for_each(|(key_id, key)| {
Self::track_one_result(results, missing, &batch.server_name, key_id, key.key.clone());
});
}
/// Tracks a single signing key result by adding it to the results map and
/// removing it from the missing map. If all missing keys for an origin are
/// found (the missing list is empty), the origin is removed from the
/// missing map entirely.
fn track_one_result(
results: &mut PubKeyMap,
missing: &mut ServerKeysQuery,
origin: &ServerName,
key_id: &ServerSigningKeyId,
key: Base64,
) {
results
.entry(origin.to_string())
.or_default()
.insert(key_id.to_string(), key);
missing.entry(origin.to_owned()).and_modify(|keys| {
keys.remove(key_id);
});
if missing.get(origin).is_some_and(BTreeMap::is_empty) {
missing.remove(origin);
}
}
/// Fetches a single verify key.
///
/// If a valid key cannot be found locally, the origin is queried. If it
/// fails to respond, or the desired verify key is not contained in the
/// response/expired before `min_valid_ts`, `None` is returned.
#[tracing::instrument(skip(self))]
pub async fn get_single_verify_key(
&self,
origin: &ServerName,
key_id: &ServerSigningKeyId,
) -> Result<VerifyKey> {
if let Ok(server_key) = self.server_request(origin).await {
self.add_signing_keys(server_key.clone()).await;
if let Some(result) = extract_key(server_key, key_id) {
return Ok(result);
}
min_valid_ts: MilliSecondsSinceUnixEpoch,
) -> Option<VerifyKey> {
if let Some(vk) = self
.get_signing_key(origin, key_id)
.await
.and_then(|ssk| verify_key_from_response(key_id, &ssk, min_valid_ts))
{
trace!("Found verify key locally");
return Some(vk);
}
Err!(Request(NotFound("Failed to fetch signing-key from origin")))
trace!("Local miss for single key, asking origin");
let response = self
.origin_request(origin.to_owned(), min_valid_ts)
.await
.inspect_err(|e| info!("Could not acquire signing key from origin server: {e:?}"))
.ok()?;
verify_key_from_response(key_id, &response, min_valid_ts)
.inspect(|_| self.add_signing_keys(&response))
}
}
/// Generates a QueryCriteria with the minimum_valid_until_ts set to the target
/// epoch.
fn new_query_criteria(min_valid_ts: MilliSecondsSinceUnixEpoch) -> QueryCriteria {
assign!(QueryCriteria::new(), { minimum_valid_until_ts: Some(min_valid_ts) })
}
/// Extracts a verify key with the given ID from the response.
///
/// If the provided `ServerSigningKeys` is dated to before the min valid TS,
/// this will throw an assertion panic in debug builds, or always return `None`
/// in release. If the key is found in `verify_keys`, it is returned as-is. If
/// the key is found in `old_verify_keys`, it is only returned if the expiry
/// timestamp is greater than or equal to the minimum valid timestamp.
fn verify_key_from_response(
key_id: &ServerSigningKeyId,
server_keys: &ServerSigningKeys,
min_valid_ts: MilliSecondsSinceUnixEpoch,
) -> Option<VerifyKey> {
if server_keys.valid_until_ts < min_valid_ts || server_keys.valid_until_ts > in_one_week() {
return None;
}
if let Some(key) = server_keys
.verify_keys
.iter()
.filter(|(k, _)| *k == key_id)
.map(|(_, v)| v)
.next()
{
return Some(key.to_owned());
}
server_keys
.old_verify_keys
.iter()
.filter(|(k, v)| {
*k == key_id && v.expired_ts.min(server_keys.valid_until_ts) >= min_valid_ts
})
.map(|(_, v)| v)
.next()
.map(|old_key| VerifyKey::new(old_key.key.clone()))
}
#[cfg(test)]
mod test {
use ruma::Int;
use serde_json::json;
use super::*;
use crate::server_keys::{Service, request::ServerKeysQuerySet};
#[test]
fn min_valid_ts_refuses_garbage() {
let garbage = vec![
json!("string"),
json!("0"), // string ints aren't ints
CanonicalJsonValue::Null.into(),
json!([]),
json!({}),
json!(" "),
];
for fixture in garbage {
let obj: CanonicalJsonObject =
serde_json::from_value(json!({"origin_server_ts": fixture})).unwrap();
assert!(
Service::min_valid_ts_from_event(&obj).is_err(),
"garbage should always error instead of trying to guess"
);
}
}
#[test]
fn min_valid_ts_requires_field() {
// This just ensures we don't accidentally default to 0 or something
// dumb like that
assert!(
Service::min_valid_ts_from_event(&CanonicalJsonObject::new()).is_err(),
"origin_server_ts should be a required field and thus should error if not provided"
);
}
#[test]
fn min_valid_ts_accepts_negative_values() {
// Since we convert to u64 with saturating for MSSinceEpoch, this should
// give us a final timestamp of zero (since unsigned ints cant be
// negative).
let obj: CanonicalJsonObject =
serde_json::from_value(json!({"origin_server_ts": Int::MIN})).unwrap();
assert_eq!(
Service::min_valid_ts_from_event(&obj)
.expect("Int::MIN should be a valid min timestamp")
.0,
uint!(0),
"Saturating Int::MIN to u64 should result in zero"
);
}
#[test]
fn tracking_server_result_removes_empty_server_from_missing_map() {
let server_name = ruma::owned_server_name!("example.com");
let key_id = ServerSigningKeyId::parse("ed25519:1").unwrap();
let mut missing = ServerKeysQuery::new();
missing.insert(server_name.clone(), ServerKeysQuerySet::new());
missing.entry(server_name.clone()).and_modify(|set| {
set.insert(key_id.clone(), QueryCriteria::new());
});
let mut results = PubKeyMap::new();
Service::track_one_result(
&mut results,
&mut missing,
&server_name,
&key_id,
Base64::new(Vec::new()),
);
assert!(missing.is_empty(), "missing should be empty after tracking the result");
assert!(!results.is_empty(), "results should not be empty after tracking the result");
}
}
+4 -7
View File
@@ -9,7 +9,8 @@ use super::VerifyKeys;
pub(super) fn init(db: &Arc<Database>) -> Result<(Box<Ed25519KeyPair>, VerifyKeys)> {
let keypair = load(db).inspect_err(|_e| {
error!("Keypair invalid. Deleting...");
remove(db);
let global = &db["global"];
global.remove(b"keypair");
})?;
let verify_key = VerifyKey::new(Base64::new(keypair.public_key().to_vec()));
@@ -20,6 +21,7 @@ pub(super) fn init(db: &Arc<Database>) -> Result<(Box<Ed25519KeyPair>, VerifyKey
Ok((keypair, verify_keys))
}
/// Loads the existing keypair from the database, otherwise creates a new one.
fn load(db: &Arc<Database>) -> Result<Box<Ed25519KeyPair>> {
let (version, key) = db["global"]
.get_blocking(b"keypair")
@@ -44,6 +46,7 @@ fn load(db: &Arc<Database>) -> Result<Box<Ed25519KeyPair>> {
Ok(Box::new(key))
}
/// Creates a new keypair and writes it directly to the db.
fn create(db: &Arc<Database>) -> Result<(String, Vec<u8>)> {
let keypair = Ed25519KeyPair::generate();
@@ -55,9 +58,3 @@ fn create(db: &Arc<Database>) -> Result<(String, Vec<u8>)> {
Ok(value)
}
#[inline]
fn remove(db: &Arc<Database>) {
let global = &db["global"];
global.remove(b"keypair");
}
+72 -93
View File
@@ -1,4 +1,3 @@
mod acquire;
mod get;
mod keypair;
mod request;
@@ -6,30 +5,28 @@ mod sign;
mod util;
mod verify;
use std::{collections::BTreeMap, sync::Arc, time::Duration};
use std::{collections::BTreeMap, sync::Arc};
use conduwuit::{
Result, Server,
utils::{IterStream, timepoint_from_now},
utils::{IterStream, ReadyExt, stream::TryIgnore},
};
use database::{Deserialized, Json, Map};
use futures::StreamExt;
use database::{Deserialized, Ignore, Interfix, Json, Map};
use futures::{Stream, StreamExt};
pub use request::in_one_week;
use ruma::{
CanonicalJsonObject, MilliSecondsSinceUnixEpoch, OwnedServerSigningKeyId, ServerName,
ServerSigningKeyId,
api::federation::discovery::{ServerSigningKeys, VerifyKey},
room_version_rules::RoomVersionRules,
serde::Raw,
signatures::{Ed25519KeyPair, PublicKeyMap, PublicKeySet},
};
use serde_json::value::RawValue as RawJsonValue;
use crate::{Dep, globals, sending, server_keys::util::required_keys};
pub struct Service {
keypair: Box<Ed25519KeyPair>,
verify_keys: VerifyKeys,
minimum_valid: Duration,
services: Services,
db: Data,
}
@@ -41,7 +38,7 @@ struct Services {
}
struct Data {
server_signingkeys: Arc<Map>,
servernamekeyid_response: Arc<Map>,
}
pub type VerifyKeys = BTreeMap<OwnedServerSigningKeyId, VerifyKey>;
@@ -50,22 +47,19 @@ pub type PubKeys = PublicKeySet;
impl crate::Service for Service {
fn build(args: crate::Args<'_>) -> Result<Arc<Self>> {
let minimum_valid = Duration::from_hours(1);
let (keypair, verify_keys) = keypair::init(args.db)?;
debug_assert!(verify_keys.len() == 1, "only one active verify_key supported");
Ok(Arc::new(Self {
keypair,
verify_keys,
minimum_valid,
services: Services {
globals: args.depend::<globals::Service>("globals"),
sending: args.depend::<sending::Service>("sending"),
server: args.server.clone(),
},
db: Data {
server_signingkeys: args.db["server_signingkeys"].clone(),
servernamekeyid_response: args.db["servernamekeyid_response"].clone(),
},
}))
}
@@ -77,40 +71,32 @@ impl Service {
/// Fetches the server's active keypair.
pub fn keypair(&self) -> &Ed25519KeyPair { &self.keypair }
/// Fetches the server's active signing key ID.
pub fn active_key_id(&self) -> &ServerSigningKeyId { self.active_verify_key().0 }
/// Fetches the server's active signing key. Panics if there's more than 1
/// active key.
pub fn active_verify_key(&self) -> (&ServerSigningKeyId, &VerifyKey) {
pub fn active_verify_key(&self) -> (OwnedServerSigningKeyId, VerifyKey) {
debug_assert!(self.verify_keys.len() <= 1, "more than one active verify_key");
self.verify_keys
.iter()
.next()
.map(|(id, key)| (id.as_ref(), key))
.map(|(id, key)| (id.to_owned(), key.to_owned()))
.expect("missing active verify_key")
}
/// Combines signing keys from the incoming response to the existing
/// database set.
async fn add_signing_keys(&self, new_keys: ServerSigningKeys) {
let origin = &new_keys.server_name;
// (timo) Not atomic, but this is not critical
let mut keys: ServerSigningKeys = self
.db
.server_signingkeys
.get(origin)
.await
.deserialized()
.unwrap_or_else(|_| {
// Just insert "now", it doesn't matter
ServerSigningKeys::new(origin.to_owned(), MilliSecondsSinceUnixEpoch::now())
});
keys.verify_keys.extend(new_keys.verify_keys);
keys.old_verify_keys.extend(new_keys.old_verify_keys);
self.db.server_signingkeys.raw_put(origin, Json(&keys));
/// Adds a signing key response to the database.
fn add_signing_keys(&self, new_keys: &ServerSigningKeys) {
// TODO(nex): this results in massive amounts of data duplication.
// Perhaps a sliding approach would be better? where we just update the
// last known response if the only thing that changed was the
// valid_until_ts. This would require a more complex data structure
// though. For now, just storing the duplicated data is fine.
for key in new_keys
.verify_keys
.keys()
.chain(new_keys.old_verify_keys.keys())
{
let db_key = (new_keys.server_name.as_str(), key.as_str());
self.db.servernamekeyid_response.put(db_key, Json(new_keys));
}
}
/// Checks if the keys required to verify an incoming PDU are already known
@@ -123,11 +109,14 @@ impl Service {
let Ok(required_keys) = required_keys(object, &room_version_rules.signatures) else {
return false;
};
let Ok(min_valid_ts) = Self::min_valid_ts_from_event(object) else {
return false;
};
required_keys
.iter()
.flat_map(|(server, key_ids)| key_ids.iter().map(move |key_id| (server, key_id)))
.stream()
.all(|(server, key_id)| self.verify_key_exists(server, key_id))
.all(|(server, key_id)| self.verify_key_exists(server, key_id, min_valid_ts))
.await
}
@@ -137,40 +126,54 @@ impl Service {
&self,
origin: &ServerName,
key_id: &ServerSigningKeyId,
minimum_valid_ts: MilliSecondsSinceUnixEpoch,
) -> bool {
type KeysMap = BTreeMap<OwnedServerSigningKeyId, Box<RawJsonValue>>;
let Ok(keys) = self
.db
.server_signingkeys
.get(origin)
.await
.deserialized::<Raw<ServerSigningKeys>>()
else {
let Some(keys) = self.get_signing_key(origin, key_id).await else {
// We have no keys for this server
return false;
};
if let Ok(Some(verify_keys)) = keys.get_field::<KeysMap>("verify_keys") {
if verify_keys.contains_key(key_id) {
return true;
}
if keys.valid_until_ts < minimum_valid_ts || keys.valid_until_ts > in_one_week() {
// This key response is stale and cannot be trusted.
return false;
}
if let Ok(Some(old_verify_keys)) = keys.get_field::<KeysMap>("old_verify_keys") {
if old_verify_keys.contains_key(key_id) {
return true;
}
if keys.verify_keys.contains_key(key_id) {
// An in-date, active key exists for this server.
return true;
}
if let Some(expired_key) = keys.old_verify_keys.get(key_id) {
// This key can still be used to verify signatures provided it
// expired within the minimum valid time window.
return expired_key.expired_ts.min(keys.valid_until_ts) >= minimum_valid_ts;
}
false
}
/// Returns all verify keys for the origin.
pub async fn get_signing_key(
&self,
origin: &ServerName,
key_id: &ServerSigningKeyId,
) -> Option<ServerSigningKeys> {
self.db
.servernamekeyid_response
.qry(&(origin.as_str(), key_id.as_str()))
.await
.deserialized::<ServerSigningKeys>()
.ok()
}
/// Returns all (active) verify keys for the origin.
pub async fn verify_keys_for(&self, origin: &ServerName) -> VerifyKeys {
let mut keys = self
.signing_keys_for(origin)
.await
.map_or(BTreeMap::new(), |keys| merge_old_keys(keys).verify_keys);
.ready_fold(BTreeMap::new(), |mut base, keys| {
base.extend(keys.verify_keys);
base
})
.await;
if self.services.globals.server_is_ours(origin) {
keys.extend(self.verify_keys.clone());
@@ -179,39 +182,15 @@ impl Service {
keys
}
/// Returns the stored server signing keys response for the origin.
pub async fn signing_keys_for(&self, origin: &ServerName) -> Result<ServerSigningKeys> {
self.db.server_signingkeys.get(origin).await.deserialized()
}
/// Returns the minimum_valid_ts for a key response from the current point
/// in time, plus the minimum valid time.
fn minimum_valid_ts(&self) -> MilliSecondsSinceUnixEpoch {
let timepoint =
timepoint_from_now(self.minimum_valid).expect("SystemTime should not overflow");
MilliSecondsSinceUnixEpoch::from_system_time(timepoint).expect("UInt should not overflow")
/// Returns the stored server signing keys responses for the origin.
pub fn signing_keys_for<'a>(
&'a self,
origin: &'a ServerName,
) -> impl Stream<Item = ServerSigningKeys> + Send + 'a {
self.db
.servernamekeyid_response
.stream_prefix(&(origin, Interfix))
.ignore_err()
.map(|(_, v): (Ignore, ServerSigningKeys)| v)
}
}
fn merge_old_keys(mut keys: ServerSigningKeys) -> ServerSigningKeys {
keys.verify_keys.extend(
keys.old_verify_keys
.clone()
.into_iter()
.map(|(key_id, old)| (key_id, VerifyKey::new(old.key))),
);
keys
}
fn extract_key(mut keys: ServerSigningKeys, key_id: &ServerSigningKeyId) -> Option<VerifyKey> {
keys.verify_keys.remove(key_id).or_else(|| {
keys.old_verify_keys
.remove(key_id)
.map(|old| VerifyKey::new(old.key))
})
}
fn key_exists(keys: &ServerSigningKeys, key_id: &ServerSigningKeyId) -> bool {
keys.verify_keys.contains_key(key_id) || keys.old_verify_keys.contains_key(key_id)
}
+185 -94
View File
@@ -1,110 +1,39 @@
use std::{collections::BTreeMap, fmt::Debug};
use std::collections::BTreeMap;
use conduwuit::{Err, Result, debug};
use get_remote_server_keys_batch::v2::Request;
use conduwuit::{Err, Result, trace, utils::millis_since_unix_epoch, warn};
use ruma::{
OwnedServerName, OwnedServerSigningKeyId, ServerName, ServerSigningKeyId,
MilliSecondsSinceUnixEpoch, OwnedServerName, OwnedServerSigningKeyId, ServerName, UInt,
api::federation::discovery::{
ServerSigningKeys, get_remote_server_keys,
get_remote_server_keys_batch::{self, v2::QueryCriteria},
ServerSigningKeys,
get_remote_server_keys_batch::{
self,
v2::{QueryCriteria, Response as NotaryResponse},
},
get_server_keys,
},
serde::Base64,
uint,
};
type RumaBatch = BTreeMap<OwnedServerName, BTreeMap<OwnedServerSigningKeyId, QueryCriteria>>;
pub(crate) type ServerKeysQuery = BTreeMap<OwnedServerName, ServerKeysQuerySet>;
pub(crate) type ServerKeysQuerySet = BTreeMap<OwnedServerSigningKeyId, QueryCriteria>;
impl super::Service {
/// Handles a batched notary request.
/// If the batch is empty, an empty vec is returned.
pub(super) async fn batch_notary_request<'a, S, K>(
/// Asks the origin server directly for its signing keys.
///
/// An error is returned if the server responds with invalid keys; this is
/// in contrast with the notary request, which simply drops such responses.
pub async fn origin_request(
&self,
notary: &ServerName,
batch: S,
) -> Result<Vec<ServerSigningKeys>>
where
S: Iterator<Item = (&'a ServerName, K)> + Send,
K: Iterator<Item = &'a ServerSigningKeyId> + Send,
{
let mut criteria = QueryCriteria::new();
criteria.minimum_valid_until_ts = Some(self.minimum_valid_ts());
let mut server_keys = batch.fold(RumaBatch::new(), |mut batch, (server, key_ids)| {
batch
.entry(server.into())
.or_default()
.extend(key_ids.map(|key_id| (key_id.into(), criteria.clone())));
batch
});
if server_keys.is_empty() {
return Ok(vec![]);
}
let mut results = Vec::new();
while let Some(batch) = server_keys
.keys()
.rev()
.take(self.services.server.config.trusted_server_batch_size)
.next_back()
.cloned()
{
let request = Request::new(server_keys.split_off(&batch));
debug!(
?notary,
?batch,
remaining = %server_keys.len(),
requesting = ?request.server_keys.keys(),
"notary request"
);
let response = self
.services
.sending
.send_unauthenticated_request(notary, request)
.await?
.server_keys
.into_iter()
.map(|key| key.deserialize())
.filter_map(Result::ok);
results.extend(response);
}
Ok(results)
}
/// Makes a request to a notary server to fetch signing keys.
pub async fn notary_request(
&self,
notary: &ServerName,
target: &ServerName,
) -> Result<impl Iterator<Item = ServerSigningKeys> + Clone + Debug + Send + use<>> {
use get_remote_server_keys::v2::Request;
let request = Request::new(target.into(), self.minimum_valid_ts());
let response = self
.services
.sending
.send_unauthenticated_request(notary, request)
.await?
.server_keys
.into_iter()
.map(|key| key.deserialize())
.filter_map(Result::ok);
Ok(response)
}
/// Makes a request to an origin server to fetch signing keys.
pub async fn server_request(&self, target: &ServerName) -> Result<ServerSigningKeys> {
target: OwnedServerName,
min_valid_ts: MilliSecondsSinceUnixEpoch,
) -> Result<ServerSigningKeys> {
use get_server_keys::v2::Request;
let server_signing_key = self
.services
.sending
.send_unauthenticated_request(target, Request::new())
.send_unauthenticated_request(&target, Request::new())
.await
.map(|response| response.server_key)
.and_then(|key| key.deserialize().map_err(Into::into))?;
@@ -113,10 +42,172 @@ impl super::Service {
return Err!(BadServerResponse(debug_warn!(
requested = ?target,
response = ?server_signing_key.server_name,
"Server responded with bogus server_name"
"Origin server responded with bogus server_name"
)));
}
if server_signing_key.valid_until_ts < min_valid_ts
|| server_signing_key.valid_until_ts > in_one_week()
{
return Err!(BadServerResponse(debug_warn!(
?target,
?min_valid_ts,
?server_signing_key.valid_until_ts,
"Origin server responded with server keys that are not in-date"
)));
}
if let Err(e) = Self::verify_server_keys_response(&server_signing_key, None) {
return Err!(BadServerResponse(debug_warn!(
?target,
"Origin server failed to self-sign key response: {e}"
)));
}
Ok(server_signing_key)
}
/// Sends a request to a notary server, asking for the specified keys for
/// the given servers.
///
/// The response is filtered before it is returned, dropping any chunks that
/// are invalid or do not match the requested keys. As a result, the request
/// may be successful, but the returned vec may be empty.
///
///
/// An error is returned if the notary server itself returns an error,
/// because of invalid data.
///
/// Does not make a request if there are no keys specified in the batch.
pub async fn notary_request(
&self,
notary: &ServerName,
batch: ServerKeysQuery,
) -> Result<Vec<ServerSigningKeys>> {
use get_remote_server_keys_batch::v2::Request;
let is_empty = batch.is_empty() || batch.values().all(BTreeMap::is_empty);
if is_empty {
return Ok(Vec::new());
}
let request = Request::new(batch.clone());
let notary_keys = self
.services
.server
.config
.trusted_servers
.iter()
.find(|n| n.server_name() == notary)
.map(|nk| nk.verify_keys().to_owned())
.unwrap_or_default();
let keys = self
.services
.sending
.send_unauthenticated_request(notary, request)
.await
.map(|response| Self::parse_notary_response(&response))?
.into_iter()
.filter(|keys| {
Self::validate_notary_chunk(&batch, keys, notary, notary_keys.as_slice())
})
.collect();
Ok(keys)
}
/// Parses a notary response, returning a vec of server signing key
/// responses. Drops any chunks that cannot be deserialized.
fn parse_notary_response(raw: &NotaryResponse) -> Vec<ServerSigningKeys> {
raw.server_keys
.iter()
.map(ruma::serde::Raw::deserialize)
.filter_map(Result::ok)
.collect()
}
/// Validates a signing keys response received from a notary. Returns false
/// if the response should be dropped, true if it should be kept.
fn validate_notary_chunk(
batch: &ServerKeysQuery,
keys: &ServerSigningKeys,
notary_name: &ServerName,
notary_keys: &[Base64],
) -> bool {
let Some(expected_keys) = batch.get(&keys.server_name) else {
trace!("Dropping signing keys chunk for unrequested server: {}", keys.server_name);
return false;
};
let min_valid_ts = expected_keys
.values()
.map(|q| {
q.minimum_valid_until_ts
.unwrap_or_else(|| MilliSecondsSinceUnixEpoch(uint!(0)))
})
.min()
.unwrap_or_else(|| MilliSecondsSinceUnixEpoch(uint!(0)));
// If this notary response is older than any of the min_valid_ts we
// requested, then we should drop it.
if keys.valid_until_ts < min_valid_ts || keys.valid_until_ts > in_one_week() {
trace!(
?min_valid_ts,
?keys.valid_until_ts,
"Dropping signing keys chunk for server {} because it is older than the minimum \
valid timestamp we requested",
keys.server_name
);
return false;
}
// Does this chunk even contain any keys we asked for?
// Note that this enforces min_valid_ts per-key, instead of using the
// lowest denominator.
let all_key_ids = keys
.verify_keys
.keys()
.chain(keys.old_verify_keys.keys())
.map(ToOwned::to_owned)
.collect::<Vec<_>>();
let interesting = expected_keys.iter().any(|(k, q)| {
all_key_ids.contains(k)
&& q.minimum_valid_until_ts
.is_none_or(|m| m <= keys.valid_until_ts)
});
if !interesting {
trace!(
"Dropping signing keys chunk for server {} because it does not contain any keys \
we requested",
keys.server_name
);
return false;
}
Self::verify_server_keys_response(keys, Some((notary_name, notary_keys)))
.inspect_err(|e| {
trace!(
?e,
?keys,
?notary_name,
?notary_keys,
"Dropping illegal server signing keys response chunk"
);
warn!(
%notary_name,
server_name=%keys.server_name,
"Dropping illegal server signing keys response chunk: {e:?}"
);
})
.is_ok()
}
}
#[must_use]
pub fn in_one_week() -> MilliSecondsSinceUnixEpoch {
MilliSecondsSinceUnixEpoch(
UInt::new_saturating(millis_since_unix_epoch()).saturating_add(uint!(604_800_000)),
)
}
// TODO(nex): Unit tests for validate_notary_chunk would be really good, but are
// hard to do by nature of time advancing, signatures, and external notaries.
// We can rely on integration tests for now.
+4 -1
View File
@@ -7,7 +7,10 @@ use ruma::{
signatures::{VerificationError, required_server_signatures_to_verify_event},
};
/// Extracts the server names and key ids to check signatures for given event.
/// Extracts the server names and key ids required to check signatures for the
/// given event.
///
/// Spec: https://spec.matrix.org/v1.19/server-server-api/#validating-hashes-and-signatures-on-received-events
pub(super) fn required_keys(
object: &CanonicalJsonObject,
rules: &SignaturesRules,
+275 -35
View File
@@ -1,44 +1,32 @@
use conduwuit::{Err, Result, matrix::event::gen_event_id_canonical_json};
use conduwuit::{
Err, Result, matrix::event::gen_event_id_canonical_json, trace, utils::to_canonical_object,
};
use ruma::{
CanonicalJsonObject, CanonicalJsonValue, OwnedEventId, room_version_rules::RoomVersionRules,
CanonicalJsonObject, CanonicalJsonValue, OwnedEventId, ServerName, SigningKeyAlgorithm,
api::federation::discovery::ServerSigningKeys,
room_version_rules::RoomVersionRules,
serde::{Base64, base64::Standard},
signatures::Verified,
};
use serde_json::value::RawValue as RawJsonValue;
impl super::Service {
/// Validates the incoming event, and then inserts the calculated event ID
/// into the event_id field.
pub async fn validate_and_add_event_id(
&self,
pdu: &RawJsonValue,
room_version_rules: &RoomVersionRules,
) -> Result<(OwnedEventId, CanonicalJsonObject)> {
let (event_id, mut value) = gen_event_id_canonical_json(pdu, room_version_rules)?;
if let Err(e) = self.verify_event(&value, room_version_rules).await {
return Err!(BadServerResponse(debug_error!(
"Event {event_id} failed verification: {e:?}"
)));
}
value.insert("event_id".into(), CanonicalJsonValue::String(event_id.as_str().into()));
Ok((event_id, value))
}
/// Validates the incoming event only using locally known verification keys.
/// Inserts the calculated event ID into the event_id field upon success.
pub async fn validate_and_add_event_id_no_fetch(
pub async fn verify_event_json_no_fetch_add_event_id(
&self,
pdu: &RawJsonValue,
room_version_rules: &RoomVersionRules,
) -> Result<(OwnedEventId, CanonicalJsonObject)> {
// TODO(nex): this function is used in two places (same file) and is
// awfully named, can we just kill it?
let (event_id, mut value) = gen_event_id_canonical_json(pdu, room_version_rules)?;
if !self.required_keys_exist(&value, room_version_rules).await {
return Err!(BadServerResponse(debug_warn!(
"Event {event_id} cannot be verified: missing keys."
"Event {event_id} cannot be verified: missing verify key(s) locally."
)));
}
if let Err(e) = self.verify_event(&value, room_version_rules).await {
if let Err(e) = self.verify_event_json(&value, room_version_rules).await {
return Err!(BadServerResponse(debug_error!(
"Event {event_id} failed verification: {e:?}"
)));
@@ -50,7 +38,7 @@ impl super::Service {
}
/// Verifies the incoming event after fetching the keys required to do so.
pub async fn verify_event(
pub async fn verify_event_json(
&self,
event: &CanonicalJsonObject,
room_version_rules: &RoomVersionRules,
@@ -59,15 +47,267 @@ impl super::Service {
ruma::signatures::verify_event(&keys, event, room_version_rules).map_err(Into::into)
}
/// Verifies an arbitrary JSON object after fetching the keys required to do
/// so.
pub async fn verify_json(
&self,
event: &CanonicalJsonObject,
room_version_rules: &RoomVersionRules,
) -> Result {
// TODO: is this actually used anywhere?
let keys = self.get_event_keys(event, room_version_rules).await?;
ruma::signatures::verify_json(&keys, event).map_err(Into::into)
/// Verifies a server signing keys response, either as the
/// result of `GET /_matrix/key/v2/server`, or a chunk from
/// `POST /_matrix/key/v2/query`.
///
/// Returns an error if the self-signature is invalid, or strict notary
/// validation is used (`notary` is non-empty) and the notary did not sign
/// the response.
///
/// `notary` must either be empty or have ONE entry. Extraneous notaries
/// will be ignored.
pub fn verify_server_keys_response(
server_keys: &ServerSigningKeys,
notary: Option<(&ServerName, &[Base64])>,
) -> Result<()> {
if let Some((notary_name, notary_keys)) = notary {
if !notary_keys.is_empty() {
Self::verify_notary_signature(server_keys, notary_name, notary_keys)?;
}
}
// NOTE: old_verify_keys aren't considered here because old keys can't
// be used to sign key responses.
let mut set = ruma::signatures::PublicKeySet::new();
server_keys.verify_keys.iter().for_each(|(key_id, key)| {
set.insert(key_id.to_string(), key.key.clone());
});
let mut pubkey_map = ruma::signatures::PublicKeyMap::new();
pubkey_map.insert(server_keys.server_name.to_string(), set);
let mut canonical = to_canonical_object(server_keys)?;
Self::strip_extraneous_signatures(&mut canonical, &server_keys.server_name, &[]);
ruma::signatures::verify_json(&pubkey_map, &canonical).map_err(Into::into)
}
/// Verifies that the notary has at least one valid signature present with
/// the provided signing keys.
fn verify_notary_signature(
server_keys: &ServerSigningKeys,
notary_name: &ServerName,
notary_keys: &[Base64],
) -> Result<()> {
// This strictly verifies that the notary signed this response.
// This is used to mitigate exploits where a notary's TLS
// certificate has been compromised, which would otherwise allow
// them to produce attacker-controlled signing keys with valid
// self-signatures.
let Some(notary_signatures) = server_keys.signatures.get(notary_name) else {
return Err!(Request(Forbidden(
"Missing notary server signature from {notary_name}"
)));
};
let mut canonical_object = to_canonical_object(notary_signatures)?;
Self::strip_extraneous_signatures(&mut canonical_object, &server_keys.server_name, &[
notary_name,
]);
let for_verify = ruma::signatures::to_canonical_json_string_for_signing(
&to_canonical_object(server_keys).expect("server keys object must be canonical json"),
)
.expect("canonical JSON must be stringable");
let raw_notary_keys = notary_keys.iter().map(Base64::as_bytes).collect::<Vec<_>>();
let raw_notary_signatures = notary_signatures
.values()
.flat_map(Base64::<Standard>::parse);
for signature in raw_notary_signatures {
let signature_bytes = signature.as_bytes();
for notary_key in raw_notary_keys.iter().copied() {
let verified = ruma::signatures::verify_canonical_json_bytes(
&SigningKeyAlgorithm::Ed25519,
notary_key,
signature_bytes,
for_verify.as_bytes(),
);
if verified.is_ok() {
trace!(
"Valid signature from {notary_name} present on signing keys response \
for {}",
server_keys.server_name
);
return Ok(());
}
}
}
Err!(Request(Forbidden(
"No valid signature from {notary_name} present on signing keys response"
)))
}
fn strip_extraneous_signatures(
canonical: &mut CanonicalJsonObject,
origin: &ServerName,
notaries: &[&ServerName],
) {
canonical.entry("signatures".to_owned()).and_modify(|sigs| {
sigs.as_object_mut().map(|s| {
s.retain(|server_name, _| {
let Ok(server_name) = ServerName::parse(server_name) else { return false };
origin == server_name || notaries.iter().any(|ns| *ns == server_name)
});
Some(s)
});
});
}
}
#[cfg(test)]
mod tests {
use ruma::{
ServerSigningKeyId,
api::federation::discovery::get_server_keys::v2::Response as DirectKeysResponse,
owned_server_name, serde::Raw, server_name,
};
use super::*;
use crate::server_keys::Service;
#[test]
fn verify_direct_server_keys_response() {
let response_data = serde_json::from_value(serde_json::json!(
{
"old_verify_keys": {},
"server_name": "continuwuity.org",
"signatures": {
"continuwuity.org": {
"ed25519:PwHlNsFu": "YUA+lUu5tQcRhd44bxFcY+WRu8X9nLXT8FUpY44O48/oKF/9rWUhqJGw5uZeOuo2oBMAxtUmO6HT2kBI9WftBA"
}
},
"valid_until_ts": 1787690539756_i64,
"verify_keys": {
"ed25519:PwHlNsFu": {
"key": "8eNx2s0zWW+heKAmOH5zKv/nCPkEpraDJfGHxDu6hFI"
}
}
}
)).map(Raw::from_json).unwrap();
Service::verify_server_keys_response(&response_data.deserialize().unwrap(), None)
.expect("Self-signature of real key response is valid");
}
#[test]
fn verify_direct_server_keys_response_ignores_extraneous_signatures() {
let response_data = serde_json::from_value(serde_json::json!(
{
"old_verify_keys": {},
"server_name": "continuwuity.org",
"signatures": {
"continuwuity.org": {
"ed25519:PwHlNsFu": "YUA+lUu5tQcRhd44bxFcY+WRu8X9nLXT8FUpY44O48/oKF/9rWUhqJGw5uZeOuo2oBMAxtUmO6HT2kBI9WftBA"
},
"example.org": {
"ed25519:1": "yadda yadda"
}
},
"valid_until_ts": 1787690539756_i64,
"verify_keys": {
"ed25519:PwHlNsFu": {
"key": "8eNx2s0zWW+heKAmOH5zKv/nCPkEpraDJfGHxDu6hFI"
}
}
}
)).map(Raw::from_json).unwrap();
Service::verify_server_keys_response(&response_data.deserialize().unwrap(), None)
.expect("Self-signature of real key response is valid");
}
#[test]
fn verify_notary_response_checks_valid_notary_signature() {
let response_data = serde_json::from_value(serde_json::json!(
{
"old_verify_keys": {},
"server_name": "continuwuity.org",
"signatures": {
"continuwuity.org": {
"ed25519:PwHlNsFu": "4YjHGqJ0HN93tvev4kx0qx/3w/h1nlEkFZni15h+5MZ91xUhQNa0BzqdraPU4/KJoXVcOQpmR+hsCVYb2/WlAQ"
},
"starstruck.systems": {
"ed25519:a_Herg": "G4uXHfn9lizmixPFCBRXVnyf1HB1yyrxPVbtSOdViNHRFrFF66xmEpzDPs341L/Pjxtwfndi4Hw7zIsyTRqoCQ"
}
},
"valid_until_ts": 1787691347175_i64,
"verify_keys": {
"ed25519:PwHlNsFu": {
"key": "8eNx2s0zWW+heKAmOH5zKv/nCPkEpraDJfGHxDu6hFI"
}
}
}
)).map(Raw::from_json).unwrap();
let starstruck_pubkey =
Base64::<Standard>::parse("Z4oHuEjdUbtC5W/z/gzc5B8raxNk+8gGwbNwgxHqW8Y").unwrap();
Service::verify_server_keys_response(
&response_data.deserialize().unwrap(),
Some((server_name!("starstruck.systems"), &[starstruck_pubkey])),
)
.expect("Server keys from notary are valid");
}
#[test]
fn ensure_verify_server_keys_response_rejects_invalid_self_signature() {
// even something as small as a mismatch in the timestamp should be
// enough to trigger this.
let response_data = serde_json::from_value(serde_json::json!(
{
"old_verify_keys": {},
"server_name": "continuwuity.org",
"signatures": {
"continuwuity.org": {
"ed25519:PwHlNsFu": "YUA+lUu5tQcRhd44bxFcY+WRu8X9nLXT8FUpY44O48/oKF/9rWUhqJGw5uZeOuo2oBMAxtUmO6HT2kBI9WftBA"
}
},
"valid_until_ts": 1787690539757_i64,
"verify_keys": {
"ed25519:PwHlNsFu": {
"key": "8eNx2s0zWW+heKAmOH5zKv/nCPkEpraDJfGHxDu6hFI"
}
}
}
)).map(Raw::from_json).unwrap();
Service::verify_server_keys_response(&response_data.deserialize().unwrap(), None)
.expect_err("Tampered response should fail self-signature check");
}
#[test]
fn ensure_mismatched_notary_signature_fails() {
// The signature on a notary response is typically useless. However,
// when additional verification is configured, it can be used to prevent
// an attack where the notary's TLS is intercepted, and the attacker
// returns a valid signing key response that is signed by the attacker
// with the attacker controlled key. In this case, the self-signature
// would pass, and the attacker would be allowed to inject an
// adversarial signing key into our database, allowing them to
// completely forge any event they ever want to as the target.
// However, by basically doing key pinning, this attack can be mitigated
// by raising the attack difficulty to requiring both TLS interception,
// and the target notary's private key.
// In this case, we'll use a valid self-signature, BUT incorrect
// notary signature.
// Without the additional notary validation, this would permit as TOFU.
let response_data = serde_json::from_value(serde_json::json!(
{
"old_verify_keys": {},
"server_name": "continuwuity.org",
"signatures": {
"continuwuity.org": {
"ed25519:PwHlNsFu": "Ov3RCRYRUEgaMPfHP4YPUvjoBKE0yVTERNlgeHpxvh/+fknGeRnIZ4e6b0n1aI+gPU04Eb1DE4gaeTFigCudAg"
}
},
"valid_until_ts": 1787690539756_i64,
"verify_keys": {
"ed25519:PwHlNsFu": {
"key": "8eNx2s0zWW+heKAmOH5zKv/nCPkEpraDJfGHxDu6hFI"
}
}
}
)).map(Raw::from_json).unwrap();
Service::verify_server_keys_response(&response_data.deserialize().unwrap(), None)
.expect_err("Intercepted notary response should fail strict validation");
}
}
+2 -1
View File
@@ -198,7 +198,8 @@ impl Service {
&& let active_count = state
.values()
.filter(|v| matches!(v, TxnState::Active(_)))
.count() && active_count >= max_active_txns
.count()
&& active_count >= max_active_txns
{
warn!(
active = active_count,
+2 -1
View File
@@ -373,7 +373,8 @@ impl super::Service {
.services
.appservice
.is_exclusive_user_id(&user_id)
.await && !emergency_mode_enabled
.await
&& !emergency_mode_enabled
{
return Err!(Request(Exclusive("Username is reserved by an appservice.")));
}
+2 -1
View File
@@ -81,7 +81,8 @@ impl super::Service {
if let Ok(canonical_profile) = to_canonical_object(full_profile) {
if serde_json::to_string(&canonical_profile)
.expect("should be able to serialize to string")
.len() > MAX_PROFILE_LENGTH_BYTES
.len()
> MAX_PROFILE_LENGTH_BYTES
{
return Err!(
"Profile data must not exceed {MAX_PROFILE_LENGTH_BYTES} bytes in \
+2
View File
@@ -314,6 +314,7 @@ async fn get_register_email_validate(
Some(email),
&client
)
.boxed()
.await?
)
}
@@ -508,6 +509,7 @@ async fn begin_registration(
// If email isn't required we can immediately complete registration
Ok(response!(
complete_registration(services, session_store, completed_registration, None, client)
.boxed()
.await?
))
}
+76
View File
@@ -10,7 +10,11 @@
--background-lightness: 0.9;
--c1: oklch(0.44 0.177 353.06);
--c1-light: oklch(from var(--c1) 0.5 c h);
--c1-bg: oklch(from var(--c1) var(--background-lightness) c h);
--c2: oklch(0.59 0.158 150.88);
--c2-light: oklch(from var(--c2) var(--name-lightness) c h);
--c2-bg: oklch(from var(--c2) var(--background-lightness) c h);
.rp-home-hero__title-brand {
--rp-home-hero-title-bg: linear-gradient(
130deg,
@@ -19,6 +23,10 @@
);
}
/* Use panel colors for rp buttons */
--panel-bg: oklch(0.91 0.042 317.27);
--panel-bg-dark: oklch(0.24 0.03 317.27);
--card-bg: oklch(from var(--panel-bg) var(--background-lightness) c h);
--rp-home-background-bg: radial-gradient(42.12% 56.13% at 100% 0%, oklch(from var(--c2) var(--background-lightness) c h) 0%, #fff0 100%), radial-gradient(42.01% 79.63% at 52.86% 0%, #d9ff5333 0%, #fff0 100%), radial-gradient(79.67% 58.09% at 0% 0%, oklch(from var(--c1) var(--background-lightness) c h) 0%, #fff0 100%), var(--rp-c-bg);
@@ -62,6 +70,19 @@
--rp-c-text-code-bg: oklch(0.59 0.158 150.88 / 0.15);
--rp-c-link: oklch(0.69 0.158 150.88);
--card-bg: oklch(from var(--panel-bg-dark) calc(l - 0.05) c h);
/* Make background gradient undisrupted in dark mode */
.custom-section, .rp-home-feature {
background:none
}
/* Use lighter color for nav bars */
.rp-nav-menu__item--active .rp-nav-menu__item__container,
.rp-nav-menu__item:hover .rp-nav-menu__item__container {
color: oklch(from var(--rp-c-brand) 0.8 c h)
}
}
img {
@@ -75,8 +96,63 @@ img {
color: var(--rp-c-link);
}
/* Smaller subtitle font so it fits in one line */
.rp-home-hero__subtitle {
font-size: 3rem;
}
/* CSS for main buttons */
/* Do not mute colors when hovered */
/* Hovering will invoke color rather than opacity changes */
.rp-button:hover {
opacity:1
}
/* No transitions, make button hovers snappier */
.rp-button:hover {
transition: all 0s
}
/* Square buttons just like c10y's account mgmt page */
.rp-button--big {
padding-inline: 1.25em;
border-radius: 5px
}
/* Colors from c10y's mgmt page */
.rp-button--brand {
background: var(--c1);
}
.rp-button--brand:hover {
background: var(--c1-light);
}
.rp-button--alt, .dark .rp-button--alt {
background: var(--card-bg);
}
.rp-button--alt {
border: 1px solid var(--rp-c-gray-light-1);
}
.rp-button--alt:hover {
border-color: var(--c1-light)
}
/* Stylize "Donate to us" button */
/* Light mode */
a.rp-button:nth-child(4) {
border: 1.5px solid var(--c2);
}
a.rp-button:nth-child(4):hover {
color: var(--c2);
}
/* Dark */
.dark a.rp-button:nth-child(4) {
color: var(--rp-c-text-0);
background: var(--card-bg)
}
.dark a.rp-button:nth-child(4):hover {
background: var(--c2-bg)
}
/* Vertical layout */