Compare commits

...
Author SHA1 Message Date
timedout 7e81918639 wip 2026-11-11 14:36:24 +00:00
timedout 302f535e66 refactor(wip): Replace local state resolution with upstream Ruma's 2026-11-11 14:17:05 +00:00
Renovate Bot 93173313ea chore(deps): update pre-commit hook crate-ci/typos to v1.52.2 2026-11-11 05:08:21 +00:00
Jade Ellis 38fb7897bb chore: Release 2026-11-10 23:23:41 +00:00
stratself ba0ea752a9 docs(generic): Recommend the MTRNord tester 2026-11-10 23:12:53 +00:00
stratself c7c54f880a docs(arch): Link to AUR/archlinuxcn instructions 2026-11-10 23:12:53 +00:00
stratself c65eb6de91 docs(push): Update some things (see commit comments)
* Make "gateway discovery" test from both client and server
* Add docker caveat to locally-hosted push components
* Edit a name to match author
2026-11-10 23:12:53 +00:00
stratself 596f269a90 docs(performance): Update trusted servers tuning info 2026-11-10 23:12:53 +00:00
stratself 49ebbcc63e fix(docs): Don't say "check Maintenance for Maintenance" 2026-11-10 23:12:53 +00:00
stratself 7241150204 docs(arch-linux): Add some wordings
Ideally want the installation commands here, but will leave for a future
commit
2026-11-10 23:12:53 +00:00
stratself c66a863bb2 docs(kubernetes): Small wording improvements and postinstall section 2026-11-10 23:12:53 +00:00
stratself 6dbe40238d docs(nixos): Beautify nix a bit
* Add well-known config example to nixfile
* Add postinstall redirect section
2026-11-10 23:12:53 +00:00
stratself 77bc7476f3 docs(auth): Reword registration defaults 2026-11-10 23:12:53 +00:00
stratself 94270e732c docs(livekit): Address review comments, add note for EC's dev mode 2026-11-10 23:00:08 +00:00
stratself 0bc55cb590 docs(livekit): Use "MatrixRTC" phrasing in lieu of "Element Call"
The calling protocol matters not the client product
2026-11-10 23:00:08 +00:00
stratself 0375643ade docs(livekit): Add stun_servers to TURN instructions too 2026-11-10 23:00:08 +00:00
stratself c67f08a208 docs(livekit): Add appendix sectionon creating livekit tokens via livekit-cli 2026-11-10 23:00:08 +00:00
stratself fb3113ea69 docs(livekit): Add appendix section on calling modes 2026-11-10 23:00:08 +00:00
stratself 6f6fba4310 docs(livekit): Pin token exchange flow link to jwt-svc 0.6.0 2026-11-10 23:00:08 +00:00
stratself 9ace03c107 docs(livekit): Add more relevant MSCs 2026-11-10 23:00:08 +00:00
stratself 304bf69c32 docs(livekit): Add LiveKit advertised IP addresses restrictions 2026-11-10 23:00:08 +00:00
stratself 2173577bac docs(livekit): Update Element Call UI errors 2026-11-10 23:00:08 +00:00
stratself bbf3bd1294 docs(calls): Small fixup of wordings
* Use indirect links and fix them
* Make Livekit's TURN configuration optional
2026-11-10 23:00:08 +00:00
stratself dd11fc5a25 docs(livekit): Remove workaround for non-federating servers
Fixed by #2326
2026-11-10 23:00:08 +00:00
stratself f1798c004a docs(calls): Redirect to support room 2026-11-10 23:00:08 +00:00
Erwan Leboucher b570420b9f fix: preserve predecessors sharing a state-before ID
Signed-off-by: Erwan Leboucher <erwanleboucher@gmail.com>
2026-11-10 23:56:33 +01:00
47 changed files with 839 additions and 5946 deletions

No files matched your search

+1 -1
View File
@@ -24,7 +24,7 @@ repos:
- id: check-added-large-files
- repo: https://github.com/crate-ci/typos
rev: v1.51.1
rev: v1.52.2
hooks:
- id: typos
- id: typos
+70
View File
@@ -1,3 +1,73 @@
# Continuwuity v26.11.0-alpha.1 (2026-11-10)
## Features
- Added opt-in OpenTelemetry metrics, exported over OTLP or scraped by Prometheus from `/_continuwuity/metrics`, and a Grafana dashboard. Contributed by @eleboucher. (prometheus-metrics)
- Add support for using custom DNS servers instead of using /etc/resolv.conf. Contributed by @s1lv3r. ([#2056](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2056))
- Continuwuity now supports configuring additional trusted audience values for deployments using OIDC with an IDP that sends them. Contributed by @ginger ([#2226](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2226))
- "!admin users list" now hides deactivated users by default, but can display them with the optional "--show-deactivated" flag. "--hide-suspended" and "--hide-locked" have been added as well, which stop suspended and locked users from being displayed respectively. Contributed by @Clubs. ([#2292](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2292))
- Added support for [MSC4494: Membership-based invite blocking](https://github.com/matrix-org/matrix-spec-proposals/pull/4494) ([#2297](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2297))
- Appservices now receive typing indicators and read receipts. Contributed by @Jade ([#2327](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2327))
- Added support for [MSC4363](https://github.com/matrix-org/matrix-spec-proposals/pull/4363) (OAuth step-up authentication). Contributed by @ginger.
- Added support for [MSC4484](https://github.com/matrix-org/matrix-spec-proposals/pull/4484). Clients which use the built-in Matrix server administration APIs will need to ask for the appropriate scope when signing in with OAuth. Contributed by @ginger.
- Implemented unstable support for [MSC4560] (Stripped state requests over federation). Contributed by @ginger.
[MSC4560]: https://github.com/matrix-org/matrix-spec-proposals/pull/4560
- Profile banners are now treated as images when imported from OIDC claims, like avatars already were.
- The first-run version of the web UI now includes a link to the registration page. Contributed by @ginger
## Bugfixes
- Invitees are now added as room creators when creating a v12+ trusted private chat. Contributed by @ginger. ([#2162](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2162))
- Fixed push gateway URLs containing the configured notification path outside of their suffix. Contributed by @eleboucher. ([#2261](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2261))
- "!admin users change-email" now verifies a user exists before attempting to change their email. Contributed by @Clubs. ([#2298](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2298))
- Fixed URL preview image MIME types and rejected failed downloads. Contributed by @eleboucher. ([#2308](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2308))
- Correctly set RocksDB column options, including enabling block caches. Contributed by @Jade ([#2345](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2345))
- Fix some possible deadlocks/freezes and reduce lock contention (improving performance). Contributed by @Jade ([#2351](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2351))
- Once again fixed how room extremities are handled, fixing a problem that made continuwuity gradually make rooms needlessly complex, which also caused more frequent state resets. ([#2356](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2356))
- The OAuth identity reset page is no longer linked from the account management UI to reduce confusion. Contributed by @ginger. ([#2383](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2383))
- Fixed a worker panic when knocking on a room over federation. Contributed by @eleboucher. ([#2387](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2387))
- Perform more validation on events received during atomic gapfill from remote servers, fixing a panic if bad data was persisted by another server and later lazily fetched by us. ([#2400](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2400))
- Redirect URIs in OAuth clients' metadata are now properly validated to be on a subdomain of the OAuth issuer URL domain. Contributed by @ginger. ([#2421](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2421))
- Fix v12 rooms created by the same user in the same millisecond ending up as the same room. Contributed by @eleboucher. ([#2424](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2424))
- Client-server API endpoints are now properly ratelimited for unauthenticated requests. Contributed by @ginger. ([#2444](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2444))
- Setting a profile key longer than 255 bytes now returns 400 `M_KEY_TOO_LARGE` as required by the spec, instead of 413 `M_TOO_LARGE`. ([#2451](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2451))
- Setting `m.fully_read` via the account data endpoints now returns 405 `M_BAD_JSON` as required by the spec, instead of 400. ([#2451](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2451))
- Appservices masquerading as a device that does not exist now receive 400 `M_UNKNOWN_DEVICE` as required by the spec, instead of 403 `M_FORBIDDEN`. ([#2451](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2451))
- Setting profile data that would make the profile exceed 64 KiB now returns 400 `M_PROFILE_TOO_LARGE` as required by the spec, instead of a 500 error. ([#2451](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2451))
- The `m.forget_forced_upon_leave` capability now reflects the `forget_forced_upon_leave` config option instead of always being advertised as enabled. ([#2452](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2452))
- Fixed push notifications for membership events setting `user_is_target` based on the sender rather than the user being notified. ([#2454](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2454))
- Ensure HTTP clients actually follow the `dns_servers` config option. Contributed by @silversys ([#2460](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2460))
- Reworked event relations to follow the spec more closely. Pagination of `/relations` now returns the correct pages and `next_batch` tokens, recursive relations are returned in the right order without duplicates, and bundled `m.reference` aggregations now include every reference. Contributed by @Henry-Hiles (QuadRadical). ([#2465](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2465))
- Read receipts are now more spec compliant, will work better in threads and are less likely to end up causing loops with other software. Contributed by @Jade ([#2481](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2481))
- Fix gap filling skipping outlier events that are missing their previous events. Contributed by @eleboucher.
- Fix incoming federation events losing state changes when predecessors share the same state-before snapshot. Contributed by @eleboucher.
- Fix new devices sometimes never reaching other users' device lists. Contributed by @eleboucher.
- Fix state resolution applying events out of order, which could undo room joins. Contributed by @eleboucher.
- Fix to-device messages, including room keys, sometimes being lost before delivery. Contributed by @eleboucher.
- For deployments using OIDC with `profile_key_import_mode` set to `on_login`, users' avatars will no longer be clobbered if they didn't change their avatar since their last login.
- Room summary visibility checks are no longer incorrectly applied for remote rooms. Contributed by @ginger.
- The userinfo endpoint is now accessible even if federation is disabled, allowing LiveKit to be used on servers with federation disabled without using the server name whitelist.
- `!admin token issue` can no longer issue registration tokens when `allow_registration` is false. Contributed by @Clubs.
## Improved Documentation
- Add the Authentication section to document available auth methods for Continuwuity. The "delegated authentication" guide has also been migrated to here. Contributed by @stratself. ([#2234](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2234))
- Add a guide for push notifications, detailing how it works, push gateway specifics, troubleshooting common issues, and pointing to external guides for UnifiedPush backends. Contributed by @stratself ([#2244](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2244))
- The deploying page now includes vital general information about installing Continuwuity: release tracks, update steps, and post-installation recommendations. Contributed by @stratself ([#2252](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2252))
- Add some missing instructions on configuring Continuwuity on the RPM page. Users are encouraged to use the [generic deploying docs](https://continuwuity.org/deploying/generic) for the bulk of guidance anyhow. Contributed by @stratself ([#2275](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2275))
- Better, more succinct [introduction page](https://continuwuity.org/introduction.html). Contributed by @stratself ([#2279](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2279))
- Add a section on [troubleshooting Docker issues](https://continuwuity.org/troubleshooting#docker-issues). Contributed by @stratself. ([#2324](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2324))
- Instructions for Authelia added to the [OIDC guide](https://continuwuity.org/authentication/oidc.html). Contributed by @fhoekstra ([#2341](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2341))
- Docker compose examples now use bind mounts by default, instead of Docker volumes, as they are easier to work with esp. for backup/restore. Contributed by @stratself. ([#2353](https://forgejo.ellis.link/continuwuation/continuwuity/pulls/2353))
- The documentation now refers to the Continuwuity Docker service as `continuwuity` instead of `homeserver`.
## Misc
- Updated internal Sentry integration setup to the new pattern required by the SDK. #2423
- Upgraded rocksdb from 11.1.1 to 11.12.0 and jemalloc to 5.4.0.
# Continuwuity v26.9.1 (2026-09-28)
## Features
Generated
+14 -12
View File
@@ -865,7 +865,7 @@ dependencies = [
[[package]]
name = "conduwuit"
version = "26.9.1"
version = "26.11.0-alpha.1"
dependencies = [
"clap",
"conduwuit_build_metadata",
@@ -897,7 +897,7 @@ dependencies = [
[[package]]
name = "conduwuit_admin"
version = "26.9.1"
version = "26.11.0-alpha.1"
dependencies = [
"clap",
"conduwuit_api",
@@ -921,7 +921,7 @@ dependencies = [
[[package]]
name = "conduwuit_api"
version = "26.9.1"
version = "26.11.0-alpha.1"
dependencies = [
"axum",
"base64 0.23.1",
@@ -953,7 +953,7 @@ dependencies = [
[[package]]
name = "conduwuit_build_metadata"
version = "26.9.1"
version = "26.11.0-alpha.1"
dependencies = [
"built",
"cargo_metadata",
@@ -961,7 +961,7 @@ dependencies = [
[[package]]
name = "conduwuit_core"
version = "26.9.1"
version = "26.11.0-alpha.1"
dependencies = [
"argon2",
"arrayvec",
@@ -1031,7 +1031,7 @@ dependencies = [
[[package]]
name = "conduwuit_database"
version = "26.9.1"
version = "26.11.0-alpha.1"
dependencies = [
"async-channel",
"conduwuit_core",
@@ -1051,7 +1051,7 @@ dependencies = [
[[package]]
name = "conduwuit_macros"
version = "26.9.1"
version = "26.11.0-alpha.1"
dependencies = [
"itertools 0.15.0",
"proc-macro2",
@@ -1061,7 +1061,7 @@ dependencies = [
[[package]]
name = "conduwuit_router"
version = "26.9.1"
version = "26.11.0-alpha.1"
dependencies = [
"axum",
"axum-server",
@@ -1094,7 +1094,7 @@ dependencies = [
[[package]]
name = "conduwuit_service"
version = "26.9.1"
version = "26.11.0-alpha.1"
dependencies = [
"askama",
"assign",
@@ -1122,6 +1122,7 @@ dependencies = [
"lru-cache",
"nonzero_ext",
"openidconnect",
"opentelemetry",
"p256 0.14.0",
"rand 0.10.3",
"regex",
@@ -1130,6 +1131,7 @@ dependencies = [
"resolvematrix",
"ruma",
"ruminuwuity",
"rustls",
"rustyline-async",
"sd-notify",
"serde",
@@ -1148,7 +1150,7 @@ dependencies = [
[[package]]
name = "conduwuit_web"
version = "26.9.1"
version = "26.11.0-alpha.1"
dependencies = [
"askama",
"async-trait",
@@ -5165,7 +5167,7 @@ dependencies = [
[[package]]
name = "ruminuwuity"
version = "26.9.1"
version = "26.11.0-alpha.1"
dependencies = [
"assign",
"http",
@@ -7349,7 +7351,7 @@ dependencies = [
[[package]]
name = "xtask"
version = "26.9.1"
version = "26.11.0-alpha.1"
dependencies = [
"askama",
"cargo_metadata",
+1 -1
View File
@@ -12,7 +12,7 @@ license = "Apache-2.0"
# See also `rust-toolchain.toml`
readme = "README.md"
repository = "https://forgejo.ellis.link/continuwuation/continuwuity"
version = "26.9.1"
version = "26.11.0-alpha.1"
[workspace.metadata.crane]
name = "conduwuit"
@@ -0,0 +1 @@
Fix incoming federation events losing state changes when predecessors share the same state-before snapshot. Contributed by @eleboucher.
+3 -1
View File
@@ -4,7 +4,7 @@ Continuwuity comes with an internal database for user authentication, which it u
## Registration
Registration is disabled by default. To enable it, set `allow_registration = true` in your config file and use at least one of the registration methods below.
The following registration methods are available for Continuwuity:
| Method | Description | Additional configuration needed |
| ------ | --------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
@@ -19,6 +19,8 @@ Admin-issued tokens are the recommended registration method. These tokens are we
Other registration methods are meant for **untrusted, public registration**. Care must be taken when allowing these modes of operation.
If you wish to disable registration entirely, set `allow_registration = false` in your config file.
## Email configuration
Continuwuity can be configured to send emails via an SMTP relay of your choice. These are used for verifying user email addresses and sending password reset links to them.
+9 -5
View File
@@ -2,12 +2,16 @@
Matrix supports two types of calls:
- Element Call powered by [MatrixRTC](https://half-shot.github.io/msc-crafter/#msc/4143) and [LiveKit](https://github.com/livekit/livekit)
- Legacy calls, supported by a STUN/TURN server.
- [MatrixRTC][MSC4143] calls, powered by [LiveKit][livekit-server-github] and frontends such as [Element Call][element-call].
- Legacy 1-to-1 calls, supported by a STUN/TURN server.
Both types of calls are supported by different sets of clients, but most clients are moving towards MatrixRTC / Element Call.
[MSC4143]: https://github.com/matrix-org/matrix-spec-proposals/pull/4143
[element-call-github]: https://github.com/element-hq/element-call/
[livekit-server-github]: https://github.com/livekit/livekit
Both types of calls are supported by different sets of clients, but most clients are moving towards MatrixRTC.
For either one to work correctly, you have to do some additional setup:
- For legacy calls to work, you need to set up a TURN/STUN server. [Read the TURN guide for tips on how to set up coturn](./calls/turn.mdx)
- For MatrixRTC / Element Call to work, you have to set up the LiveKit backend. LiveKit also uses TURN/STUN to increase reliability - you can set up its built-in TURN server, or integrate with an existing one. [Read the LiveKit guide](./calls/livekit.mdx)
- For legacy calls to work, you need to set up a TURN/STUN server. [Read the TURN guide for instructions on how to set up coturn](./calls/turn.mdx)
- For MatrixRTC calls to work, you have to set up the LiveKit backend, and optionally configure a TURN server for increased reliability. [Read the LiveKit guide](./calls/livekit.mdx)
+86 -23
View File
@@ -1,13 +1,15 @@
import { Tab, Tabs} from '@rspress/core/theme';
import { Tab, Tabs } from '@rspress/core/theme';
# MatrixRTC/Element Call Setup
# MatrixRTC/LiveKit Setup
This guide assumes that you are using docker compose for deployment.
This guide assumes that you are using `docker compose` for deployment.
:::tip
You can find help setting up MatrixRTC in our dedicated room - [#matrixrtc:continuwuity.org](https://matrix.to/#/%23matrixrtc%3Acontinuwuity.org)
You can find help setting up MatrixRTC in our support room - [`#continuwuity:continuwuity.org`][support-room]
:::
[support-room]: https://matrix.to/#/%23continuwuity:continuwuity.org
## Instructions
### 1. Set up your domain
@@ -58,7 +60,9 @@ services:
- ./livekit.yaml:/etc/livekit.yaml:ro
network_mode: "host" # /!\ LiveKit binds to all addresses by default.
# Make sure port 7880 is blocked by your firewall to prevent access bypassing your reverse proxy
# Alternatively, uncomment the lines below and comment `network_mode: "host"` above to specify port mappings.
# Alternatively, uncomment the lines below and comment out `network_mode: "host"`
# above to use bridge networking and manually specify port mappings.
# ports:
# - "127.0.0.1:7880:7880/tcp"
# - "7881:7881/tcp"
@@ -218,7 +222,9 @@ Start up the services using your usual method - for example `docker compose up -
## Testing
To test that LiveKit is successfully integrated with Continuwuity, you will need to replicate its [Token Exchange Flow](https://github.com/element-hq/lk-jwt-service#%EF%B8%8F-how-it-works--token-exchange-flow). Follow the steps below while checking Docker logs (`docker compose logs --follow`), in order to help [troubleshooting](#troubleshooting) any issues.
To test that LiveKit is successfully integrated with Continuwuity, you will need to replicate its [Token Exchange Flow][token-exchange-flow]. Follow the steps below while checking Docker logs (`docker compose logs --follow`), in order to help [troubleshooting](#troubleshooting) any issues.
[token-exchange-flow]: https://github.com/element-hq/lk-jwt-service/tree/v0.6.0#%EF%B8%8F-how-it-works--token-exchange-flow
First, you will need an access token for your current login session. These can be found in your client's settings or obtained via [this website][starstruck-mxtoken2].
@@ -279,7 +285,7 @@ You can then send this payload to the lk-jwt-service:
curl -X POST -d @payload.json https://livekit.example.com/get_token
```
The lk-jwt-service will, after checking against Continuwuity, answer with a `jwt` token to create a LiveKit media room:
The lk-jwt-service will, after checking against Continuwuity, answer with a `jwt` token to [create a LiveKit media room](#livekit-cli):
```json
{"url":"wss://livekit.example.com","jwt":"a_really_really_long_string"}
@@ -293,13 +299,20 @@ Use this token to test at the [LiveKit Connection Tester][livekit-connection-tes
To debug any issues, you can place a call or redo the Testing instructions, and check the container logs for any specific errors. Use `docker compose logs --follow` to follow these logs in real-time.
Before debugging, ensure the following prerequisites:
- The user has permissions to send call events into the room. See the Appendix section on [call modes](#call-modes) for more details.
- WebRTC is enabled, if using a browser client.
### Common errors in Element Call UI
- `MISSING_MATRIX_RTC_FOCUS`/`MISSING_MATRIX_RTC_TRANSPORT`: LiveKit is missing from Continuwuity's config file
- "Waiting for media" popup always showing for yourself: a LiveKit URL has been configured in Continuwuity, but your client cannot connect to it for some reason
- `OPEN_ID_ERROR`: Your client can reach out to `lk-jwt-service`, but has problems authenticating with it. In this case, check `lk-jwt-service` logs for more details
- `MISSING_MATRIX_RTC_FOCUS`/`MISSING_MATRIX_RTC_TRANSPORT`: the client cannot find a suitable SFU. This is likely due to LiveKit being missing from Continuwuity's config file, and therefore not being exposed properly.
For browser-based clients, you can also inspect connections using DevTools' Networking tab, to see which requests are erroring out.
- `OPEN_ID_ERROR`: Your client can reach out to `lk-jwt-service`, but `lk-jwt-service` has problems authenticating you against Continuwuity. In this case, check `lk-jwt-service` logs for more details. The section below on [Docker loopback networking issues](#docker-loopback-networking-issues) may be of help.
- Calls fail to start with `ServerUnreachable` error: a LiveKit URL has been properly configured and exposed in Continuwuity, but the client cannot connect to it for some reason.
For browser-based clients, you can inspect connections using DevTools' Networking tab, to see which requests are erroring out. You may also want to toggle on Element Call's Developer tab via _Settings > Preferences > Developer mode_ \- various call details and advanced settings can be further tweaked there.
### Docker loopback networking issues
@@ -312,9 +325,9 @@ docker run --rm --net container:lk-jwt-service docker.io/curlimages/curl https:/
To resolve this, follow the steps in [Docker loopback networking issues](../troubleshooting.mdx#docker-loopback-networking-issues) of the Troubleshooting page.
### Incorrect IP address for LiveKit
### Incorrect IP addresses for LiveKit
By default, LiveKit automatically finds its public IP address(es), which can be seen in the "Establishing WebRTC connection" section of the connection test page. If these IPs are incorrect, you may want to hardcode your own IP by doing the following:
By default, LiveKit automatically finds its public IP address(es), which are reflected in the "Establishing WebRTC connection" section of the connection test page. If these IPs are incorrect, you may want to hardcode your own IP by doing the following:
```diff
### in your livekit.yaml ###
@@ -325,20 +338,21 @@ rtc:
+ node_ip: "1.2.3.4"
```
### Workaround for non-federating servers
Run the LiveKit tests again, and the new updated `node_ip` should be included as one of the candidates.
When deploying on servers with federation disabled (`allow_federation = false`), LiveKit will fail as it can't fetch the required [OpenID endpoint](https://spec.matrix.org/v1.17/server-server-api/#get_matrixfederationv1openiduserinfo) via federation paths.
You may further restrict the advertised address to only the `node_ip` with the following configurations:
As a workaround, you can enable federation, but forbid all remote servers via the following config parameters:
```toml
### in your continuwuity.toml file ###
allow_federation = true
forbidden_remote_server_names = [".*"]
```diff
### in your livekit.yaml ###
rtc:
# ... other configs here ...
+ enable_loopback_candidate: false
+ interfaces:
+ includes: []
+ ips:
+ includes: [ "1.2.3.4/32" ]
```
Subscribe to issue [!1440](https://forgejo.ellis.link/continuwuation/continuwuity/issues/1440) for future updates on this matter.
## Related Documentation
Guides:
@@ -359,9 +373,15 @@ Specifications:
- [MSC4143: MatrixRTC \- Real-time communication over Matrix][MSC4143]
- [MSC4195: MatrixRTC Transport using LiveKit Backend][MSC4195]
- [MSC3401: Native Group VoIP Signalling][MSC3401] - state events for legacy calls
- [MSC4075: MatrixRTC invites and notifications][MSC4075] - for ringing users
- [MSC4354: Sticky Events][MSC4354] - for MatrixRTC 2.0 calls
[MSC4143]: https://github.com/matrix-org/matrix-spec-proposals/pull/4143
[MSC4195]: https://github.com/matrix-org/matrix-spec-proposals/pull/4195
[MSC3401]: https://github.com/matrix-org/matrix-spec-proposals/pull/3401
[MSC4075]: https://github.com/matrix-org/matrix-spec-proposals/pull/4075
[MSC4354]: https://github.com/matrix-org/matrix-spec-proposals/pull/4354
Source code:
@@ -381,6 +401,47 @@ Other:
## Appendix
### MatrixRTC calling modes and enabling sticky events {#call-modes}
MatrixRTC currently supports 3 calling modes:
- **Legacy mode** (deprecated): calls are established by sending state events to a room. The call initiator's SFU hosts the entire call for all remote and local participants.
- **Compatibility mode*: same as Legacy mode, but utilizes multiple SFUs between federating servers. Clients publish media streams to their own SFUs, and fetch remote streams from remote SFUs. This is the most commonly used mode nowadays.
- **MatrixRTC 2.0 mode**: also utilizes multiple SFUs, but establishes calls with [sticky events] instead of state events.
To support Legacy and Compatibility mode, ensure that the user is allowed to send the relevant state events. Usually, this means promoting them to a power level of 50 or above. Alternatively, set the power level required to send `org.matrix.msc3401.call.member` events to 0.
To support MatrixRTC 2.0 mode, sticky events has to be enabled on the Matrix server. In Continuwuity, this is as easy as setting `enable_sticky_events = true` in the config file.
### Creating LiveKit tokens via `livekit-cli` {#livekit-cli}
To create room tokens directly without going through `lk-jwt-service`, use the [Livekit CLI tool][livekit-cli].
<Tabs groupId="livekit">
<Tab label="Bare Binary">
First, download the [release tarball][livekit-cli-releases] and extract it. Then:
```sh
./lk --url https://livekit.example.com \
--api-key APIUxUnMnSkuFWV \
--api-secret t93ZVjPeoEdyx7Wbet3kG4L3NGZIZVEFvqe0UuiVc22A \
token create --create --join
```
</Tab>
<Tab label="Docker">
```sh
docker run --rm docker.io/livekit/livekit-cli --url https://livekit.example.com \
--api-key APIUxUnMnSkuFWV \
--api-secret t93ZVjPeoEdyx7Wbet3kG4L3NGZIZVEFvqe0UuiVc22A \
token create --create --join
```
</Tab>
</Tabs>
[livekit-cli]: https://github.com/livekit/livekit-cli
[livekit-cli-releases]: https://github.com/livekit/livekit-cli
### Additional TURN-over-TLS on 443 configuration
Most of the time, LiveKit [**does not need TURN**][sspaeth-matrix-voip-turn] to function. However, there are situations where clients may be unable to use nonstandard ports or UDP connections, such as in highly restrictive networks. In such cases, a TURN-over-TLS server on port 443 could relay traffic for the clients.
@@ -487,6 +548,8 @@ Then configure LiveKit, making sure to replace `COTURN_SECRET` with the one you
### in your livekit.yaml ###
rtc:
# ... other configs here ...
stun_servers:
- coturn.example.com
turn_servers:
- host: coturn.example.com
port: 443
+4 -3
View File
@@ -1,16 +1,17 @@
# Setting up TURN/STUN
[TURN][turn] and [STUN][stun] are used as components in many calling systems. Matrix uses them directly for legacy calls and indirectly for MatrixRTC via Livekit.
[TURN][turn] and [STUN][stun] are used as components in many calling systems. Matrix uses them directly for legacy 1-to-1 calls and indirectly to assist MatrixRTC calls via Livekit.
Continuwuity recommends using [coturn][coturn] as your TURN/STUN server, which is available as a Docker image or a distro package. This guide assumes that you are using docker compose for deployment.
Continuwuity recommends using [coturn][coturn] as your TURN/STUN server, which is available as a Docker image or a distro package. This guide assumes that you are using `docker compose` for deployment.
:::tip
You can find help setting up TURN/STUN in our MatrixRTC room - [#matrixrtc:continuwuity.org](https://matrix.to/#/%23matrixrtc%3Acontinuwuity.org)
You can find help setting up TURN/STUN in our support room - [`#continuwuity:continuwuity.org`][support-room]
:::
[turn]: https://en.wikipedia.org/wiki/Traversal_Using_Relays_around_NAT
[stun]: https://en.wikipedia.org/wiki/STUN
[coturn]: https://github.com/coturn/coturn
[support-room]: https://matrix.to/#/%23continuwuity:continuwuity.org
## Installing coturn
+1 -1
View File
@@ -101,7 +101,7 @@ After Continuwuity is up and running, consider the following:
- See the [**Authentication chapter**](./authentication.mdx) for various ways to access to your homeserver, such as logging in with an external OIDC provider.
- Consult the [**Maintenance**](./maintenance.mdx) page for guidance on maintaining your homeserver.
- Consult the [**Maintenance**](./maintenance.mdx) page for guidance on backups, moderation, and other topics for homeserver management.
- If you want to set up an appservice, take a look at the [**Appservice Guide**](./appservices.mdx).
+8 -2
View File
@@ -2,10 +2,16 @@
Continuwuity is available in the [`archlinuxcn`][archlinuxcn] repository and AUR with the same package name, [`continuwuity`][aur-continuwuity].
The development version is available on the AUR as [`continuwuity-git`][aur-continuwuity-git].
For instructions on adding the archlinuxcn repo to your machine, see the [archlinuxcn README][archlinuxcn-readme] page. For AUR instructions, consult its [entry][aur-archwiki] on the Arch Wiki.
The development version, which tracks latest commits to the main branch, is also available on the AUR as [`continuwuity-git`][aur-continuwuity-git].
[archlinuxcn]: https://www.archlinuxcn.org/
[archlinuxcn-readme]: https://github.com/archlinuxcn/repo#usage
[aur-archwiki]: https://wiki.archlinux.org/title/Arch_User_Repository
[aur-continuwuity]: https://aur.archlinux.org/packages/continuwuity
[aur-continuwuity-git]: https://aur.archlinux.org/packages/continuwuity-git
Simply install the `continuwuity` package. Configure the service in `/etc/conduwuit/conduwuit.toml`, then enable and start the `continuwuity.service`.
Once you're done configuring your repos, simply install the `continuwuity` package with `pacman -S continuwuity`. Consult the [generic deployment instructions](./generic.mdx) for detailed steps, which involves editing `/etc/conduwuit/conduwuit.toml` and setting up a reverse proxy. Finally, enable and start the `continuwuity.service`.
After successfully setting up Continuwuity, navigate to [Post-installation](../deploying.mdx#post-installation) for next steps.
+2 -2
View File
@@ -246,8 +246,8 @@ to the `#admin:example.com` room and be able to [issue admin commands](../refere
To check if your server can communicate with other homeservers,
use an external testing tool:
- [Matrix Connectivity Tester](https://federationtester.mtrnord.blog/)
- [Matrix Federation Tester](https://federationtester.matrix.org/)
- [Matrix Connectivity Tester](https://federationtester.mtrnord.blog/) by MTRNord (recommended, with more tests).
- [Matrix Federation Tester](https://federationtester.matrix.org/) by f0x.
If you can register your account but cannot join federated rooms, check your configuration
and verify that your federation endpoints are opened and forwarded correctly.
+17 -9
View File
@@ -1,11 +1,11 @@
# Continuwuity for Kubernetes
Continuwuity doesn't support horizontal scalability or distributed loading
natively. However, a deployment in Kubernetes is very similar to the docker
setup. This is because Continuwuity can be fully configured using environment
variables. A sample StatefulSet is shared below. The only thing missing is
natively. However, a deployment in Kubernetes is very similar to the [docker setup](./docker.mdx).
This is because Continuwuity can be fully configured using environment
variables. A sample `StatefulSet` is shared below. The only thing missing is
a PVC definition (named `continuwuity-data`) for the volume mounted to
the StatefulSet, an Ingress resources to point your webserver to the
the `StatefulSet`, an Ingress resources to point your webserver to the
Continuwuity Pods, and a Service resource (targeting `app.kubernetes.io/name: continuwuity`)
to glue the Ingress and Pod together.
@@ -103,10 +103,18 @@ spec:
---
Apart from manually configuring the containers,
[a community-maintained Helm Chart is available here to run
conduwuit on Kubernetes](https://gitlab.cronce.io/charts/conduwuit)
## Community maintained Helm Chart
This should be compatible with Continuwuity, but you will need to change the image reference.
Apart from manually configuring the containers, you can consult
a community-maintained Helm Chart for the former `conduwuit` project,
which is available [here][community-helm-chart].
If changes need to be made, please reach out to the maintainer, as this is not maintained or controlled by the Continuwuity maintainers.
[community-helm-chart]: https://gitlab.cronce.io/charts/conduwuit
This should be compatible with Continuwuity, but you will need to switch the image reference.
If changes are needed to be made, please reach out to the maintainer of that repository, as it is not maintained nor controlled by the Continuwuity maintainers.
## Post-installation
After successfully setting up your Continuwuity, navigate to [Post-installation](../deploying.mdx#post-installation) for next steps.
+14 -1
View File
@@ -31,7 +31,14 @@ services.matrix-continuwuity = {
# address and port are handled automatically
# You can add any further configuration here, e.g.
# trusted_servers = [ "matrix.org" ];
# trusted_servers = [ "matrix.org" ];
## (if using well-known delegation)
# well_known = {
# client = "https://matrix.example.com";
# server = "matrix.example.com:443";
# }
};
};
};
@@ -79,3 +86,9 @@ client = "https://matrix.example.com"
You'll need to set up a reverse proxy (like NGINX or Caddy) to expose Continuwuity to the internet. You can configure your reverse proxy using NixOS options (e.g. `services.caddy`).
See the [reverse proxy setup guide](./generic.mdx#setting-up-the-reverse-proxy) for information on correct reverse proxy configuration.
Then, test that your setup works by following these [instructions](./generic.mdx#how-do-i-know-it-works).
## What's next?
Navigate to [Post-installation](../deploying.mdx#post-installation) for next steps.
+15 -19
View File
@@ -82,6 +82,21 @@ Consult these documents for more information on compression tuning and levels:
## Other tweaks
### Tuning your trusted servers
:::important Important: Vet your trusted servers!
Trusted servers are your first point of contact when obtaining public keys from the wider federated network. They could theoretically impersonate other servers and cause significant harm to your deployment. Please thoroughly verify your trusted servers' credibility before adding them to your configuration.
If you need suggestions for trusted servers, ask in the Continuwuity support room.
:::
For performance and reliability, it is recommended to set up 3 to 5 trusted servers for your deployment. Currently, these must all be Synapse servers.
```toml
# Example config, using maintainers' recommended homeservers
trusted_servers = ["codestorm.net","starstruck.systems","unredacted.org","matrix.org"]
```
### Using UNIX sockets
If your homeserver and reverse proxy live on the same machine, you may wish to expose Continuwuity on a UNIX socket instead of a port. This removes TCP overhead between the two programs.
@@ -111,25 +126,6 @@ https://matrix.example.com {
</details>
### Tuning your trusted servers
:::important Important: Vet your trusted servers!
Trusted servers are your first point of contact when obtaining public keys from other servers, and they could theoretically impersonate other servers and cause significant harm to your deployment. Please thoroughly verify your trusted servers' credibility before adding them to your configuration.
If you need suggestions for trusted servers, ask in the Continuwuity main room.
:::
Trusted servers are queried sequentially in the order they are listed. If you have multiple trusted servers configured, put the faster ones first:
```toml
# Example config, using maintainers' recommended homeservers
trusted_servers = ["codestorm.net","starstruck.systems","unredacted.org","matrix.org"]
```
Avoid prioritising `matrix.org` as your primary trusted server, as it tends to be quite slow.
Some users have also reported that increasing `trusted_server_batch_size` helps with faster joins for huge rooms. Start with doubling the default to `2048` until you find a suitable value.
### Enable HTTP/3 on your reverse proxy
Consider enabling the newer **HTTP/3** protocol for inbound connections to Continuwuity. In Caddy HTTP/3 is allowed by default, but you must expose port :443/**udp** on your firewall.
+6 -4
View File
@@ -78,9 +78,9 @@ To debug and test your UnifiedPush installation, use the [UP-Example] app for An
### UnifiedPush and push gateways
The push gateway URL is usually pre-configured by the client app. However, some clients (such as Element X) implement UnifiedPush's [Gateway Discovery] mechanism, and attempt to detect a push gateway _on the same domain as the push provider_, before falling back to a hardcoded one.
Normally, the push gateway URL is pre-configured by the client app. However, some clients (such as Element X) implement UnifiedPush's [Gateway Discovery] mechanism, and attempt to detect a push gateway _on the same domain as the push provider_ to use before falling back to a hardcoded one. Users of these clients can therefore self-host their own push gateway.
Per the [spec][spec-push-gateway-api], these push gateways must be available on the `/_matrix/push/v1/notify` path of said domain. To test that a push gateway is accessible to the homeserver, make a request for it from the homeserver's IP:
Per the [spec][spec-push-gateway-api], these push gateways must be available on the `/_matrix/push/v1/notify` path of said domain. To test that a push gateway is usable, make a request to it from both the homeserver and the client. The returned response should look like the following:
```txt
curl https://push.example.com/_matrix/push/v1/notify
@@ -122,7 +122,7 @@ Community tutorials for selfhosting some of these stacks are listed below. Unles
- Push gateway: [common-proxies]
- Push server: [ejabberd's mod_unified_push]
- Push app: [Conversations] (and forks)
- [Element X push notifications via Conversations][alfredolin-xmpp], by Alfredo Lin.
- [Element X push notifications via Conversations][alfredolin-xmpp], by Alfredolin.
- Push gateway: [common-proxies]
- Push server: [Prosody's mod_unified_push]
- Push app: [Conversations]
@@ -156,6 +156,8 @@ WARN conduwuit_service::pusher: Pusher URL is a forbidden remote address url=htt
To fix this, remove your push gateway or push server's address from `ip_range_denylist`, and restart Continuwuity to apply the changes.
Docker deployments with the push server/gateway on the same host as Continuwuity may run into some further network issues. In this case, consult the [Docker loopback networking issues] section and its workarounds.
### Push notifications not audible/not displaying
This is likely an issue with your account's _push rules_, which determines which events are shown to to the user as notifications. Check these push rules in the client's notifications settings and tweak accordingly. Continuwuity also provides an [`!admin users reset-push-rules`](../reference/admin/users#admin-users-reset-push-rules) command, which restores the rules to the default of notifying for every message.
@@ -175,7 +177,7 @@ If you find spurious log entries like this in your Continuwuity deployment:
WARN push{user_id="@user:example.com" pushkey="very-long-string-of-letters" events=1}:send_push_notice:send_notice:send_request: conduwuit_service::pusher: Could not send request to pusher https//push.example.com: error sending request for url (https://push.example.com/_matrix/push/v1/notify)
```
Then, your server is most likely sending push requests to too many invalid targets. This could be due to one of the following reasons:
Then, your server is sending push requests to too many invalid targets. This could be due to one of the following reasons:
#### Connection issues between the server and the push gateway
+1 -1
View File
@@ -130,7 +130,7 @@ pub enum Error {
#[error(transparent)]
SignatureVerification(#[from] ruma::signatures::VerificationError),
#[error(transparent)]
StateRes(#[from] crate::state_res::Error),
StateRes(#[from] ruma::state_res::Error),
#[error("uiaa")]
Uiaa(ruma::api::client::uiaa::UiaaInfo),
-2
View File
@@ -3,10 +3,8 @@
pub mod event;
pub mod pdu;
pub mod state_key;
pub mod state_res;
pub mod versions;
pub use event::{Event, TypeExt as EventTypeExt};
pub use pdu::{PartialPdu, Pdu, PduCount, PduEvent, PduId, RawPduId, ShortId};
pub use state_key::StateKey;
pub use state_res::{StateMap, TypeStateKey};
+35
View File
@@ -73,6 +73,9 @@ pub struct Pdu {
// BTreeMap<Box<ServerName>, BTreeMap<ServerSigningKeyId, String>>
#[serde(default, skip_serializing_if = "Option::is_none")]
pub signatures: Option<Box<RawJsonValue>>,
#[serde(default)]
pub rejected: bool,
}
/// Content hashes of a PDU.
@@ -254,3 +257,35 @@ impl Ord for Pdu {
impl PartialOrd for Pdu {
fn partial_cmp(&self, other: &Self) -> Option<Ordering> { Some(self.cmp(other)) }
}
impl ruma::state_res::Event for Pdu {
type Id = OwnedEventId;
fn event_id(&self) -> &Self::Id { &self.event_id }
fn room_id(&self) -> Option<&RoomId> { self.room_id.as_deref() }
fn sender(&self) -> &UserId { self.sender.as_ref() }
fn origin_server_ts(&self) -> MilliSecondsSinceUnixEpoch {
MilliSecondsSinceUnixEpoch(self.origin_server_ts)
}
fn event_type(&self) -> &TimelineEventType { &self.kind }
fn content(&self) -> &RawJsonValue { &self.content }
fn state_key(&self) -> Option<&str> { self.state_key.as_deref() }
fn prev_events(&self) -> Box<dyn DoubleEndedIterator<Item = &Self::Id> + '_> {
Box::new(self.prev_events.iter())
}
fn auth_events(&self) -> Box<dyn DoubleEndedIterator<Item = &Self::Id> + '_> {
Box::new(self.auth_events.iter())
}
fn redacts(&self) -> Option<&Self::Id> { self.redacts.as_ref() }
fn rejected(&self) -> bool { self.rejected }
}
-17
View File
@@ -1,17 +0,0 @@
//! Permission is hereby granted, free of charge, to any person obtaining a copy
//! of this software and associated documentation files (the "Software"), to
//! deal in the Software without restriction, including without limitation the
//! rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
//! sell copies of the Software, and to permit persons to whom the Software is
//! furnished to do so, subject to the following conditions:
//! The above copyright notice and this permission notice shall be included in
//! all copies or substantial portions of the Software.
//! THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
//! IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
//! FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
//! AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
//! LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
//! FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
//! IN THE SOFTWARE.
-554
View File
@@ -1,554 +0,0 @@
#[cfg(conduwuit_bench)]
extern crate test;
use std::{
borrow::Borrow,
collections::{HashMap, HashSet},
sync::atomic::{AtomicU64, Ordering::SeqCst},
};
use futures::{future, future::ready};
use maplit::{btreemap, hashmap, hashset};
use ruma::{
EventId, MilliSecondsSinceUnixEpoch, OwnedEventId, RoomId, RoomVersionId, Signatures, UserId,
events::{
StateEventType, TimelineEventType,
room::{
join_rules::{JoinRule, RoomJoinRulesEventContent},
member::{MembershipState, RoomMemberEventContent},
},
},
int, room_id, uint, user_id,
};
use serde_json::{
json,
value::{RawValue as RawJsonValue, to_raw_value as to_raw_json_value},
};
use crate::{
matrix::{Event, Pdu, pdu::EventHash},
state_res::{self as state_res, Error, Result, StateMap},
};
static SERVER_TIMESTAMP: AtomicU64 = AtomicU64::new(0);
#[cfg(conduwuit_bench)]
#[cfg_attr(conduwuit_bench, bench)]
fn lexico_topo_sort(c: &mut test::Bencher) {
let graph = hashmap! {
event_id("l") => hashset![event_id("o")],
event_id("m") => hashset![event_id("n"), event_id("o")],
event_id("n") => hashset![event_id("o")],
event_id("o") => hashset![], // "o" has zero outgoing edges but 4 incoming edges
event_id("p") => hashset![event_id("o")],
};
c.iter(|| {
let _ = state_res::lexicographical_topological_sort(&graph, &|_| {
future::ok((int!(0), MilliSecondsSinceUnixEpoch(uint!(0))))
});
});
}
#[cfg(conduwuit_bench)]
#[cfg_attr(conduwuit_bench, bench)]
fn resolution_shallow_auth_chain(c: &mut test::Bencher) {
let mut store = TestStore(hashmap! {});
// build up the DAG
let (state_at_bob, state_at_charlie, _) = store.set_up();
c.iter(|| async {
let ev_map = store.0.clone();
let state_sets = [&state_at_bob, &state_at_charlie];
let fetch = |id: OwnedEventId| ready(ev_map.get(&id).map(ToOwned::to_owned));
let exists = |id: OwnedEventId| ready(ev_map.get(&id).is_some());
let auth_chain_sets: Vec<HashSet<_>> = state_sets
.iter()
.map(|map| {
store
.auth_event_ids(room_id(), map.values().cloned().collect())
.unwrap()
})
.collect();
let _ = match state_res::resolve(
&RoomVersionId::V6,
state_sets.into_iter(),
&auth_chain_sets,
&fetch,
&exists,
)
.await
{
| Ok(state) => state,
| Err(e) => panic!("{e}"),
};
});
}
#[cfg(conduwuit_bench)]
#[cfg_attr(conduwuit_bench, bench)]
fn resolve_deeper_event_set(c: &mut test::Bencher) {
let mut inner = INITIAL_EVENTS();
let ban = BAN_STATE_SET();
inner.extend(ban);
let store = TestStore(inner.clone());
let state_set_a = [
inner.get(&event_id("CREATE")).unwrap(),
inner.get(&event_id("IJR")).unwrap(),
inner.get(&event_id("IMA")).unwrap(),
inner.get(&event_id("IMB")).unwrap(),
inner.get(&event_id("IMC")).unwrap(),
inner.get(&event_id("MB")).unwrap(),
inner.get(&event_id("PA")).unwrap(),
]
.iter()
.map(|ev| {
(
(ev.event_type().clone().into(), ev.state_key().unwrap().into()),
ev.event_id().to_owned(),
)
})
.collect::<StateMap<_>>();
let state_set_b = [
inner.get(&event_id("CREATE")).unwrap(),
inner.get(&event_id("IJR")).unwrap(),
inner.get(&event_id("IMA")).unwrap(),
inner.get(&event_id("IMB")).unwrap(),
inner.get(&event_id("IMC")).unwrap(),
inner.get(&event_id("IME")).unwrap(),
inner.get(&event_id("PA")).unwrap(),
]
.iter()
.map(|ev| {
(
(ev.event_type().clone().into(), ev.state_key().unwrap().into()),
ev.event_id().to_owned(),
)
})
.collect::<StateMap<_>>();
c.iter(|| async {
let state_sets = [&state_set_a, &state_set_b];
let auth_chain_sets: Vec<HashSet<_>> = state_sets
.iter()
.map(|map| {
store
.auth_event_ids(room_id(), map.values().cloned().collect())
.unwrap()
})
.collect();
let fetch = |id: OwnedEventId| ready(inner.get(&id).map(ToOwned::to_owned));
let exists = |id: OwnedEventId| ready(inner.get(&id).is_some());
let _ = match state_res::resolve(
&RoomVersionId::V6,
state_sets.into_iter(),
&auth_chain_sets,
&fetch,
&exists,
)
.await
{
| Ok(state) => state,
| Err(_) => panic!("resolution failed during benchmarking"),
};
});
}
//*/////////////////////////////////////////////////////////////////////
//
// IMPLEMENTATION DETAILS AHEAD
//
/////////////////////////////////////////////////////////////////////*/
struct TestStore<E: Event>(HashMap<OwnedEventId, E>);
#[allow(unused)]
impl<E: Event + Clone> TestStore<E> {
fn get_event(&self, room_id: &RoomId, event_id: &EventId) -> Result<E> {
self.0
.get(event_id)
.cloned()
.ok_or_else(|| Error::NotFound(format!("{} not found", event_id)))
}
/// Returns the events that correspond to the `event_ids` sorted in the same
/// order.
fn get_events(&self, room_id: &RoomId, event_ids: &[OwnedEventId]) -> Result<Vec<E>> {
let mut events = vec![];
for id in event_ids {
events.push(self.get_event(room_id, id)?);
}
Ok(events)
}
/// Returns a Vec of the related auth events to the given `event`.
fn auth_event_ids(
&self,
room_id: &RoomId,
event_ids: Vec<OwnedEventId>,
) -> Result<HashSet<OwnedEventId>> {
let mut result = HashSet::new();
let mut stack = event_ids;
// DFS for auth event chain
while !stack.is_empty() {
let ev_id = stack.pop().unwrap();
if result.contains(&ev_id) {
continue;
}
result.insert(ev_id.clone());
let event = self.get_event(room_id, ev_id.borrow())?;
stack.extend(event.auth_events().map(ToOwned::to_owned));
}
Ok(result)
}
/// Returns a vector representing the difference in auth chains of the given
/// `events`.
fn auth_chain_diff(
&self,
room_id: &RoomId,
event_ids: Vec<Vec<OwnedEventId>>,
) -> Result<Vec<OwnedEventId>> {
let mut auth_chain_sets = vec![];
for ids in event_ids {
// TODO state store `auth_event_ids` returns self in the event ids
// list when an event returns `auth_event_ids` self is not
// contained
let chain = self
.auth_event_ids(room_id, ids)?
.into_iter()
.collect::<HashSet<_>>();
auth_chain_sets.push(chain);
}
if let Some(first) = auth_chain_sets.first().cloned() {
let common = auth_chain_sets
.iter()
.skip(1)
.fold(first, |a, b| a.intersection(b).cloned().collect::<HashSet<_>>());
Ok(auth_chain_sets
.into_iter()
.flatten()
.filter(|id| !common.contains(id))
.collect())
} else {
Ok(vec![])
}
}
}
impl TestStore<Pdu> {
#[allow(clippy::type_complexity)]
fn set_up(
&mut self,
) -> (StateMap<OwnedEventId>, StateMap<OwnedEventId>, StateMap<OwnedEventId>) {
let create_event = to_pdu_event::<&EventId>(
"CREATE",
alice(),
TimelineEventType::RoomCreate,
Some(""),
to_raw_json_value(&json!({ "creator": alice() })).unwrap(),
&[],
&[],
);
let cre = create_event.event_id().to_owned();
self.0.insert(cre.clone(), create_event.clone());
let alice_mem = to_pdu_event(
"IMA",
alice(),
TimelineEventType::RoomMember,
Some(alice().to_string().as_str()),
member_content_join(),
&[cre.clone()],
&[cre.clone()],
);
self.0
.insert(alice_mem.event_id().to_owned(), alice_mem.clone());
let join_rules = to_pdu_event(
"IJR",
alice(),
TimelineEventType::RoomJoinRules,
Some(""),
to_raw_json_value(&RoomJoinRulesEventContent::new(JoinRule::Public)).unwrap(),
&[cre.clone(), alice_mem.event_id().to_owned()],
&[alice_mem.event_id().to_owned()],
);
self.0
.insert(join_rules.event_id().to_owned(), join_rules.clone());
// Bob and Charlie join at the same time, so there is a fork
// this will be represented in the state_sets when we resolve
let bob_mem = to_pdu_event(
"IMB",
bob(),
TimelineEventType::RoomMember,
Some(bob().to_string().as_str()),
member_content_join(),
&[cre.clone(), join_rules.event_id().to_owned()],
&[join_rules.event_id().to_owned()],
);
self.0
.insert(bob_mem.event_id().to_owned(), bob_mem.clone());
let charlie_mem = to_pdu_event(
"IMC",
charlie(),
TimelineEventType::RoomMember,
Some(charlie().to_string().as_str()),
member_content_join(),
&[cre, join_rules.event_id().to_owned()],
&[join_rules.event_id().to_owned()],
);
self.0
.insert(charlie_mem.event_id().to_owned(), charlie_mem.clone());
let state_at_bob = [&create_event, &alice_mem, &join_rules, &bob_mem]
.iter()
.map(|ev| {
(
(ev.event_type().to_string().into(), ev.state_key().unwrap().into()),
ev.event_id().to_owned(),
)
})
.collect::<StateMap<_>>();
let state_at_charlie = [&create_event, &alice_mem, &join_rules, &charlie_mem]
.iter()
.map(|ev| {
(
(ev.event_type().to_string().into(), ev.state_key().unwrap().into()),
ev.event_id().to_owned(),
)
})
.collect::<StateMap<_>>();
let expected = [&create_event, &alice_mem, &join_rules, &bob_mem, &charlie_mem]
.iter()
.map(|ev| {
(
(ev.event_type().to_string().into(), ev.state_key().unwrap().into()),
ev.event_id().to_owned(),
)
})
.collect::<StateMap<_>>();
(state_at_bob, state_at_charlie, expected)
}
}
fn event_id(id: &str) -> OwnedEventId {
if id.contains('$') {
return id.try_into().unwrap();
}
format!("${}:foo", id).try_into().unwrap()
}
fn alice() -> &'static UserId { user_id!("@alice:foo") }
fn bob() -> &'static UserId { user_id!("@bob:foo") }
fn charlie() -> &'static UserId { user_id!("@charlie:foo") }
fn ella() -> &'static UserId { user_id!("@ella:foo") }
fn room_id() -> &'static RoomId { room_id!("!test:foo") }
fn member_content_ban() -> Box<RawJsonValue> {
to_raw_json_value(&RoomMemberEventContent::new(MembershipState::Ban)).unwrap()
}
fn member_content_join() -> Box<RawJsonValue> {
to_raw_json_value(&RoomMemberEventContent::new(MembershipState::Join)).unwrap()
}
fn to_pdu_event<S>(
id: &str,
sender: &UserId,
ev_type: TimelineEventType,
state_key: Option<&str>,
content: Box<RawJsonValue>,
auth_events: &[S],
prev_events: &[S],
) -> Pdu
where
S: AsRef<str>,
{
// We don't care if the addition happens in order just that it is atomic
// (each event has its own value)
let ts = SERVER_TIMESTAMP.fetch_add(1, SeqCst);
let id = if id.contains('$') {
id.to_owned()
} else {
format!("${}:foo", id)
};
let auth_events = auth_events
.iter()
.map(AsRef::as_ref)
.map(event_id)
.collect::<Vec<_>>();
let prev_events = prev_events
.iter()
.map(AsRef::as_ref)
.map(event_id)
.collect::<Vec<_>>();
Pdu {
event_id: id.try_into().unwrap(),
room_id: Some(room_id().to_owned()),
sender: sender.to_owned(),
origin_server_ts: ts.try_into().unwrap(),
state_key: state_key.map(Into::into),
kind: ev_type,
content,
origin: None,
redacts: None,
sticky: None,
unsigned: None,
auth_events,
prev_events,
depth: uint!(0),
hashes: EventHash { sha256: String::new() },
signatures: None,
}
}
// all graphs start with these input events
#[allow(non_snake_case)]
fn INITIAL_EVENTS() -> HashMap<OwnedEventId, Pdu> {
vec![
to_pdu_event::<&EventId>(
"CREATE",
alice(),
TimelineEventType::RoomCreate,
Some(""),
to_raw_json_value(&json!({ "creator": alice() })).unwrap(),
&[],
&[],
),
to_pdu_event(
"IMA",
alice(),
TimelineEventType::RoomMember,
Some(alice().as_str()),
member_content_join(),
&["CREATE"],
&["CREATE"],
),
to_pdu_event(
"IPOWER",
alice(),
TimelineEventType::RoomPowerLevels,
Some(""),
to_raw_json_value(&json!({ "users": { alice(): 100 } })).unwrap(),
&["CREATE", "IMA"],
&["IMA"],
),
to_pdu_event(
"IJR",
alice(),
TimelineEventType::RoomJoinRules,
Some(""),
to_raw_json_value(&RoomJoinRulesEventContent::new(JoinRule::Public)).unwrap(),
&["CREATE", "IMA", "IPOWER"],
&["IPOWER"],
),
to_pdu_event(
"IMB",
bob(),
TimelineEventType::RoomMember,
Some(bob().to_string().as_str()),
member_content_join(),
&["CREATE", "IJR", "IPOWER"],
&["IJR"],
),
to_pdu_event(
"IMC",
charlie(),
TimelineEventType::RoomMember,
Some(charlie().to_string().as_str()),
member_content_join(),
&["CREATE", "IJR", "IPOWER"],
&["IMB"],
),
to_pdu_event::<&EventId>(
"START",
charlie(),
TimelineEventType::RoomTopic,
Some(""),
to_raw_json_value(&json!({})).unwrap(),
&[],
&[],
),
to_pdu_event::<&EventId>(
"END",
charlie(),
TimelineEventType::RoomTopic,
Some(""),
to_raw_json_value(&json!({})).unwrap(),
&[],
&[],
),
]
.into_iter()
.map(|ev| (ev.event_id().to_owned(), ev))
.collect()
}
// all graphs start with these input events
#[allow(non_snake_case)]
fn BAN_STATE_SET() -> HashMap<OwnedEventId, Pdu> {
vec![
to_pdu_event(
"PA",
alice(),
TimelineEventType::RoomPowerLevels,
Some(""),
to_raw_json_value(&json!({ "users": { alice(): 100, bob(): 50 } })).unwrap(),
&["CREATE", "IMA", "IPOWER"], // auth_events
&["START"], // prev_events
),
to_pdu_event(
"PB",
alice(),
TimelineEventType::RoomPowerLevels,
Some(""),
to_raw_json_value(&json!({ "users": { alice(): 100, bob(): 50 } })).unwrap(),
&["CREATE", "IMA", "IPOWER"],
&["END"],
),
to_pdu_event(
"MB",
alice(),
TimelineEventType::RoomMember,
Some(ella().as_str()),
member_content_ban(),
&["CREATE", "IMA", "PB"],
&["PA"],
),
to_pdu_event(
"IME",
ella(),
TimelineEventType::RoomMember,
Some(ella().as_str()),
member_content_join(),
&["CREATE", "IJR", "PA"],
&["MB"],
),
]
.into_iter()
.map(|ev| (ev.event_id().to_owned(), ev))
.collect()
}
-23
View File
@@ -1,23 +0,0 @@
use serde_json::Error as JsonError;
use thiserror::Error;
/// Represents the various errors that arise when resolving state.
#[derive(Error, Debug)]
#[non_exhaustive]
pub enum Error {
/// A deserialization error.
#[error(transparent)]
SerdeJson(#[from] JsonError),
/// The given option or version is unsupported.
#[error("Unsupported room version: {0}")]
Unsupported(String),
/// The given event was not found.
#[error("Not found error: {0}")]
NotFound(String),
/// Invalid fields in the given PDU.
#[error("Invalid PDU: {0}")]
InvalidPdu(String),
}
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
-104
View File
@@ -1,104 +0,0 @@
11/29/2020 BRANCH: timo-spec-comp REV: d2a85669cc6056679ce6ca0fde4658a879ad2b08
lexicographical topological sort
time: [1.7123 us 1.7157 us 1.7199 us]
change: [-1.7584% -1.5433% -1.3205%] (p = 0.00 < 0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
2 (2.00%) low mild
5 (5.00%) high mild
1 (1.00%) high severe
resolve state of 5 events one fork
time: [10.981 us 10.998 us 11.020 us]
Found 3 outliers among 100 measurements (3.00%)
3 (3.00%) high mild
resolve state of 10 events 3 conflicting
time: [26.858 us 26.946 us 27.037 us]
11/29/2020 BRANCH: event-trait REV: f0eb1310efd49d722979f57f20bd1ac3592b0479
lexicographical topological sort
time: [1.7686 us 1.7738 us 1.7810 us]
change: [-3.2752% -2.4634% -1.7635%] (p = 0.00 < 0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severe
resolve state of 5 events one fork
time: [10.643 us 10.656 us 10.669 us]
change: [-4.9990% -3.8078% -2.8319%] (p = 0.00 < 0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severe
resolve state of 10 events 3 conflicting
time: [29.149 us 29.252 us 29.375 us]
change: [-0.8433% -0.3270% +0.2656%] (p = 0.25 > 0.05)
No change in performance detected.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
4/26/2020 BRANCH: fix-test-serde REV:
lexicographical topological sort
time: [1.6793 us 1.6823 us 1.6857 us]
Found 9 outliers among 100 measurements (9.00%)
1 (1.00%) low mild
4 (4.00%) high mild
4 (4.00%) high severe
resolve state of 5 events one fork
time: [9.9993 us 10.062 us 10.159 us]
Found 9 outliers among 100 measurements (9.00%)
7 (7.00%) high mild
2 (2.00%) high severe
resolve state of 10 events 3 conflicting
time: [26.004 us 26.092 us 26.195 us]
Found 16 outliers among 100 measurements (16.00%)
11 (11.00%) high mild
5 (5.00%) high severe
6/30/2021 BRANCH: state-closure REV: 174c3e2a72232ad75b3fb14b3551f5f746f4fe84
lexicographical topological sort
time: [1.5496 us 1.5536 us 1.5586 us]
Found 9 outliers among 100 measurements (9.00%)
1 (1.00%) low mild
1 (1.00%) high mild
7 (7.00%) high severe
resolve state of 5 events one fork
time: [10.319 us 10.333 us 10.347 us]
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high severe
resolve state of 10 events 3 conflicting
time: [25.770 us 25.805 us 25.839 us]
Found 7 outliers among 100 measurements (7.00%)
5 (5.00%) high mild
2 (2.00%) high severe
7/20/2021 BRANCH stateres-result REV:
This marks the switch to HashSet/Map
lexicographical topological sort
time: [1.8122 us 1.8177 us 1.8233 us]
change: [+15.205% +15.919% +16.502%] (p = 0.00 < 0.05)
Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
5 (5.00%) high mild
2 (2.00%) high severe
resolve state of 5 events one fork
time: [11.966 us 12.010 us 12.059 us]
change: [+16.089% +16.730% +17.469%] (p = 0.00 < 0.05)
Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
3 (3.00%) high mild
4 (4.00%) high severe
resolve state of 10 events 3 conflicting
time: [29.092 us 29.201 us 29.311 us]
change: [+12.447% +12.847% +13.280%] (p = 0.00 < 0.05)
Performance has regressed.
Found 9 outliers among 100 measurements (9.00%)
6 (6.00%) high mild
3 (3.00%) high severe
-241
View File
@@ -1,241 +0,0 @@
use std::collections::BTreeMap;
use ruma::{
Int, OwnedUserId, UserId,
events::{TimelineEventType, room::power_levels::RoomPowerLevelsEventContent},
power_levels::{NotificationPowerLevels, default_power_level},
room_version_rules::{AuthorizationRules, RoomVersionRules},
serde::deserialize_v1_powerlevel,
};
use serde::Deserialize;
use serde_json::{Error, from_str as from_json_str};
use super::{
Result,
serde_backports::{
vec_deserialize_int_powerlevel_values, vec_deserialize_v1_powerlevel_values,
},
};
use crate::error;
#[derive(Deserialize)]
struct IntRoomPowerLevelsEventContent {
#[serde(default = "default_power_level")]
ban: Int,
#[serde(default)]
events: BTreeMap<TimelineEventType, Int>,
#[serde(default)]
events_default: Int,
#[serde(default)]
invite: Int,
#[serde(default = "default_power_level")]
kick: Int,
#[serde(default = "default_power_level")]
redact: Int,
#[serde(default = "default_power_level")]
state_default: Int,
#[serde(default)]
users: BTreeMap<OwnedUserId, Int>,
#[serde(default)]
users_default: Int,
#[serde(default)]
notifications: IntNotificationPowerLevels,
}
impl IntRoomPowerLevelsEventContent {
fn into_room_power_levels_content(
self,
auth_rules: &AuthorizationRules,
) -> RoomPowerLevelsEventContent {
let Self {
ban,
events,
events_default,
invite,
kick,
redact,
state_default,
users,
users_default,
notifications,
} = self;
let mut pl = RoomPowerLevelsEventContent::new(auth_rules);
pl.ban = ban;
pl.events = events;
pl.events_default = events_default;
pl.invite = invite;
pl.kick = kick;
pl.redact = redact;
pl.state_default = state_default;
pl.users = users;
pl.users_default = users_default;
pl.notifications = notifications.into();
pl
}
}
#[derive(Deserialize)]
struct IntNotificationPowerLevels {
#[serde(default = "default_power_level")]
room: Int,
}
impl Default for IntNotificationPowerLevels {
fn default() -> Self { Self { room: default_power_level() } }
}
impl From<IntNotificationPowerLevels> for NotificationPowerLevels {
fn from(int_notif: IntNotificationPowerLevels) -> Self {
let mut notif = Self::new();
notif.room = int_notif.room;
notif
}
}
#[inline]
pub(crate) fn deserialize_power_levels(
content: &str,
room_version: &RoomVersionRules,
) -> Option<RoomPowerLevelsEventContent> {
if room_version.authorization.integer_power_levels {
deserialize_integer_power_levels(content, &room_version.authorization)
} else {
deserialize_legacy_power_levels(content)
}
}
fn deserialize_integer_power_levels(
content: &str,
auth_rules: &AuthorizationRules,
) -> Option<RoomPowerLevelsEventContent> {
match from_json_str::<IntRoomPowerLevelsEventContent>(content) {
| Ok(content) => Some(content.into_room_power_levels_content(auth_rules)),
| Err(_) => {
error!("m.room.power_levels event is not valid with integer values");
None
},
}
}
fn deserialize_legacy_power_levels(content: &str) -> Option<RoomPowerLevelsEventContent> {
match from_json_str(content) {
| Ok(content) => Some(content),
| Err(_) => {
error!(
"m.room.power_levels event is not valid with integer or string integer values"
);
None
},
}
}
#[derive(Deserialize)]
pub(crate) struct PowerLevelsContentFields {
#[serde(default, deserialize_with = "vec_deserialize_v1_powerlevel_values")]
pub(crate) users: Vec<(OwnedUserId, Int)>,
#[serde(default, deserialize_with = "deserialize_v1_powerlevel")]
pub(crate) users_default: Int,
}
impl PowerLevelsContentFields {
pub(crate) fn get_user_power(&self, user_id: &UserId) -> Option<&Int> {
let comparator = |item: &(OwnedUserId, Int)| {
let item: &UserId = &item.0;
item.cmp(user_id)
};
self.users
.binary_search_by(comparator)
.ok()
.and_then(|idx| self.users.get(idx).map(|item| &item.1))
}
}
#[derive(Deserialize)]
struct IntPowerLevelsContentFields {
#[serde(default, deserialize_with = "vec_deserialize_int_powerlevel_values")]
users: Vec<(OwnedUserId, Int)>,
#[serde(default)]
users_default: Int,
}
impl From<IntPowerLevelsContentFields> for PowerLevelsContentFields {
fn from(pl: IntPowerLevelsContentFields) -> Self {
let IntPowerLevelsContentFields { users, users_default } = pl;
Self { users, users_default }
}
}
#[derive(Deserialize)]
pub(crate) struct PowerLevelsContentInvite {
#[serde(default, deserialize_with = "deserialize_v1_powerlevel")]
pub(crate) invite: Int,
}
#[derive(Deserialize)]
struct IntPowerLevelsContentInvite {
#[serde(default)]
invite: Int,
}
impl From<IntPowerLevelsContentInvite> for PowerLevelsContentInvite {
fn from(pl: IntPowerLevelsContentInvite) -> Self {
let IntPowerLevelsContentInvite { invite } = pl;
Self { invite }
}
}
pub(crate) fn deserialize_power_levels_content_invite(
content: &str,
room_version: &RoomVersionRules,
) -> Result<PowerLevelsContentInvite, Error> {
if room_version.authorization.integer_power_levels {
from_json_str::<IntPowerLevelsContentInvite>(content).map(Into::into)
} else {
from_json_str(content)
}
}
#[derive(Deserialize)]
pub(crate) struct PowerLevelsContentRedact {
#[serde(default = "default_power_level", deserialize_with = "deserialize_v1_powerlevel")]
pub(crate) redact: Int,
}
#[derive(Deserialize)]
pub(crate) struct IntPowerLevelsContentRedact {
#[serde(default = "default_power_level")]
redact: Int,
}
impl From<IntPowerLevelsContentRedact> for PowerLevelsContentRedact {
fn from(pl: IntPowerLevelsContentRedact) -> Self {
let IntPowerLevelsContentRedact { redact } = pl;
Self { redact }
}
}
pub(crate) fn deserialize_power_levels_content_redact(
content: &str,
room_version: &RoomVersionRules,
) -> Result<PowerLevelsContentRedact, Error> {
if room_version.authorization.integer_power_levels {
from_json_str::<IntPowerLevelsContentRedact>(content).map(Into::into)
} else {
from_json_str(content)
}
}
@@ -1,124 +0,0 @@
//! These functions are copied from an old version of Ruma. power_levels.rs uses
//! them to lazily deserialize power level events. Upstream Ruma uses a much
//! more elegant approach in its state resolution code, which we may want
//! to look into at some point.
use std::{fmt, marker::PhantomData};
use ruma::{Int, serde::deserialize_v1_powerlevel};
use serde::{
Deserialize, Deserializer,
de::{MapAccess, Visitor},
};
/// Take a Map with values of either an integer number or a string and
/// deserialize those to integer numbers in a Vec of sorted pairs.
///
/// To be used like this:
/// `#[serde(deserialize_with = "vec_deserialize_v1_powerlevel_values")]`
pub(super) fn vec_deserialize_v1_powerlevel_values<'de, D, T>(
de: D,
) -> Result<Vec<(T, Int)>, D::Error>
where
D: Deserializer<'de>,
T: Deserialize<'de> + Ord,
{
#[repr(transparent)]
struct IntWrap(Int);
impl<'de> Deserialize<'de> for IntWrap {
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: Deserializer<'de>,
{
deserialize_v1_powerlevel(deserializer).map(IntWrap)
}
}
struct IntMapVisitor<T> {
_phantom: PhantomData<T>,
}
impl<T> IntMapVisitor<T> {
fn new() -> Self { Self { _phantom: PhantomData } }
}
impl<'de, T> Visitor<'de> for IntMapVisitor<T>
where
T: Deserialize<'de> + Ord,
{
type Value = Vec<(T, Int)>;
fn expecting(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter.write_str("a map with integers or strings as values")
}
fn visit_map<A: MapAccess<'de>>(self, mut map: A) -> Result<Self::Value, A::Error> {
let mut res = Vec::new();
if let Some(hint) = map.size_hint() {
res.reserve(hint);
}
while let Some((k, IntWrap(v))) = map.next_entry()? {
res.push((k, v));
}
res.sort_unstable();
res.dedup_by(|a, b| a.0 == b.0);
Ok(res)
}
}
de.deserialize_map(IntMapVisitor::new())
}
/// Take a Map with integer values and deserialize those to a Vec of sorted
/// pairs
///
/// To be used like this:
/// `#[serde(deserialize_with = "vec_deserialize_int_powerlevel_values")]`
pub(super) fn vec_deserialize_int_powerlevel_values<'de, D, T>(
de: D,
) -> Result<Vec<(T, Int)>, D::Error>
where
D: Deserializer<'de>,
T: Deserialize<'de> + Ord,
{
struct IntMapVisitor<T> {
_phantom: PhantomData<T>,
}
impl<T> IntMapVisitor<T> {
fn new() -> Self { Self { _phantom: PhantomData } }
}
impl<'de, T> Visitor<'de> for IntMapVisitor<T>
where
T: Deserialize<'de> + Ord,
{
type Value = Vec<(T, Int)>;
fn expecting(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result {
formatter.write_str("a map with integers as values")
}
fn visit_map<A: MapAccess<'de>>(self, mut map: A) -> Result<Self::Value, A::Error> {
let mut res = Vec::new();
if let Some(hint) = map.size_hint() {
res.reserve(hint);
}
while let Some(item) = map.next_entry()? {
res.push(item);
}
res.sort_unstable();
res.dedup_by(|a, b| a.0 == b.0);
Ok(res)
}
}
de.deserialize_map(IntMapVisitor::new())
}
-582
View File
@@ -1,582 +0,0 @@
use std::{
borrow::Borrow,
collections::{BTreeMap, HashMap, HashSet},
sync::atomic::{AtomicU64, Ordering::SeqCst},
};
use futures::future::ready;
use ruma::{
EventId, MilliSecondsSinceUnixEpoch, OwnedEventId, RoomId, RoomVersionId, ServerSignatures,
UserId, event_id,
events::{
TimelineEventType,
room::{
join_rules::{JoinRule, RoomJoinRulesEventContent},
member::{MembershipState, RoomMemberEventContent},
},
},
int, room_id,
room_version_rules::RoomVersionRules,
uint, user_id,
};
use serde_json::{
json,
value::{RawValue as RawJsonValue, to_raw_value as to_raw_json_value},
};
use super::auth_types_for_event;
use crate::{
Result, info,
matrix::{Event, EventTypeExt, Pdu, StateMap, pdu::EventHash},
};
static SERVER_TIMESTAMP: AtomicU64 = AtomicU64::new(0);
pub(crate) async fn do_check(
events: &[Pdu],
edges: Vec<Vec<OwnedEventId>>,
expected_state_ids: Vec<OwnedEventId>,
) {
// To activate logging use `RUST_LOG=debug cargo t`
let init_events = INITIAL_EVENTS();
let mut store = TestStore(
init_events
.values()
.chain(events)
.map(|ev| (ev.event_id().to_owned(), ev.clone()))
.collect(),
);
// This will be lexi_topo_sorted for resolution
let mut graph = HashMap::new();
// This is the same as in `resolve` event_id -> OriginalStateEvent
let mut fake_event_map = HashMap::new();
// Create the DB of events that led up to this point
// TODO maybe clean up some of these clones it is just tests but...
for ev in init_events.values().chain(events) {
graph.insert(ev.event_id().to_owned(), HashSet::new());
fake_event_map.insert(ev.event_id().to_owned(), ev.clone());
}
for pair in INITIAL_EDGES().windows(2) {
if let [a, b] = &pair {
graph
.entry(a.to_owned())
.or_insert_with(HashSet::new)
.insert(b.clone());
}
}
for edge_list in edges {
for pair in edge_list.windows(2) {
if let [a, b] = &pair {
graph
.entry(a.to_owned())
.or_insert_with(HashSet::new)
.insert(b.clone());
}
}
}
// event_id -> Pdu
let mut event_map: HashMap<OwnedEventId, Pdu> = HashMap::new();
// event_id -> StateMap<OwnedEventId>
let mut state_at_event: HashMap<OwnedEventId, StateMap<OwnedEventId>> = HashMap::new();
// Resolve the current state and add it to the state_at_event map then
// continue on in "time"
for node in super::lexicographical_topological_sort(&graph, &|_id| async {
Ok((int!(0), MilliSecondsSinceUnixEpoch(uint!(0))))
})
.await
.unwrap()
{
let fake_event = fake_event_map.get(&node).unwrap();
let event_id = fake_event.event_id().to_owned();
let prev_events = graph.get(&node).unwrap();
let state_before: StateMap<OwnedEventId> = if prev_events.is_empty() {
HashMap::new()
} else if prev_events.len() == 1 {
state_at_event
.get(prev_events.iter().next().unwrap())
.unwrap()
.clone()
} else {
let state_sets = prev_events
.iter()
.filter_map(|k| state_at_event.get(k))
.collect::<Vec<_>>();
info!(
"{:#?}",
state_sets
.iter()
.map(|map| map
.iter()
.map(|((ty, key), id)| format!("(({ty}{key:?}), {id})"))
.collect::<Vec<_>>())
.collect::<Vec<_>>()
);
let auth_chain_sets: Vec<_> = state_sets
.iter()
.map(|map| {
store
.auth_event_ids(room_id(), map.values().cloned().collect())
.unwrap()
})
.collect();
let event_map = &event_map;
let fetch = |id: OwnedEventId| ready(event_map.get(&id).cloned());
let exists = |id: OwnedEventId| ready(event_map.get(&id).is_some());
let resolved = super::resolve(
&RoomVersionRules::V6,
state_sets,
&auth_chain_sets,
&fetch,
&exists,
)
.await;
match resolved {
| Ok(state) => state,
| Err(e) => panic!("resolution for {node} failed: {e}"),
}
};
let mut state_after = state_before.clone();
let ty = fake_event.event_type();
let key = fake_event.state_key().unwrap();
state_after.insert(ty.with_state_key(key), event_id.to_owned());
let auth_types = auth_types_for_event(
fake_event.event_type(),
fake_event.sender(),
fake_event.state_key(),
fake_event.content(),
&RoomVersionRules::V6,
)
.unwrap();
let mut auth_events = vec![];
for key in auth_types {
if state_before.contains_key(&key) {
auth_events.push(state_before[&key].clone());
}
}
// TODO The event is just remade, adding the auth_events and prev_events
// here the `to_pdu_event` was split into `init` and the fn below,
// could be better
let e = fake_event;
let ev_id = e.event_id();
let event = to_pdu_event(
e.event_id().as_str(),
e.sender(),
e.event_type().clone(),
e.state_key(),
e.content().to_owned(),
&auth_events,
&prev_events.iter().cloned().collect::<Vec<_>>(),
);
// We have to update our store, an actual user of this lib would
// be giving us state from a DB.
store.0.insert(ev_id.to_owned(), event.clone());
state_at_event.insert(node, state_after);
event_map.insert(event_id.to_owned(), store.0.get(ev_id).unwrap().clone());
}
let mut expected_state = StateMap::new();
for node in expected_state_ids {
let ev = event_map.get(&node).unwrap_or_else(|| {
panic!(
"{node} not found in {:?}",
event_map
.keys()
.map(ToString::to_string)
.collect::<Vec<_>>()
)
});
let key = ev.event_type().with_state_key(ev.state_key().unwrap());
expected_state.insert(key, node);
}
let start_state = state_at_event.get(event_id!("$START:foo")).unwrap();
let end_state = state_at_event
.get(event_id!("$END:foo"))
.unwrap()
.iter()
.filter(|(k, v)| {
expected_state.contains_key(k)
|| start_state.get(k) != Some(*v)
// Filter out the dummy messages events.
// These act as points in time where there should be a known state to
// test against.
&& **k != ("m.room.message".into(), "dummy".into())
})
.map(|(k, v)| (k.clone(), v.clone()))
.collect::<StateMap<OwnedEventId>>();
assert_eq!(expected_state, end_state);
}
#[allow(clippy::exhaustive_structs)]
pub(crate) struct TestStore<E: Event>(pub(crate) HashMap<OwnedEventId, E>);
impl<E: Event + Clone> TestStore<E> {
pub(crate) fn get_event(&self, _: &RoomId, event_id: &EventId) -> Result<E> {
self.0
.get(event_id)
.cloned()
.ok_or_else(|| super::Error::NotFound(format!("{event_id} not found")))
.map_err(Into::into)
}
/// Returns a Vec of the related auth events to the given `event`.
pub(crate) fn auth_event_ids(
&self,
room_id: &RoomId,
event_ids: Vec<OwnedEventId>,
) -> Result<HashSet<OwnedEventId>> {
let mut result = HashSet::new();
let mut stack = event_ids;
// DFS for auth event chain
while let Some(ev_id) = stack.pop() {
if result.contains(&ev_id) {
continue;
}
result.insert(ev_id.clone());
let event = self.get_event(room_id, ev_id.borrow())?;
stack.extend(event.auth_events().map(ToOwned::to_owned));
}
Ok(result)
}
}
// A StateStore implementation for testing
#[allow(clippy::type_complexity)]
impl TestStore<Pdu> {
pub(crate) fn set_up(
&mut self,
) -> (StateMap<OwnedEventId>, StateMap<OwnedEventId>, StateMap<OwnedEventId>) {
let create_event = to_pdu_event::<&EventId>(
"CREATE",
alice(),
TimelineEventType::RoomCreate,
Some(""),
to_raw_json_value(&json!({ "creator": alice() })).unwrap(),
&[],
&[],
);
let cre = create_event.event_id().to_owned();
self.0.insert(cre.clone(), create_event.clone());
let alice_mem = to_pdu_event(
"IMA",
alice(),
TimelineEventType::RoomMember,
Some(alice().as_str()),
member_content_join(),
&[cre.clone()],
&[cre.clone()],
);
self.0
.insert(alice_mem.event_id().to_owned(), alice_mem.clone());
let join_rules = to_pdu_event(
"IJR",
alice(),
TimelineEventType::RoomJoinRules,
Some(""),
to_raw_json_value(&RoomJoinRulesEventContent::new(JoinRule::Public)).unwrap(),
&[cre.clone(), alice_mem.event_id().to_owned()],
&[alice_mem.event_id().to_owned()],
);
self.0
.insert(join_rules.event_id().to_owned(), join_rules.clone());
// Bob and Charlie join at the same time, so there is a fork
// this will be represented in the state_sets when we resolve
let bob_mem = to_pdu_event(
"IMB",
bob(),
TimelineEventType::RoomMember,
Some(bob().as_str()),
member_content_join(),
&[cre.clone(), join_rules.event_id().to_owned()],
&[join_rules.event_id().to_owned()],
);
self.0
.insert(bob_mem.event_id().to_owned(), bob_mem.clone());
let charlie_mem = to_pdu_event(
"IMC",
charlie(),
TimelineEventType::RoomMember,
Some(charlie().as_str()),
member_content_join(),
&[cre, join_rules.event_id().to_owned()],
&[join_rules.event_id().to_owned()],
);
self.0
.insert(charlie_mem.event_id().to_owned(), charlie_mem.clone());
let state_at_bob = [&create_event, &alice_mem, &join_rules, &bob_mem]
.iter()
.map(|e| {
(e.event_type().with_state_key(e.state_key().unwrap()), e.event_id().to_owned())
})
.collect::<StateMap<_>>();
let state_at_charlie = [&create_event, &alice_mem, &join_rules, &charlie_mem]
.iter()
.map(|e| {
(e.event_type().with_state_key(e.state_key().unwrap()), e.event_id().to_owned())
})
.collect::<StateMap<_>>();
let expected = [&create_event, &alice_mem, &join_rules, &bob_mem, &charlie_mem]
.iter()
.map(|e| {
(e.event_type().with_state_key(e.state_key().unwrap()), e.event_id().to_owned())
})
.collect::<StateMap<_>>();
(state_at_bob, state_at_charlie, expected)
}
}
pub(crate) fn event_id(id: &str) -> OwnedEventId {
if id.contains('$') {
return id.try_into().unwrap();
}
format!("${id}:foo").try_into().unwrap()
}
pub(crate) fn alice() -> &'static UserId { user_id!("@alice:foo") }
pub(crate) fn bob() -> &'static UserId { user_id!("@bob:foo") }
pub(crate) fn charlie() -> &'static UserId { user_id!("@charlie:foo") }
pub(crate) fn ella() -> &'static UserId { user_id!("@ella:foo") }
pub(crate) fn zara() -> &'static UserId { user_id!("@zara:foo") }
pub(crate) fn room_id() -> &'static RoomId { room_id!("!test:foo") }
pub(crate) fn member_content_ban() -> Box<RawJsonValue> {
to_raw_json_value(&RoomMemberEventContent::new(MembershipState::Ban)).unwrap()
}
pub(crate) fn member_content_join() -> Box<RawJsonValue> {
to_raw_json_value(&RoomMemberEventContent::new(MembershipState::Join)).unwrap()
}
pub(crate) fn to_init_pdu_event(
id: &str,
sender: &UserId,
ev_type: TimelineEventType,
state_key: Option<&str>,
content: Box<RawJsonValue>,
) -> Pdu {
let ts = SERVER_TIMESTAMP.fetch_add(1, SeqCst);
let id = if id.contains('$') {
id.to_owned()
} else {
format!("${id}:foo")
};
Pdu {
event_id: id.try_into().unwrap(),
room_id: Some(room_id().to_owned()),
sender: sender.to_owned(),
origin_server_ts: ts.try_into().unwrap(),
state_key: state_key.map(Into::into),
kind: ev_type,
content,
origin: None,
redacts: None,
sticky: None,
unsigned: None,
auth_events: vec![],
prev_events: vec![],
depth: uint!(0),
hashes: EventHash { sha256: "".to_owned() },
signatures: None,
}
}
pub(crate) fn to_pdu_event<S>(
id: &str,
sender: &UserId,
ev_type: TimelineEventType,
state_key: Option<&str>,
content: Box<RawJsonValue>,
auth_events: &[S],
prev_events: &[S],
) -> Pdu
where
S: AsRef<str>,
{
let ts = SERVER_TIMESTAMP.fetch_add(1, SeqCst);
let id = if id.contains('$') {
id.to_owned()
} else {
format!("${id}:foo")
};
let auth_events = auth_events
.iter()
.map(AsRef::as_ref)
.map(event_id)
.collect::<Vec<_>>();
let prev_events = prev_events
.iter()
.map(AsRef::as_ref)
.map(event_id)
.collect::<Vec<_>>();
Pdu {
event_id: id.try_into().unwrap(),
room_id: Some(room_id().to_owned()),
sender: sender.to_owned(),
origin_server_ts: ts.try_into().unwrap(),
state_key: state_key.map(Into::into),
kind: ev_type,
content,
origin: None,
redacts: None,
sticky: None,
unsigned: None,
auth_events,
prev_events,
depth: uint!(0),
hashes: EventHash { sha256: "".to_owned() },
signatures: None,
}
}
// all graphs start with these input events
#[allow(non_snake_case)]
pub(crate) fn INITIAL_EVENTS() -> HashMap<OwnedEventId, Pdu> {
vec![
to_pdu_event::<&EventId>(
"CREATE",
alice(),
TimelineEventType::RoomCreate,
Some(""),
to_raw_json_value(&json!({ "creator": alice() })).unwrap(),
&[],
&[],
),
to_pdu_event(
"IMA",
alice(),
TimelineEventType::RoomMember,
Some(alice().as_str()),
member_content_join(),
&["CREATE"],
&["CREATE"],
),
to_pdu_event(
"IPOWER",
alice(),
TimelineEventType::RoomPowerLevels,
Some(""),
to_raw_json_value(&json!({ "users": { alice(): 100 } })).unwrap(),
&["CREATE", "IMA"],
&["IMA"],
),
to_pdu_event(
"IJR",
alice(),
TimelineEventType::RoomJoinRules,
Some(""),
to_raw_json_value(&RoomJoinRulesEventContent::new(JoinRule::Public)).unwrap(),
&["CREATE", "IMA", "IPOWER"],
&["IPOWER"],
),
to_pdu_event(
"IMB",
bob(),
TimelineEventType::RoomMember,
Some(bob().as_str()),
member_content_join(),
&["CREATE", "IJR", "IPOWER"],
&["IJR"],
),
to_pdu_event(
"IMC",
charlie(),
TimelineEventType::RoomMember,
Some(charlie().as_str()),
member_content_join(),
&["CREATE", "IJR", "IPOWER"],
&["IMB"],
),
to_pdu_event::<&EventId>(
"START",
charlie(),
TimelineEventType::RoomMessage,
Some("dummy"),
to_raw_json_value(&json!({})).unwrap(),
&[],
&[],
),
to_pdu_event::<&EventId>(
"END",
charlie(),
TimelineEventType::RoomMessage,
Some("dummy"),
to_raw_json_value(&json!({})).unwrap(),
&[],
&[],
),
]
.into_iter()
.map(|ev| (ev.event_id().to_owned(), ev))
.collect()
}
// all graphs start with these input events
#[allow(non_snake_case)]
pub(crate) fn INITIAL_EVENTS_CREATE_ROOM() -> HashMap<OwnedEventId, Pdu> {
vec![to_pdu_event::<&EventId>(
"CREATE",
alice(),
TimelineEventType::RoomCreate,
Some(""),
to_raw_json_value(&json!({ "creator": alice() })).unwrap(),
&[],
&[],
)]
.into_iter()
.map(|ev| (ev.event_id().to_owned(), ev))
.collect()
}
#[allow(non_snake_case)]
pub(crate) fn INITIAL_EDGES() -> Vec<OwnedEventId> {
vec!["START", "IMC", "IMB", "IJR", "IPOWER", "IMA", "CREATE"]
.into_iter()
.map(event_id)
.collect::<Vec<_>>()
}
+1 -1
View File
@@ -35,7 +35,7 @@ pub use conduwuit_build_metadata as build_metadata;
pub use config::Config;
pub use error::Error;
pub use info::version::*;
pub use matrix::{Event, EventTypeExt, Pdu, PduCount, PduEvent, PduId, pdu, state_res};
pub use matrix::{Event, EventTypeExt, Pdu, PduCount, PduEvent, PduId, pdu};
pub use parking_lot::{
Mutex as SyncMutex, RwLock as SyncRwLock, RwLockReadGuard as SyncRwLockReadGuard,
RwLockWriteGuard as SyncRwLockWriteGuard,
+17
View File
@@ -53,4 +53,21 @@ impl super::Map {
let key = ser::serialize(buf, key).expect("failed to serialize query key");
self.get(key)
}
pub fn qry_blocking<K>(self: &Arc<Self>, key: &K) -> Result<Handle<'_>>
where
K: Serialize + ?Sized + Debug,
{
let mut buf = KeyBuf::new();
self.bqry_blocking(key, &mut buf)
}
pub fn bqry_blocking<K, B>(self: &Arc<Self>, key: &K, buf: &mut B) -> Result<Handle<'_>>
where
K: Serialize + ?Sized + Debug,
B: Write + AsRef<[u8]>,
{
let key = ser::serialize(buf, key).expect("failed to serialize query key");
self.get_blocking(key)
}
}
+4
View File
@@ -145,3 +145,7 @@ workspace = true
[package.metadata.cargo-shear]
ignored = ["reqwest_0_12"]
[dev-dependencies]
opentelemetry.workspace = true
rustls = { workspace = true, features = ["ring"] }
@@ -10,7 +10,6 @@ use conduwuit::error;
use conduwuit::{
Err, Event, PduEvent, Result, debug, debug_info, debug_warn, err,
result::{FlatOk, LogErr},
state_res::lexicographical_topological_sort,
trace,
utils::{IterStream, math::Expected, stream::BroadbandExt},
warn,
@@ -23,7 +22,9 @@ use ruma::{
int,
room_version_rules::RoomVersionRules,
};
use ruma::events::room::power_levels::UserPowerLevel;
use ruma::state_res::utils::event_id_map::EventIdMap;
use conduwuit::debug::DebugInspect;
use super::get_room_version_rules;
use crate::rooms::event_handler::parse_incoming_pdu::expect_event_id_array;
@@ -64,8 +65,7 @@ pub async fn build_local_dag(
return Ok(pdu_map.keys().cloned().collect());
}
let mut dag: HashMap<OwnedEventId, HashSet<OwnedEventId>> =
HashMap::with_capacity(pdu_map.len());
let mut dag = EventIdMap::with_capacity(pdu_map.len());
let mut id_origin_ts: HashMap<OwnedEventId, _> = HashMap::with_capacity(pdu_map.len());
for (event_id, value) in pdu_map {
@@ -94,17 +94,16 @@ pub async fn build_local_dag(
}
debug!(count = dag.len(), "Sorting incoming events with partial graph");
lexicographical_topological_sort(&dag, &async |node_id| {
ruma::state_res::reverse_topological_power_sort(&dag, |node_id| {
// Note: we don't bother fetching power levels because that would
// massively slow this function down. This is a best-effort attempt to
// order events correctly for processing, however ultimately that
// should be the sender's job.
let ts = id_origin_ts.get(&node_id).copied().unwrap_or_default();
Ok((int!(0), MilliSecondsSinceUnixEpoch(ts)))
let ts = id_origin_ts.get(node_id).copied().unwrap_or_default();
Ok((UserPowerLevel::Infinite, MilliSecondsSinceUnixEpoch(ts)))
})
.await
.inspect(|sorted| {
debug_assert_eq!(
.debug_inspect(|sorted| {
assert_eq!(
sorted.len(),
pdu_map.len(),
"Sorted graph was not the same size as the input graph"
+5 -10
View File
@@ -1,6 +1,8 @@
use std::collections::{HashMap, hash_map};
use conduwuit::{Err, Event, EventTypeExt, PduEvent, Result, err, warn};
use conduwuit::{
Err, Event, EventTypeExt, PduEvent, Result, debug_info, err, warn,
};
use ruma::{
OwnedEventId, ServerName, api::federation::authorization::get_event_authorization,
room_version_rules::RoomVersionRules,
@@ -191,15 +193,8 @@ impl super::Service {
},
}
}
if !self
.auth_state_check_4(
&pdu,
room_version_rules,
create_event.as_pdu(),
&auth_events_by_key,
)
.await?
{
if let Some(msg) = self.auth_state_check_4(&pdu, room_version_rules) {
debug_info!("Rejecting auth chain event {event_id}: {msg}");
self.reject_and_persist(&event_id, &pdu.to_canonical_object());
}
}
@@ -195,18 +195,10 @@ impl super::Service {
}
}
if !self
.auth_state_check_4(
&pdu_event,
&room_version_rules,
create_event.as_pdu(),
&auth_events_by_key,
)
.await?
{
if let Some(msg) = self.auth_state_check_4(&pdu_event, &room_version_rules) {
self.reject_and_persist(event_id, &incoming_pdu);
return Err!(Request(Forbidden(debug_warn!(
"Event authorisation fails based on event's claimed auth events"
"Event authorisation fails based on event's claimed auth events: {msg}"
))));
}
+7 -3
View File
@@ -17,10 +17,14 @@ use std::{collections::HashMap, fmt::Write, sync::Arc, time::Instant};
use assign::assign;
use async_trait::async_trait;
use conduwuit::{Err, Error, Event, PduEvent, Result, Server, SyncRwLock, utils::MutexMap};
use conduwuit::{
Err, Error, Event, PduEvent, Result, Server, SyncRwLock,
utils::{MutexMap, TryFutureExtExt},
};
pub use fetch_and_handle_outliers::{
DagBuilderTree, GET_MISSING_EVENTS_MAX_BATCH_SIZE, build_local_dag,
};
use futures::future::ok;
use http::StatusCode;
use ruma::{
EventId, OwnedEventId, OwnedRoomId, OwnedServerName,
@@ -131,8 +135,8 @@ impl Service {
}
/// Fetches a single PDU, returning None if there is an error.
async fn event_fetch(&self, event_id: OwnedEventId) -> Option<PduEvent> {
self.services.timeline.get_pdu(&event_id).await.ok()
fn event_fetch(&self, event_id: &EventId) -> Option<PduEvent> {
self.services.timeline.get_pdu_blocking(event_id).ok()
}
/// Returns a rate-limit error if the requested event had a recent failed
+38 -45
View File
@@ -3,7 +3,7 @@ use std::collections::HashMap;
use base64::Engine;
use conduwuit::{
Err, Event, EventTypeExt, PduEvent, Result, debug, debug::DebugInspect, debug_error,
debug_info, err, info, matrix::StateKey, state_res, trace,
debug_info, err, info, matrix::StateKey, trace,
};
use futures::future::ready;
use ruma::{
@@ -12,6 +12,8 @@ use ruma::{
canonical_json::redact,
events::StateEventType,
room_version_rules::{EventIdFormatVersion, RoomVersionRules},
state_res,
state_res::check_state_independent_auth_rules,
};
use crate::rooms::{
@@ -144,24 +146,21 @@ impl super::Service {
/// Checks PDU check 4: Passes authorisation rules based on the event's auth
/// events ([spec]).
///
/// If the auth check fails, false is returned, otherwise true.
/// If the auth check fails, the error message is returned, otherwise
/// `None`.
///
/// [spec]: https://spec.matrix.org/v1.19/server-server-api/#checks-performed-on-receipt-of-a-pdu
pub async fn auth_state_check_4(
pub fn auth_state_check_4(
&self,
pdu: &PduEvent,
room_version_rules: &RoomVersionRules,
create_event: &PduEvent,
auth_events_by_key: &HashMap<(StateEventType, StateKey), PduEvent>,
) -> Result<bool> {
let state_fetch = |ty: &StateEventType, sk: &str| {
let key = (ty.to_owned(), sk.into());
ready(auth_events_by_key.get(&key).map(ToOwned::to_owned))
};
state_res::event_auth::auth_check(room_version_rules, pdu, state_fetch, create_event)
.await
.map_err(|e| err!("Event self-authentication failed: {e:?}"))
) -> Option<String> {
match check_state_independent_auth_rules(&room_version_rules.authorization, pdu, |e| {
self.event_fetch(e)
}) {
| Ok(_) => None,
| Err(msg) => Some(msg),
}
}
/// Checks that the event passes PDU check 5, which ensures that the event
@@ -177,7 +176,7 @@ impl super::Service {
room_version_rules: &RoomVersionRules,
create_event: &PduEvent,
origin: &ServerName,
) -> Result<(bool, HashMap<u64, OwnedEventId>)> {
) -> Result<(Option<String>, HashMap<u64, OwnedEventId>)> {
debug!(
event_id = %incoming_pdu.event_id,
"Resolving state at event"
@@ -218,29 +217,27 @@ impl super::Service {
}
trace!(state_events = state_before.len(), "Calculated incoming state");
let state_fetch_state = &state_before;
let state_fetch = |k: StateEventType, s: StateKey| async move {
let shortstatekey = self.services.short.get_shortstatekey(&k, &s).await.ok()?;
let event_id = state_fetch_state.get(&shortstatekey)?;
self.services.timeline.get_pdu(event_id).await.ok()
};
debug!(
event_id = %incoming_pdu.event_id,
"Running state-before auth check"
);
// PDU check: 5
let auth_check = state_res::event_auth::auth_check(
room_version_rules,
let state_fetch_state = &state_before;
let auth_result = state_res::check_state_dependent_auth_rules(
&room_version_rules.authorization,
incoming_pdu,
|ty, sk| state_fetch(ty.clone(), sk.into()),
create_event.as_pdu(),
)
.await
.map_err(|e| err!(Request(Forbidden("Auth check failed: {e:?}"))))?;
Ok((auth_check, state_before))
|event_type, state_key| {
let ssk = self
.services
.short
.get_shortstatekey_blocking(event_type, state_key)
.ok()?;
let event_id = state_fetch_state.get(&ssk)?;
self.services.timeline.get_pdu_blocking(event_id).ok()
},
);
Ok((auth_result.err(), state_before))
}
/// Checks that the event passes PDU check 6, which ensures that the event
@@ -253,8 +250,7 @@ impl super::Service {
&self,
incoming_pdu: &PduEvent,
room_version_rules: &RoomVersionRules,
create_event: &PduEvent,
) -> Result<bool> {
) -> Result<(), String> {
debug!(
event_id = %incoming_pdu.event_id,
"Gathering auth events"
@@ -270,25 +266,22 @@ impl super::Service {
incoming_pdu.content(),
room_version_rules,
)
.await?;
let state_fetch = |k: &StateEventType, s: &str| {
let key = k.with_state_key(s);
ready(auth_events.get(&key).map(ToOwned::to_owned))
};
.await
.map_err(|e| e.to_string())?;
debug!(
event_id = %incoming_pdu.event_id,
"Running current state auth check"
);
state_res::event_auth::auth_check(
room_version_rules,
state_res::check_state_dependent_auth_rules(
&room_version_rules.authorization,
incoming_pdu,
state_fetch,
create_event.as_pdu(),
|t, s| {
auth_events
.get(&(t.to_owned(), s.to_owned()))
.map(ToOwned::to_owned)
},
)
.await
.map_err(|e| err!(Request(Forbidden("Auth check failed: {e:?}"))))
}
/// Performs PDU check 7 - does the policy server allow this event.
@@ -5,10 +5,8 @@
use std::{collections::BTreeMap, time::Duration};
use conduwuit::{
Err, Error, Event, PduEvent, Result, debug, debug_error, debug_info, debug_warn, err, error,
info, state_res::EventTypeExt, trace, utils::to_canonical_object, warn,
};
use conduwuit::{debug, debug_error, debug_info, debug_warn, err, error,
info, trace, utils::to_canonical_object, warn, Err, Error, Event, EventTypeExt, PduEvent, Result};
use http::StatusCode;
use ruma::{
CanonicalJsonObject, CanonicalJsonValue, KeyId, RoomId, ServerName, SigningKeyAlgorithm,
@@ -1,21 +1,28 @@
use std::{
borrow::Borrow,
collections::{HashMap, HashSet},
collections::{HashMap, HashSet, VecDeque},
sync::Arc,
};
use conduwuit::{
Error, Result, err,
state_res::{self, StateMap},
trace,
utils::stream::{IterStream, ReadyExt, TryWidebandExt, WidebandExt},
warn,
};
use futures::{FutureExt, StreamExt, TryFutureExt, TryStreamExt, future::try_join};
use ruma::{OwnedEventId, RoomId, room_version_rules::RoomVersionRules};
use itertools::cloned;
use ruma::{
EventId, OwnedEventId, RoomId,
room_version_rules::RoomVersionRules,
state_res::{Event, utils::event_id_set::EventIdSet},
};
use crate::rooms::state_compressor::CompressedState;
impl super::Service {
/// Resolves the state based on the incoming fork states with the given room
/// version rules.
pub async fn resolve_state(
&self,
room_id: &RoomId,
@@ -48,7 +55,7 @@ impl super::Service {
.event_ids_iter(room_id, state.values().map(Borrow::borrow))
.try_collect()
})
.try_collect::<Vec<HashSet<OwnedEventId>>>();
.try_collect::<Vec<EventIdSet<OwnedEventId>>>();
let fork_states = fork_states
.iter()
@@ -60,19 +67,19 @@ impl super::Service {
.short
.multi_get_statekey_from_short(shortstatekeys)
.zip(event_ids)
.ready_filter_map(|(ty_sk, id)| Some((ty_sk.ok()?, id)))
.ready_filter_map(|(ty_sk, id)| {
Some((ty_sk.ok().map(|(k, s)| (k, s.to_string()))?, id))
})
.collect()
})
.map(Ok::<_, Error>)
.try_collect::<Vec<StateMap<OwnedEventId>>>();
.try_collect::<Vec<ruma::state_res::StateMap<OwnedEventId>>>();
let (fork_states, auth_chain_sets) = try_join(fork_states, auth_chain_sets).await?;
trace!("Resolving state");
let state = self
.state_resolution(room_version_rules, fork_states.iter(), &auth_chain_sets)
.boxed()
.await?;
let state =
self.state_resolution(room_version_rules, fork_states.as_slice(), auth_chain_sets)?;
trace!("State resolution done.");
let state_events: Vec<_> = state
@@ -98,25 +105,72 @@ impl super::Service {
Ok(Arc::new(new_room_state))
}
pub async fn state_resolution<'a, StateSets>(
/// Actually performs state resolution from the given state sets, auth chain
/// sets, and room version rules.
pub fn state_resolution<'a>(
&'a self,
room_version_rules: &'a RoomVersionRules,
state_sets: StateSets,
auth_chain_sets: &'a [HashSet<OwnedEventId>],
) -> Result<StateMap<OwnedEventId>>
where
StateSets: Iterator<Item = &'a StateMap<OwnedEventId>> + Clone + Send,
{
state_sets: &[ruma::state_res::StateMap<OwnedEventId>],
auth_chain_sets: Vec<EventIdSet<OwnedEventId>>,
) -> Result<ruma::state_res::StateMap<OwnedEventId>> {
let event_fetch = |event_id| self.event_fetch(event_id);
let event_exists = |event_id| self.event_exists(event_id);
state_res::resolve(
room_version_rules,
let calc_subgraph = |ss| self.calculate_conflicted_state_subgraph(ss).ok();
ruma::state_res::resolve(
&room_version_rules.authorization,
&room_version_rules
.state_res
.v2_rules()
.expect("can only use state resolution v2 rules"),
state_sets,
auth_chain_sets,
&event_fetch,
&event_exists,
event_fetch,
calc_subgraph,
)
.map_err(|e| err!(error!("State resolution failed: {e:?}")))
.await
.map_err(|e| err!(error!("state resolution failed: {e:?}")))
}
/// Calculates the conflicted state subgraph for state resolution.
fn calculate_conflicted_state_subgraph<'a>(
&self,
state_set: &ruma::state_res::StateMap<&'a EventId>,
) -> Result<HashSet<&'a EventId>> {
let conflicted_events: HashSet<_> = state_set.values().cloned().collect();
let mut subgraph: HashSet<_> = HashSet::new();
let mut stack: Vec<Vec<_>> = vec![conflicted_events.iter().cloned().collect::<Vec<_>>()];
let mut path: Vec<_> = Vec::new();
let mut seen: HashSet<_> = HashSet::new();
let next_event = |stack: &mut Vec<Vec<_>>, path: &mut Vec<_>| {
while stack.last().is_some_and(Vec::is_empty) {
stack.pop();
path.pop();
}
stack.last_mut().and_then(Vec::pop)
};
while let Some(event_id) = next_event(&mut stack, &mut path) {
path.push(event_id.clone());
if subgraph.contains(&event_id) {
if path.len() > 1 {
subgraph.extend(path.iter().cloned());
}
path.pop();
continue;
}
if conflicted_events.contains(&event_id) && path.len() > 1 {
subgraph.extend(path.iter().cloned());
}
if seen.contains(&event_id) {
path.pop();
continue;
}
trace!(event_id = event_id.as_str(), "fetching event for its auth events");
let Some(evt) = self.event_fetch(event_id.clone()) else {
warn!("could not fetch event {} to calculate conflicted subgraph", event_id);
path.pop();
continue;
};
stack.push(evt.auth_events().collect());
seen.insert(event_id);
}
Ok(subgraph)
}
}
@@ -6,15 +6,18 @@ use std::{
use conduwuit::{
Result, debug, debug_error, err, error,
matrix::{Event, StateMap},
trace,
utils::stream::{BroadbandExt, IterStream, ReadyExt, TryBroadbandExt, TryWidebandExt},
};
use futures::{FutureExt, StreamExt, TryFutureExt, TryStreamExt, future::try_join};
use ruma::{OwnedEventId, RoomId, room_version_rules::RoomVersionRules};
use ruma::{room_version_rules::RoomVersionRules, EventId, OwnedEventId, RoomId};
use ruma::state_res::{Event, StateMap};
use ruma::state_res::utils::event_id_set::EventIdSet;
use crate::rooms::short::ShortStateHash;
#[cfg(test)]
mod tests;
impl super::Service {
/// Resolves the state before the incoming event.
///
@@ -27,7 +30,7 @@ impl super::Service {
room_version_rules: &RoomVersionRules,
) -> Result<Option<HashMap<u64, OwnedEventId>>>
where
Pdu: Event + Send + Sync,
Pdu: Event<Id = EventId> + Send + Sync,
{
if incoming_pdu.prev_events().count() == 1 {
self.state_before_incoming_degree_one(incoming_pdu).await
@@ -45,7 +48,7 @@ impl super::Service {
incoming_pdu: &Pdu,
) -> Result<Option<HashMap<u64, OwnedEventId>>>
where
Pdu: Event + Send + Sync,
Pdu: Event<Id = EventId> + Send + Sync,
{
let prev_event = incoming_pdu
.prev_events()
@@ -106,8 +109,11 @@ impl super::Service {
room_version_rules: &RoomVersionRules,
) -> Result<Option<HashMap<u64, OwnedEventId>>>
where
Pdu: Event + Send + Sync,
Pdu: Event<Id = EventId> + Send + Sync,
{
if incoming_pdu.prev_events().count() == 0 {
return Ok(None);
}
trace!("Calculating extremity statehashes...");
let Ok(extremity_sstatehashes) = incoming_pdu
.prev_events()
@@ -124,7 +130,7 @@ impl super::Service {
.pdu_shortstatehash(prev_eventid)
.map_ok(move |sstatehash| (sstatehash, prev_event))
})
.try_collect::<HashMap<_, _>>()
.try_collect::<Vec<_>>()
.inspect_err(|e| debug_error!("failed to calculate N-degree short state hashes: {e}"))
.await
else {
@@ -132,8 +138,8 @@ impl super::Service {
};
trace!("Calculating fork states...");
let room_id = &incoming_pdu.room_id_or_hash();
let (fork_states, auth_chain_sets): (Vec<StateMap<_>>, Vec<HashSet<_>>) =
let room_id = incoming_pdu.room_id().unwrap();
let (fork_states, auth_chain_sets): (Vec<StateMap<_>>, Vec<EventIdSet<_>>) =
extremity_sstatehashes
.into_iter()
.try_stream()
@@ -146,9 +152,7 @@ impl super::Service {
.await?;
let Ok(new_state) = self
.state_resolution(room_version_rules, fork_states.iter(), &auth_chain_sets)
.boxed()
.await
.state_resolution(room_version_rules, fork_states.as_slice(), auth_chain_sets)
.inspect_err(|e| error!("State resolution failed: {e:?}"))
else {
return Ok(None);
@@ -179,7 +183,7 @@ impl super::Service {
prev_event: Pdu,
) -> Result<(StateMap<OwnedEventId>, HashSet<OwnedEventId>)>
where
Pdu: Event,
Pdu: Event<Id = OwnedEventId>,
{
let mut leaf_state: HashMap<_, _> = self
.services
@@ -192,7 +196,7 @@ impl super::Service {
let shortstatekey = self
.services
.short
.get_or_create_shortstatekey(&prev_event.kind().to_string().into(), state_key)
.get_or_create_shortstatekey(&prev_event.event_type().to_string().into(), state_key)
.await;
let event_id = prev_event.event_id();
@@ -213,11 +217,10 @@ impl super::Service {
self.services
.short
.get_statekey_from_short(*k)
.map_ok(|(ty, sk)| ((ty, sk), id.clone()))
.map_ok(|(ty, sk)| ((ty, sk.to_string()), id.clone()))
})
.ready_filter_map(Result::ok)
.collect()
.map(Ok);
.collect();
try_join(fork_state, auth_chain).await
}
@@ -0,0 +1,275 @@
use std::sync::Arc;
use conduwuit::{
Config, PduEvent, Server,
log::{Log, LogLevelReloadHandles},
metrics::Metrics,
};
use futures::StreamExt;
use ruma::{RoomId, RoomVersionId, events::StateEventType};
use serde_json::json;
#[tokio::test]
async fn admitted_joins_do_not_disappear_after_a_merge() {
use conduwuit::matrix::PartialPdu;
use ruma::{
UserId,
events::room::{
join_rules::{JoinRule, RoomJoinRulesEventContent},
member::{MembershipState, RoomMemberEventContent},
},
};
let (services, path) = test_services().await;
let alice = UserId::parse("@alice:test.example").unwrap();
let bob = UserId::parse("@bob:bob.example").unwrap();
let charlie = UserId::parse("@charlie:charlie.example").unwrap();
let dave = UserId::parse("@dave:dave.example").unwrap();
let mut remote_keys = Vec::new();
for user in [&bob, &charlie, &dave] {
let key = ruma::signatures::Ed25519KeyPair::from_der(
&ruma::signatures::Ed25519KeyPair::generate(),
"test".into(),
)
.unwrap();
cache_key(&services, user.server_name(), &key);
remote_keys.push(key);
}
let bootstrap = services
.rooms
.state
.mutex
.lock("!bootstrap:test.example")
.await;
let create_id = services
.rooms
.timeline
.build_and_append_pdu(
PartialPdu {
event_type: "m.room.create".into(),
content: serde_json::value::to_raw_value(&json!({ "room_version": "12" }))
.unwrap(),
state_key: Some("".into()),
..Default::default()
},
&alice,
None,
&bootstrap,
)
.await
.unwrap();
drop(bootstrap);
let room = RoomId::parse(create_id.as_str().replacen('$', "!", 1)).unwrap();
let lock = services.rooms.state.mutex.lock(&room).await;
services
.rooms
.timeline
.build_and_append_pdu(
PartialPdu::state(
alice.to_string(),
&RoomMemberEventContent::new(MembershipState::Join),
),
&alice,
Some(&room),
&lock,
)
.await
.unwrap();
let rules_id = services
.rooms
.timeline
.build_and_append_pdu(
PartialPdu::state("", &RoomJoinRulesEventContent::new(JoinRule::Public)),
&alice,
Some(&room),
&lock,
)
.await
.unwrap();
drop(lock);
let rules = services.rooms.timeline.get_pdu(&rules_id).await.unwrap();
let (dave_join, json) = signed_remote_event(
json!({
"room_id": room, "sender": dave, "type": "m.room.member", "state_key": dave,
"content": { "membership": "join" }, "auth_events": [rules_id], "prev_events": [rules_id],
"depth": rules.depth + ruma::uint!(1), "origin_server_ts": conduwuit::utils::millis_since_unix_epoch(),
}),
dave.server_name(),
&remote_keys[2],
);
assert!(
services
.rooms
.event_handler
.handle_incoming_pdu(dave.server_name(), &room, &dave_join.event_id, json, false)
.await
.unwrap()
.is_some()
);
let mut joins = Vec::new();
for (user, key) in [&bob, &charlie].into_iter().zip(&remote_keys) {
let signed = signed_remote_event(
json!({
"room_id": room, "sender": user, "type": "m.room.member", "state_key": user,
"content": { "membership": "join" }, "auth_events": [rules_id], "prev_events": [dave_join.event_id],
"depth": dave_join.depth + ruma::uint!(1), "origin_server_ts": conduwuit::utils::millis_since_unix_epoch(),
}),
user.server_name(),
key,
);
joins.push(signed);
}
for ((pdu, json), user) in joins.iter().zip([&bob, &charlie]) {
let accepted = services
.rooms
.event_handler
.handle_incoming_pdu(user.server_name(), &room, &pdu.event_id, json.clone(), false)
.await
.unwrap();
assert!(accepted.is_some());
}
let bob_hash = services
.rooms
.state_accessor
.pdu_shortstatehash(&joins[0].0.event_id)
.await
.unwrap();
let charlie_hash = services
.rooms
.state_accessor
.pdu_shortstatehash(&joins[1].0.event_id)
.await
.unwrap();
assert_eq!(bob_hash, charlie_hash);
let (merge, json) = signed_remote_event(
json!({
"room_id": room, "sender": dave, "type": "m.room.message",
"content": { "msgtype": "m.text", "body": "merge" },
"auth_events": [dave_join.event_id],
"prev_events": [joins[0].0.event_id, joins[1].0.event_id],
"depth": joins[0].0.depth + ruma::uint!(1),
"origin_server_ts": conduwuit::utils::millis_since_unix_epoch(),
}),
dave.server_name(),
&remote_keys[2],
);
assert!(
services
.rooms
.event_handler
.handle_incoming_pdu(dave.server_name(), &room, &merge.event_id, json, false)
.await
.unwrap()
.is_some()
);
let mut results = Vec::new();
for ((user, key), (join, _)) in [&bob, &charlie].into_iter().zip(&remote_keys).zip(&joins) {
let membership = services
.rooms
.state_accessor
.room_state_get_content::<RoomMemberEventContent>(
&room,
&StateEventType::RoomMember,
user.as_str(),
)
.await
.unwrap();
assert_eq!(membership.membership, MembershipState::Join);
let prev_events = services
.rooms
.state
.get_forward_extremities(&room)
.collect::<Vec<_>>()
.await;
let depth = services
.rooms
.timeline
.get_pdu(&prev_events[0])
.await
.unwrap()
.depth
+ ruma::uint!(1);
let (message, json) = signed_remote_event(
json!({
"room_id": room, "sender": user, "type": "m.room.message",
"content": { "msgtype": "m.text", "body": format!("message from {user}") },
"auth_events": [join.event_id], "prev_events": prev_events,
"depth": depth, "origin_server_ts": conduwuit::utils::millis_since_unix_epoch(),
}),
user.server_name(),
key,
);
let result = services
.rooms
.event_handler
.handle_incoming_pdu(user.server_name(), &room, &message.event_id, json, false)
.await;
results.push(result.is_ok_and(|pdu| pdu.is_some()));
}
services.stop().await;
drop(services);
std::fs::remove_dir_all(path).unwrap();
assert_eq!(results, [true, true], "both joined users must be able to send messages");
}
async fn test_services() -> (Arc<crate::Services>, std::path::PathBuf) {
let _ = rustls::crypto::ring::default_provider().install_default();
let path = std::env::temp_dir()
.join(format!("continuwuity-shared-state-{:016x}", rand::random::<u64>()));
std::fs::create_dir(&path).unwrap();
let config: Config = serde_json::from_value(json!({
"server_name": "test.example",
"database_path": path,
"db_cache_capacity_mb": 16.0,
"rocksdb_write_buffer_capacity_mb": 4.0,
"allow_local_presence": false
}))
.unwrap();
let runtime = tokio::runtime::Handle::current();
let server = Arc::new(Server::new(
config,
Some(runtime.clone()),
Log {
reload: LogLevelReloadHandles::default(),
capture: Arc::default(),
},
Metrics::new(Some(runtime), opentelemetry::global::meter("test"), None),
));
let services = crate::Services::build(server).await.unwrap();
(services, path)
}
fn cache_key(
services: &crate::Services,
server: &ruma::ServerName,
key: &ruma::signatures::Ed25519KeyPair,
) {
let mut response = ruma::api::federation::discovery::ServerSigningKeys::new(
server.to_owned(),
crate::server_keys::in_one_week(),
);
let id: ruma::OwnedServerSigningKeyId =
format!("ed25519:{}", key.version()).try_into().unwrap();
response.verify_keys.insert(
id.clone(),
ruma::api::federation::discovery::VerifyKey::new(ruma::serde::Base64::new(
key.public_key().to_vec(),
)),
);
services.db["servernamekeyid_response"]
.put((server.as_str(), id.as_str()), database::Json(&response));
}
fn signed_remote_event(
value: serde_json::Value,
server: &ruma::ServerName,
key: &ruma::signatures::Ed25519KeyPair,
) -> (PduEvent, ruma::CanonicalJsonObject) {
let rules = RoomVersionId::V12.rules().unwrap();
let mut value = serde_json::from_value(value).unwrap();
ruma::signatures::hash_and_sign_event(server.as_str(), key, &mut value, &rules.redaction)
.unwrap();
let id = conduwuit::matrix::event::gen_event_id(&value, &rules).unwrap();
let pdu = PduEvent::from_id_val(&id, value.clone()).unwrap();
(pdu, value)
}
@@ -65,14 +65,14 @@ impl super::Service {
// perform PDU check 5 (event auth passes based on state before the
// event). To do this, we either need to have all the prev events
// locally, or ask a remote server for the state at the event.
let (passes_state_before, state_before) = self
let (state_before_failure_reason, state_before) = self
.state_before_check_5(&incoming_pdu, &room_version_rules, create_event, origin)
.await?;
if !passes_state_before {
if let Some(msg) = state_before_failure_reason {
self.reject_and_persist(incoming_pdu.event_id(), &val);
return Err!(Request(Forbidden(debug_warn!(
"Event authorisation fails based on the state before the event"
"Event authorisation fails based on the state before the event: {msg}"
))));
}
@@ -90,16 +90,16 @@ impl super::Service {
);
let state_lock = self.services.state.mutex.lock(room_id).await;
let passes_current_state = self
.current_state_check_6(&incoming_pdu, &room_version_rules, create_event)
.current_state_check_6(&incoming_pdu, &room_version_rules)
.await
.inspect(|passes| {
if !*passes {
debug_warn!(
"Event authorisation fails based on the current room state - will be \
soft-failed"
);
}
})?;
.inspect_err(|reason| {
debug_warn!(
%reason,
"Event authorisation fails based on the current room state - will be \
soft-failed"
);
})
.is_ok();
// Determine whether this PDU should be soft-failed.
// If the auth check failed, invariably yes. Otherwise, only if the user
+30 -17
View File
@@ -7,7 +7,7 @@ use conduwuit::{
event::{gen_event_id, gen_event_id_canonical_json},
},
pdu::PartialPdu,
state_res, trace,
trace,
utils::{self, IterStream, ReadyExt, to_canonical_object},
warn,
};
@@ -25,7 +25,7 @@ use ruma::{
StateEventType, StaticEventContent,
room::member::{MembershipState, RoomMemberEventContent},
},
room::{AllowRule, JoinRule, Restricted, RoomMembership},
room::{AllowRule, JoinRule, RoomMembership},
};
use crate::{
@@ -591,13 +591,17 @@ impl Service {
debug!("Running send_join auth check");
let fetch_state = &state;
let state_fetch = |k: StateEventType, s: StateKey| async move {
let shortstatekey = self.services.short.get_shortstatekey(&k, &s).await.ok()?;
let state_fetch = |k: StateEventType, s: StateKey| {
let shortstatekey = self
.services
.short
.get_shortstatekey_blocking(&k, &s)
.ok()?;
let event_id = fetch_state.get(&shortstatekey)?;
self.services.timeline.get_pdu(event_id).await.ok()
self.services.timeline.get_pdu_blocking(event_id).ok()
};
let Some(create_event) = state_fetch(StateEventType::RoomCreate, "".into()).await else {
if state_fetch(StateEventType::RoomCreate, "".into()).is_none() {
let in_auth_chain = send_join_auth_chain
.iter()
.any(|(t, s)| t == "m.room.create" && s.is_empty());
@@ -614,20 +618,29 @@ impl Service {
};
drop(send_join_auth_chain);
let auth_check = state_res::event_auth::auth_check(
&room_version.rules().unwrap(),
// TODO: These calls should be in a thread to avoid blocking the main
// event loop.
ruma::state_res::check_state_independent_auth_rules(
&room_version.rules().unwrap().authorization,
&parsed_join_pdu,
|event_id| self.services.timeline.get_pdu_blocking(event_id).ok(),
)
.map_err(|e| {
err!(Request(Forbidden(
"Join event returned by {remote_server} is not self-authorised (try again?): {e}"
)))
})?;
ruma::state_res::check_state_dependent_auth_rules(
&room_version.rules().unwrap().authorization,
&parsed_join_pdu,
|k, s| state_fetch(k.clone(), s.into()),
&create_event,
)
.await
.map_err(|e| err!(Request(Forbidden(warn!("Auth check failed: {e:?}")))))?;
.map_err(|e| {
err!(Request(Forbidden(
"Join event returned by {remote_server} is not self-authorised (try again?): {e}"
)))
})?;
if !auth_check {
return Err!(Request(Forbidden(
"Join event returned by {remote_server} is not self-authorised (try again?)"
)));
}
let resident_before = self
.services
.state_cache
@@ -845,7 +858,7 @@ impl Service {
&self,
room_id: &RoomId,
entity: JoinConditionEntity<'_>,
conditions: &Restricted,
conditions: &ruma::room::Restricted,
) -> JoinConditionResult {
if conditions.allow.is_empty() {
// This will never be authorisable, return forbidden.
+9
View File
@@ -154,6 +154,15 @@ impl Service {
.deserialized()
}
pub fn get_shortstatekey_blocking(
&self,
event_type: &StateEventType,
state_key: &str,
) -> Result<ShortStateKey> {
let key = (event_type, state_key);
self.db.shortstatekey_statekey.qry_blocking(&key).deserialized()
}
/// Gets a full event ID from a short event ID.
pub async fn get_eventid_from_short<Id>(&self, shorteventid: ShortEventId) -> Result<Id>
where
+5 -4
View File
@@ -5,7 +5,6 @@ use conduwuit::{debug, utils::stream::WidebandExt};
use conduwuit_core::{
Event, PduEvent, Result, err,
result::FlatOk,
state_res::{self, StateMap},
utils::{
IterStream, MutexMap, MutexMapGuard, ReadyExt, calculate_hash,
stream::{BroadbandExt, TryIgnore},
@@ -21,6 +20,7 @@ use ruma::{
api::federation::membership::RawStrippedState,
events::{StateEventType, TimelineEventType, room::create::RoomCreateEventContent},
room_version_rules::RoomVersionRules,
state_res,
};
use crate::{
@@ -472,7 +472,7 @@ impl Service {
state_key: Option<&str>,
content: &serde_json::value::RawValue,
room_version_rules: &RoomVersionRules,
) -> Result<StateMap<PduEvent>> {
) -> Result<HashMap<(StateEventType, String), PduEvent>> {
let Ok(shortstatehash) = self.get_room_shortstatehash(room_id).await else {
return Ok(HashMap::new());
};
@@ -482,8 +482,9 @@ impl Service {
sender,
state_key,
content,
room_version_rules,
)?;
&room_version_rules.authorization,
)
.map_err(|e| err!("failed to select auth types for event: {e:?}"))?;
debug!(?auth_types, "Auth types for event");
let sauthevents: HashMap<_, _> = auth_types
.iter()
+15 -34
View File
@@ -6,18 +6,11 @@ use conduwuit_core::{
matrix::{
event::{Event, gen_event_id},
pdu::{EventHash, PartialPdu, PduEvent},
state_res,
},
utils::{self, IterStream, ReadyExt, stream::TryIgnore},
};
use futures::{StreamExt, TryStreamExt, future, future::ready};
use ruma::{
CanonicalJsonObject, CanonicalJsonValue, OwnedEventId, OwnedRoomId, RoomId, RoomVersionId,
UserId,
events::{StateEventType, TimelineEventType, room::create::RoomCreateEventContent},
room_version_rules::RoomVersionRules,
uint,
};
use ruma::{events::{StateEventType, TimelineEventType, room::create::RoomCreateEventContent}, room_version_rules::RoomVersionRules, state_res, uint, CanonicalJsonObject, CanonicalJsonValue, OwnedEventId, OwnedRoomId, RoomId, RoomVersionId, UserId};
use serde_json::value::{RawValue, to_raw_value};
use super::RoomMutexGuard;
@@ -142,7 +135,7 @@ impl super::Service {
| None => Vec::new(),
};
let auth_events: HashMap<(StateEventType, SmallString<[u8; 48]>), PduEvent> =
let auth_events: HashMap<(StateEventType, String), PduEvent> =
match room_id {
| Some(room_id) =>
self.services
@@ -231,39 +224,27 @@ impl super::Service {
},
hashes: EventHash { sha256: String::new() },
signatures: None,
rejected: false,
};
let auth_fetch = |k: &StateEventType, s: &str| {
let key = (k.clone(), s.into());
ready(auth_events.get(&key).map(ToOwned::to_owned))
auth_events.get(&key).map(ToOwned::to_owned)
};
let room_id_or_hash = pdu.room_id_or_hash();
let create_pdu = match &pdu.kind {
| TimelineEventType::RoomCreate => None,
| _ => Some(
self.services
.state_accessor
.room_state_get(&room_id_or_hash, &StateEventType::RoomCreate, "")
.await
.map_err(|e| {
err!(Request(Forbidden(warn!("Failed to fetch room create event: {e}"))))
})?,
),
};
let create_event = match &pdu.kind {
| TimelineEventType::RoomCreate => &pdu,
| _ => create_pdu.as_ref().unwrap().as_pdu(),
};
let auth_check =
state_res::auth_check(&room_version_rules, &pdu, auth_fetch, create_event)
.await
.map_err(|e| err!(Request(Forbidden(warn!("Auth check failed: {e:?}")))))?;
if !auth_check {
return Err!(Request(Forbidden("Event is not authorized.")));
}
state_res::check_state_independent_auth_rules(
&room_version_rules.authorization,
&pdu,
|event_id| self.get_pdu_blocking(event_id).ok(),
)
.map_err(|e| err!(Request(Forbidden("{e}"))))?;
state_res::check_state_dependent_auth_rules(
&room_version_rules.authorization,
&pdu,
auth_fetch
).map_err(|e| err!(Request(Forbidden("{e}"))))?;
trace!(
"Event {} in room {} is authorized",
pdu.event_id,
+16
View File
@@ -161,6 +161,22 @@ impl Data {
select_ok([accepted, outlier]).await.map(at!(0))
}
pub(super) fn get_pdu_blocking(&self, event_id: &EventId) -> Result<PduEvent> {
if let Ok(outlier) = self
.eventid_outlierpdu
.get_blocking(event_id)?
.deserialized()
{
return Ok(outlier);
};
let pdu_id = self
.eventid_pduid
.get_blocking(event_id)
.map(|h| RawPduId::from(&*h))?;
self.pduid_pdu.get_blocking(&pdu_id)?.deserialized()
}
/// Like get_non_outlier_pdu(), but without the expense of fetching and
/// parsing the PduEvent
#[inline]
+6
View File
@@ -211,6 +211,12 @@ impl Service {
self.db.get_pdu(event_id).await
}
/// Returns the PDU. Checks the outlier tree BEFORE the timeline tree, as
/// this operation is synchronous.
pub fn get_pdu_blocking(&self, event_id: &EventId) -> Result<PduEvent> {
self.db.get_pdu_blocking(event_id)
}
/// Returns the pdu.
///
/// This does __NOT__ check the outliers `Tree`.