Commit Graph
100 Commits
Author SHA1 Message Date
Jason Volk 1d02851028 implement several broadband loops
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-04 21:50:20 +00:00
Jason Volk 59d5e3ebf1 additional stream extensions for any/all
additional stream extension TryBroadbandExt

Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-04 21:49:19 +00:00
Jason Volk c2d97aaa5e increase default db pool worker count for large systems
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-04 21:49:19 +00:00
Jason Volk 513236b3ce bump ruma for async state-res optimizations
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-04 00:51:57 +00:00
Jason Volk 784ccd6bad return stream from multi_get_eventid_from_short
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-04 00:51:57 +00:00
Jason Volk 48703173bc split get_batch from get.rs; add aqry_batch
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-04 00:51:57 +00:00
Jason Volk c01b049910 move cidr_range_denylist from globals to client service
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-04 00:51:57 +00:00
Jason Volk 9d9f403ad5 prevent adding presence timer for server's own user
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-04 00:51:57 +00:00
Jason Volk 3109c0daba perform async shutdown for database pool after services stop
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-04 00:51:57 +00:00
Jason Volk ef9b1c6303 simplify sender shutdown; prevent launching any retries
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-04 00:51:57 +00:00
Jason Volk b7df0a14c6 parallelize events_before and events_after in api/client/context
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-04 00:51:57 +00:00
Jason Volk b5006a4c41 offload initial iterator seeks to threadpool
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-03 13:25:33 +00:00
Jason Volk 320b0680bd pipeline various loops
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-03 13:25:33 +00:00
Jason Volk ed8c21ac9a modernize async srv lookup
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-03 13:25:33 +00:00
Jason Volk 9a9c071e82 use tokio for threadpool mgmt
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-03 07:39:02 +00:00
Jason Volk 89a158ab0b add delay before starting updates check
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-03 07:39:02 +00:00
Jason Volk 7d6710c033 add broadband stream extensions
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-03 07:39:02 +00:00
Jason Volk 61d9ac66fa add ref_at util macro
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-03 07:39:02 +00:00
Jason Volk 3b30bd3580 add try_filter_map to TryReadyExt
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-03 07:39:02 +00:00
Jason Volk 3fbd74310f impl transposed form of MapExpect
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-01 10:51:04 +00:00
Jason Volk 9263439af8 fix is_matching macro argument designator
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-01 10:51:04 +00:00
Jason Volk 4a3cc9fffa de-arc state_full_ids
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-30 08:38:12 +00:00
Jason Volk b5266ad9f5 parallelize sender edu selection
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-29 08:47:03 +00:00
Jason Volk 6175e72f1c simplify get_pdu() interface; eliminate unconditional Arc
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-29 08:47:02 +00:00
Jason Volk 58be22e695 fix new lints; clippy::unnecessary-map-or
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-29 06:58:45 +00:00
Jason Volk 2a9bb1ce11 add configurables for frontend pool options
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-28 07:20:43 +00:00
Jason Volk 3ad6aa59f9 use smallvec for db query buffering
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-28 06:03:33 +00:00
Jason Volk c7ae951676 add frontend threadpool to database
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-27 10:53:44 +00:00
Jason Volk 94d7b21cf0 use stricter timeout for fetching state
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-27 06:30:20 +00:00
Jason Volk 2aeee4f509 parallel query for outlier/non-outlier pdu data
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-27 06:30:20 +00:00
Jason Volk dd8c646b63 optimize state compressor I/O w/ batch operation
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-27 06:30:20 +00:00
Jason Volk 527494a34b fix oversized tracing span arguments; lints
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-27 06:30:20 +00:00
Jason Volk e83fa12451 tweak dev profile
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-27 02:57:13 +00:00
Jason Volk 4f97ff98d6 enter the tokio runtime for the scope of main init
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-27 02:57:13 +00:00
Jason Volk f69c596f56 generalize return value wrapping to not require Arc
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-26 03:45:21 +00:00
Jason Volk 238523f177 cleanup: reuse api rather than querying db
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-26 03:45:21 +00:00
Jason Volk 62d560e2fb improve tracing instruments on database::map
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-25 07:00:55 +00:00
Jason Volk 6c66391988 fix unnecessary serialization of sender query keys
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-25 06:50:15 +00:00
Jason Volk 2592f83b69 add migration fix for duplicate readreceipt entries
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-25 02:54:29 +00:00
Jason Volk c903a71807 refactor and optimize receipt service data
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-25 02:54:29 +00:00
Jason Volk 343ec59a8b use arrayvec for integer deserialization buffer
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-25 02:54:29 +00:00
Jason Volk 6f1d50dda3 panic on otherwise ignored errors in debug mode
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-25 02:54:29 +00:00
Jason Volk 1c751168c6 check-in missed example config changes
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-24 23:50:05 +00:00
Jason Volk a582d0559a bump url and cargo lock
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-24 22:16:16 +00:00
Jason Volk 4e74a1811b ci: set cancel-in-progress to true
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-24 22:16:16 +00:00
Jason Volk 97ad9afc86 default to main event for batch tokens
fix prev_batch token for legacy sync timeline

Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-24 21:47:25 +00:00
Jason Volk c519a40cb8 use multiget for shortid conversions
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-24 21:30:32 +00:00
Jason Volk 3789d60b6a refactor to iterator inputs for auth_chain/short batch functions
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-24 21:30:32 +00:00
Jason Volk 5da42fb859 refactor account_data.changes_since to stream
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-24 21:30:31 +00:00
Jason Volk fd4c447a2d move attribute argument extractor to utils
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-24 07:11:26 +00:00
Jason Volk f30b08f015 fix optional config section related
split api/client well_known

simplify well_known config access

Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-24 07:11:26 +00:00
Jason Volk 5f1cab6850 passthru worker thread count from env
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-24 05:18:36 +00:00
Jason Volk b94eeb9580 fix deletions on readreceipt update
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-22 09:38:34 +00:00
Jason Volk 3968d03868 move and improve common-rooms related
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-22 09:38:34 +00:00
Jason Volk aea82183b2 add set intersection util for two sorted streams
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-22 09:38:34 +00:00
Jason Volk bae0667066 limit sync response events to within the since/next_batch window
fixes #606

Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-22 09:02:01 +00:00
Jason Volk 2f2cebe84d implement local room preview
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-19 09:12:50 +00:00
Jason Volk e257512aa7 relax state visibility for invited modes
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-19 08:37:25 +00:00
Jason Volk 411c60009d enrich state iteration interface
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-19 08:37:25 +00:00
Jason Volk 7680d1bd5e replace yields point with consume_budget
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-19 08:37:25 +00:00
Jason Volk 8fedc358e0 typename additional shortids
cleanup/split state_compressor load

Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-19 08:37:25 +00:00
Jason Volk 90106c4c33 streamline batch insertions
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-19 08:37:25 +00:00
Jason Volk a05dc03100 use debug_warn for backfill event evals
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-19 08:37:25 +00:00
Jason Volk 26bcc7e312 fix default stateinfo cache size
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-17 00:15:13 +00:00
Jason Volk 887ae84f1e optimize sha256 interface gather/vector inputs
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-16 00:33:40 +00:00
Jason Volk 14e3b242df add database get_batch stream wrapper
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-16 00:33:40 +00:00
Jason Volk 9f7a4a012b improve tracing/logging for state_compressor
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-16 00:33:40 +00:00
Jason Volk 5f625216aa slight optimizations for statediff
calculate with_capacity for set/get_statediff() etc

Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-16 00:33:40 +00:00
Jason Volk 20836cc3db flush=false for database-backup in read-only/secondary modes; improve error
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-16 00:33:40 +00:00
Jason Volk 59834a4b05 add is_read_only()/is_secondary() to Engine
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-16 00:33:40 +00:00
Jason Volk 4ec5d1e28e replace additional use tracing::
add log:: to disallowed-macros

Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-14 04:50:07 +00:00
Jason Volk e228dec4f2 add byte counting for compressed state caches
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-13 23:14:05 +00:00
Jason Volk 6ffdc1b2a6 bump serde, image, loole, termimad etc
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-13 22:59:28 +00:00
Jason Volk 004be3bf00 prepare utf-8 check bypass for database deserializer
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-13 22:59:28 +00:00
Jason Volk 77fab2c323 use ruma visibility enum in directory interface
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-13 02:38:03 +00:00
Jason Volk 68582dd868 add parallel query for current membership state
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-13 02:38:03 +00:00
Jason Volk feefa43e65 add pretty/si-unit byte size parsing/printing utils
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-13 02:38:03 +00:00
Jason Volk 86694f2d1d move non-generic code out of generic; reduce codegen
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-13 02:38:03 +00:00
Jason Volk 999d731a65 move err macro visitor out-of-line; reduce codegen
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-11 22:45:44 +00:00
Jason Volk 3962333043 partially revert e507c31306
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-11 21:35:30 +00:00
Jason Volk 61174dd0d3 check if lazyset already contains user prior to querying
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-11 21:30:48 +00:00
Jason Volk e2afaa9f03 add config item for with_span_events
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-11 20:50:05 +00:00
Jason Volk 9790a6edc9 add unwrap_or_err to result
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-11 20:50:05 +00:00
Jason Volk 08a4e931a0 supplement a from_str for FmtSpan
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-11 20:50:05 +00:00
Jason Volk 1efc52c440 increase logging during server keys acquire
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-10 11:17:42 +00:00
Jason Volk f290d1a9c8 prevent retry for missing keys later in join process
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-10 08:47:15 +00:00
Jason Volk 5e74391c6c fix config generator macro matchers
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-10 04:33:30 +00:00
Jason Volk cc86feded3 bump ruma
fixes for key type changes

Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-10 04:33:30 +00:00
Jason Volk 14fce38403 cork around send_join response processing
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-10 04:33:30 +00:00
Jason Volk 10be301646 split large notary requests into batches
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-10 04:33:29 +00:00
Jason Volk 1ce3db727f split event_handler service
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-08 09:21:42 +00:00
Jason Volk 6eba36d788 split make_body template
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-08 09:21:42 +00:00
Jason Volk f59e8af734 slight cleanup/simplifications to backfil
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-08 09:21:30 +00:00
Jason Volk 1f2e939fd5 optional arguments for timeline pdus iterations
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-08 08:22:54 +00:00
Jason Volk 13ef6dcbcf add standalone getters for shortid service
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-08 06:09:04 +00:00
Jason Volk 27966221f1 add ready_try_fold to utils
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-08 06:09:04 +00:00
Jason Volk 79c6b51860 renames for core pdu
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-08 06:09:04 +00:00
Jason Volk e507c31306 make pdu batch tokens zeroith-indexed
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-08 06:06:18 +00:00
Jason Volk f36757027e split api/client/room
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-06 21:46:20 +00:00
Jason Volk 7450c654ae add get_pdu_owned sans Arc; improve client/room/event handler
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-06 21:46:20 +00:00