Commit Graph
100 Commits
Author SHA1 Message Date
Jason Volk 44e6b1af3c fixes for tests to be run in release-mode
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-09 19:56:42 +00:00
Jason Volk 94c8683836 improve db pool topology configuration
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-09 18:14:23 +00:00
Jason Volk d36167ab64 partially revert 9a9c071e82; use std threads for db pool.
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-09 18:14:23 +00:00
Jason Volk 925061b92d flatten timeline pdus iterations; increase concurrency
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-09 18:14:23 +00:00
Jason Volk 27328cbc01 additional futures extension utils
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-09 18:14:23 +00:00
Jason Volk a3f9432da8 eliminate the state-res mutex hazard
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-09 18:14:23 +00:00
Jason Volk 82168b972a fix heroes calculation regression
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-09 18:14:23 +00:00
Jason Volk f1af7ec08c fix unused import in release mode
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-03 02:36:24 +00:00
Jason Volk 50cadbee96 add faster shutdown point in unix socket listener
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-02 08:20:02 +00:00
Jason Volk b1182fe8a4 improve some router tracing spans
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-02 08:20:02 +00:00
Jason Volk 77d7e8a3ad add json array to serde_json::Value db deserialization without Raw 2025-01-02 06:52:32 +00:00
Jason Volk b4ef646485 translate 5xx to 404 on remote media request
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk b8f8f68634 improve memory-usage output w/ more byte-sizes
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 33635e11d1 bump ruma
tweak tracing instrument

Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 1a71798859 add Expected trait to utils; use (already transitive) num-traits.
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 7e4453620e move exponential backoff util to different submod
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 2259e2c82f batch queries to maximize throughput
query-side streams for first level of callsites

Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 1792711d09 support core affinity mask
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 0b2fca5ad9 add complement diff from e2f6753a3280 2025-01-01 23:28:01 -05:00
Jason Volk 3ffe2cd56e reduce backoff warning from infolog
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk c5b94be5b8 remove dev_release_log_level cfg
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 2bbb28bb88 get device info with libc using major/minor
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 203cf57fdf add console log event format hook
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 9c6b5b4407 add faster interruption to resolver (#649)
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk a1fc4d49ac reduce non-debug startup warnings
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk b56e480b3c integrate some std io error kinds with our status code abstraction
associate rocksdb error kinds with std io error kinds

Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 7c8eeaf4ea simplify multi_get_or_create/related stream implementations
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 9eb99f8070 abstract async shutdown waiting loop
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 0a9b6c136f refactor for ruma identifiers optimizations
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk d91570d0e6 add Error variant for FeatureDisabled
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 3a2c5be4f4 reduce exponential backoff for prev fetch
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk e21403a4d4 use VecDeque for todo queues
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 74eb30c106 add LIFO scheduling for database frontend pool workers
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 6458f4b195 refactor various Arc<EventId> to OwnedEventId
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 5a335933b8 pass stream width to ruma state res
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 3b95af9a18 move stream width related into unit
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk a407909d09 reduce unnecessary info-level spans to debug
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 7a6d657558 configurable dynamic stream concurrency scalar
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk b195107053 optimize for multi-queue storage topologies with affinity
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk e5a1309583 additional interruption points to hasten shutdown
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 03f2ac9caf simplify usage of mpmc channels which don't require receiver lock
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 5fdb8895b1 add utils to detect device io characteristics
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 749f29aaab add and/and_then to BoolExt
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk a3d87be22f enable atomic_flush for database columns
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk aefc4b4e69 add more program options for tokio
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 503210c3bf toggle worker_affinity feature from program argument
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 674acc8657 add program options for tokio histogram settings
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk af3d6a2e37 shard sender into multiple task workers by destination hash
rename Destination::Normal variant

tracing instruments

Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 98e6c81e49 offload remaining db iterator initial seeks on cache miss
consume task budget on cache hit

Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 14341bb906 simplify and improve db iter State init interface
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 60a952508e use macro for error constructions
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-18 14:16:29 -05:00
Jason Volk 4d46df2af5 abort tasks for non-async pool shudown
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-18 14:16:23 -05:00
Jason Volk 6c42ac2726 add num_workers() convenience to core metrics
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-18 13:32:13 -05:00
Jason Volk 71673b2a88 add worker_affinity feature
split runtime init from main.rs

Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-18 13:31:58 -05:00
Jason Volk 5f7636f177 set dbpool thread name (gated by tokio_unstable)
Signed-off-by: Jason Volk <jason@zemos.net>
Signed-off-by: strawberry <strawberry@puppygock.gay>
2024-12-18 13:31:41 -05:00
Jason Volk f78104a959 relax Send on input iterator
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-18 13:29:51 -05:00
Jason Volk ad8cbcaac1 check cache prior to offloading iterator seek
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-18 13:29:43 -05:00
Jason Volk 34f9e3260f additional sync v3 refactoring/optimizations and tracing instruments
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-10 06:20:30 +00:00
Jason Volk aba88ccead misc tracing span tweaks
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-10 06:20:30 +00:00
Jason Volk 5fc8e90e02 fix runtime metrics command
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-10 05:40:45 +00:00
Jason Volk cd67ca6c57 fix .gitignore to allow /target to be a symlink
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-10 05:40:45 +00:00
Jason Volk a752fb9943 rename mod with tracing inits due to macro issues
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-08 20:33:25 +00:00
Jason Volk f70fdca828 remove jemalloc_stats from main default features
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-08 20:33:25 +00:00
Jason Volk 82133ee2ea bump rust-rocksdb
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-08 07:12:27 +00:00
Jason Volk ff56ee7413 add back the consume_budget on cache hits
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-08 07:12:27 +00:00
Jason Volk 6aa35260e6 misc database options tweaks
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-08 07:12:27 +00:00
Jason Volk 139a1ac504 configure the tokio runtime tunables
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-08 07:12:27 +00:00
Jason Volk 65a370836c derive specific ReadOptions for iterators
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-08 07:12:27 +00:00
Jason Volk aa6d0fcaa7 add unwrap_or_default() to future TryExt extension
start an OptionFuture extension

Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-08 07:12:27 +00:00
Jason Volk d921b82376 add options for console tracing EnvFilter details
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-08 07:12:26 +00:00
Jason Volk da984d49cf remove unnecessary tracing of pdu conversions
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-08 07:12:26 +00:00
Jason Volk b7a41f283f optimize sync v3 asynchronicity
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-08 07:12:26 +00:00
Jason Volk b77a1eb079 move cork to remove_to_device_events fn
add cork around write-heavy database routine called in loop; cleanup

Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-08 07:12:23 +00:00
Jason Volk 23cf2b2236 add is_err() to TryFuture extension
add fold_default to Future tools extension

Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-08 07:12:23 +00:00
Jason Volk de3b137df8 eliminate future wrapping stream for all_pdus()
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-08 07:12:23 +00:00
Jason Volk 8e8c6bfe07 abstract supported room versions apis
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-08 07:12:23 +00:00
Jason Volk f0a1aaf7bc extend x-platform support for binding URL previews to interfaces via address
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-08 07:12:23 +00:00
Jason Volk 68afdb22c7 force Cargo.lock version to 3
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-04 21:55:50 +00:00
Jason Volk 1d02851028 implement several broadband loops
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-04 21:50:20 +00:00
Jason Volk 59d5e3ebf1 additional stream extensions for any/all
additional stream extension TryBroadbandExt

Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-04 21:49:19 +00:00
Jason Volk c2d97aaa5e increase default db pool worker count for large systems
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-04 21:49:19 +00:00
Jason Volk 513236b3ce bump ruma for async state-res optimizations
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-04 00:51:57 +00:00
Jason Volk 784ccd6bad return stream from multi_get_eventid_from_short
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-04 00:51:57 +00:00
Jason Volk 48703173bc split get_batch from get.rs; add aqry_batch
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-04 00:51:57 +00:00
Jason Volk c01b049910 move cidr_range_denylist from globals to client service
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-04 00:51:57 +00:00
Jason Volk 9d9f403ad5 prevent adding presence timer for server's own user
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-04 00:51:57 +00:00
Jason Volk 3109c0daba perform async shutdown for database pool after services stop
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-04 00:51:57 +00:00
Jason Volk ef9b1c6303 simplify sender shutdown; prevent launching any retries
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-04 00:51:57 +00:00
Jason Volk b7df0a14c6 parallelize events_before and events_after in api/client/context
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-04 00:51:57 +00:00
Jason Volk b5006a4c41 offload initial iterator seeks to threadpool
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-03 13:25:33 +00:00
Jason Volk 320b0680bd pipeline various loops
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-03 13:25:33 +00:00
Jason Volk ed8c21ac9a modernize async srv lookup
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-03 13:25:33 +00:00
Jason Volk 9a9c071e82 use tokio for threadpool mgmt
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-03 07:39:02 +00:00
Jason Volk 89a158ab0b add delay before starting updates check
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-03 07:39:02 +00:00
Jason Volk 7d6710c033 add broadband stream extensions
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-03 07:39:02 +00:00
Jason Volk 61d9ac66fa add ref_at util macro
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-03 07:39:02 +00:00
Jason Volk 3b30bd3580 add try_filter_map to TryReadyExt
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-03 07:39:02 +00:00
Jason Volk 3fbd74310f impl transposed form of MapExpect
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-01 10:51:04 +00:00
Jason Volk 9263439af8 fix is_matching macro argument designator
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-01 10:51:04 +00:00
Jason Volk 4a3cc9fffa de-arc state_full_ids
Signed-off-by: Jason Volk <jason@zemos.net>
2024-11-30 08:38:12 +00:00