Commit Graph
100 Commits
Author SHA1 Message Date
Jason Volk 7c0c029a4a add try_lock to MutexMap; allow TryFrom constructions
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-22 06:40:07 +00:00
Jason Volk 49023aa295 use database for resolver caches
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-22 02:16:51 +00:00
Jason Volk 0c96891008 add CBOR support to database schema
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-22 00:52:48 +00:00
Jason Volk 1f31e74024 add del to raw suite
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-22 03:50:51 +00:00
Jason Volk 9ab381e4eb generate fmt::Display for Config
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-18 07:12:20 +00:00
Jason Volk dda27ffcb1 add some compaction related interfaces
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-18 12:05:07 +00:00
Jason Volk 8ab825b12c add stream parallelism extension combinators
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-21 20:43:38 +00:00
Jason Volk 19f6d9d0e1 add index-compression and auto-readahead to descriptor
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-20 12:12:44 +00:00
Jason Volk 277b4951e8 add compression-shaping; tweak default compression levels
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-20 11:50:17 +00:00
Jason Volk 610129d162 outdent auth_chain Service impl
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-20 09:05:49 +00:00
Jason Volk 4c0ae8c2f7 parallelize get_auth_chain outer
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-20 09:02:50 +00:00
Jason Volk ea25dc04b2 parallelize current and incoming fork-state fetch
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-20 07:38:32 +00:00
Jason Volk 388730d6dd add TryWideband trait to similar to TryBroadband
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-20 07:38:19 +00:00
Jason Volk ac944496c1 optimize statekey-from-short loopsite
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-14 19:17:45 +00:00
Jason Volk 3dae02b886 add preferred jemalloc config
add muzzy/dirty configuration mallctl interface

add program argument for --gc-muzzy=false

Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-18 01:30:41 +00:00
Jason Volk 3eed408b29 additional util macros and reorg
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-20 04:48:47 +00:00
Jason Volk 4fbbfe5d30 add alt argument format for detecting cargo build phase
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-20 04:48:47 +00:00
Jason Volk df3eb95d4f additional affinity utils
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-20 04:48:47 +00:00
Jason Volk 7045481fae add from_errno construction to Error
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-20 04:48:47 +00:00
Jason Volk c6ae6adc80 pre-allocate some amount of media read buffer
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-20 04:48:47 +00:00
Jason Volk afdf5a07b5 abstract hidden line related in config generator macro 2025-01-20 04:48:47 +00:00
Jason Volk f9e76d6239 improve debug memory-stats options
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-20 04:48:47 +00:00
Jason Volk 8141ca3444 refactor admin outputs to asyncwrite
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-20 04:48:47 +00:00
Jason Volk abf33013e3 check-in additional database test related
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-20 04:48:47 +00:00
Jason Volk 96e85adc32 use cache builder for row and table cache options
add cache check using multi-get path

Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-20 04:48:47 +00:00
Jason Volk fc1170e12a additional tracing span tweaks
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-20 04:17:04 +00:00
Jason Volk 819e35f81f remove mutex lock/unlock during sync iteration.
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-20 04:17:04 +00:00
Jason Volk bab40a3747 enable hashing on large-block indexes
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-20 04:17:04 +00:00
Jason Volk aad42bdaa0 reduce block size on small tables
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-20 04:17:04 +00:00
Jason Volk 3759d1be6c tweak per-column write_buffer down from default
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-20 04:17:04 +00:00
Jason Volk 77d8e26efe integrate trim-on-park memory reclamation to runtime
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-20 04:17:04 +00:00
Jason Volk 7a8ca8842a add jemallctl base; add trim to interface w/ console cmd
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-20 04:17:02 +00:00
Jason Volk 80832cb0bb add checked math wrapper
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-20 04:16:17 +00:00
Jason Volk 98d8e5c63c add standard error trait and thread access error conversions
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-20 04:16:17 +00:00
Jason Volk 5167e1f06d add option to disable listeners
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-18 01:43:58 +00:00
Jason Volk e56d3c6cb3 add multi_get_statekey_from_short
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-18 01:43:58 +00:00
Jason Volk afcd0bfeef add deref_at macro util
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-18 01:43:58 +00:00
Jason Volk 9ebb39ca4f add missing cfg_attr on deserialisation db test
Signed-off-by: strawberry <strawberry@puppygock.gay>
2025-01-15 14:34:50 -05:00
Jason Volk 66231676f1 gracefully ignore unknown columns; add dropped flag in descriptor
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-10 22:29:05 +00:00
Jason Volk 16fa2eca87 add conf item for write buffer size
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-10 07:03:34 +00:00
Jason Volk 6a0f9add0c refactor database engine/options; add column descriptors
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-10 07:03:34 +00:00
Jason Volk 02f19cf951 tweak tracing spans; inlines
db deserializer tracing instrument cover

Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-10 07:03:34 +00:00
Jason Volk 685b127f99 simplify iterator state constructor arguments
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-10 07:03:34 +00:00
Jason Volk cc1889d135 Add default-enabled feature-gates for url_preview and media_thumbnail
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-10 07:03:34 +00:00
Jason Volk 0238f27605 prevent example-config generating in test builds
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-10 07:03:34 +00:00
Jason Volk 5dae086197 exclude config item from doctest
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-10 06:53:15 +00:00
Jason Volk 44e6b1af3c fixes for tests to be run in release-mode
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-09 19:56:42 +00:00
Jason Volk 94c8683836 improve db pool topology configuration
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-09 18:14:23 +00:00
Jason Volk d36167ab64 partially revert 9a9c071e82; use std threads for db pool.
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-09 18:14:23 +00:00
Jason Volk 925061b92d flatten timeline pdus iterations; increase concurrency
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-09 18:14:23 +00:00
Jason Volk 27328cbc01 additional futures extension utils
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-09 18:14:23 +00:00
Jason Volk a3f9432da8 eliminate the state-res mutex hazard
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-09 18:14:23 +00:00
Jason Volk 82168b972a fix heroes calculation regression
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-09 18:14:23 +00:00
Jason Volk f1af7ec08c fix unused import in release mode
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-03 02:36:24 +00:00
Jason Volk 50cadbee96 add faster shutdown point in unix socket listener
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-02 08:20:02 +00:00
Jason Volk b1182fe8a4 improve some router tracing spans
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-02 08:20:02 +00:00
Jason Volk 77d7e8a3ad add json array to serde_json::Value db deserialization without Raw 2025-01-02 06:52:32 +00:00
Jason Volk b4ef646485 translate 5xx to 404 on remote media request
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk b8f8f68634 improve memory-usage output w/ more byte-sizes
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 33635e11d1 bump ruma
tweak tracing instrument

Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 1a71798859 add Expected trait to utils; use (already transitive) num-traits.
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 7e4453620e move exponential backoff util to different submod
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 2259e2c82f batch queries to maximize throughput
query-side streams for first level of callsites

Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 1792711d09 support core affinity mask
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 0b2fca5ad9 add complement diff from e2f6753a3280 2025-01-01 23:28:01 -05:00
Jason Volk 3ffe2cd56e reduce backoff warning from infolog
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk c5b94be5b8 remove dev_release_log_level cfg
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 2bbb28bb88 get device info with libc using major/minor
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 203cf57fdf add console log event format hook
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 9c6b5b4407 add faster interruption to resolver (#649)
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk a1fc4d49ac reduce non-debug startup warnings
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk b56e480b3c integrate some std io error kinds with our status code abstraction
associate rocksdb error kinds with std io error kinds

Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 7c8eeaf4ea simplify multi_get_or_create/related stream implementations
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 9eb99f8070 abstract async shutdown waiting loop
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 0a9b6c136f refactor for ruma identifiers optimizations
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk d91570d0e6 add Error variant for FeatureDisabled
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 3a2c5be4f4 reduce exponential backoff for prev fetch
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk e21403a4d4 use VecDeque for todo queues
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 74eb30c106 add LIFO scheduling for database frontend pool workers
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 6458f4b195 refactor various Arc<EventId> to OwnedEventId
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 5a335933b8 pass stream width to ruma state res
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 3b95af9a18 move stream width related into unit
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk a407909d09 reduce unnecessary info-level spans to debug
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 7a6d657558 configurable dynamic stream concurrency scalar
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk b195107053 optimize for multi-queue storage topologies with affinity
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk e5a1309583 additional interruption points to hasten shutdown
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 03f2ac9caf simplify usage of mpmc channels which don't require receiver lock
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 5fdb8895b1 add utils to detect device io characteristics
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 749f29aaab add and/and_then to BoolExt
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk a3d87be22f enable atomic_flush for database columns
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk aefc4b4e69 add more program options for tokio
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 503210c3bf toggle worker_affinity feature from program argument
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 674acc8657 add program options for tokio histogram settings
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk af3d6a2e37 shard sender into multiple task workers by destination hash
rename Destination::Normal variant

tracing instruments

Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 98e6c81e49 offload remaining db iterator initial seeks on cache miss
consume task budget on cache hit

Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 14341bb906 simplify and improve db iter State init interface
Signed-off-by: Jason Volk <jason@zemos.net>
2025-01-01 23:28:01 -05:00
Jason Volk 60a952508e use macro for error constructions
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-18 14:16:29 -05:00
Jason Volk 4d46df2af5 abort tasks for non-async pool shudown
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-18 14:16:23 -05:00
Jason Volk 6c42ac2726 add num_workers() convenience to core metrics
Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-18 13:32:13 -05:00
Jason Volk 71673b2a88 add worker_affinity feature
split runtime init from main.rs

Signed-off-by: Jason Volk <jason@zemos.net>
2024-12-18 13:31:58 -05:00