Series Outline Link to heading
- Overview
- Framework analysis and selection: Delving into the specifics of framework selection and application.
- Core Components: Database management, modularization, internal RPC, AMQP.
- Web and API: Web Server, GraphQL API.
- Application Development: Web, CLI, desktop, and mobile app development.
- Miscellaneous Tools: Observability, logging, machine learning.
- Boilerplate project: A comprehensive guide including setup instructions for selected frameworks, architectural diagrams, and development environment configurations.
ML/stats Link to heading
| Crate | Downloads | Dependents | Github Stars | Github Contributors | Github Used By | Notes |
|---|---|---|---|---|---|---|
| polars | 500k | 88 | 21k | 302 | 4.3k | Dataframes for data prep/filtering/etc - this is easily the best choice. Can now do streaming, so, can handle > in memory as well. |
| linfa | 162k | 24 | 3k | 37 | 321 | decent set of algorithms. OpenBLAS, netlib, intel MKL (all linux) |
| rustlearn | 16k | 1 | 599 | 8 | 121 | Not nearly as robust as linfa, but does have some unique algorithms. |
Logging/Metrics Link to heading
| Crate | Downloads | Dependents | Github Stars | Github Contributors | Github Used By | Notes |
|---|---|---|---|---|---|---|
| log | 197M | 14770 | 1.9k | 102 | 605k | Default choice. syslog, systemd, slog, android, windows, database, console, and a lot more. |
| tracing | 109M | 4967 | 4.4k | 277 | 241k | Compatible with log, which is nice. metrics, tracing, and logging. wasm support. |
Distributed Locking using Redis Link to heading
Redis is assumed for caching - but for dist lock, we have lots of options…
| Crate | Downloads | Dependents | Github Stars | Github Contributors | Github Used By | Notes |
|---|---|---|---|---|---|---|
| redlock | 34k | 0 | 46 | 11 | ? | port of the ruby version. seems fine. |
| redsync | 9k | 0 | 1 | 2 | ? | Good, if we want a sync interface. |
| rust_redlock | 4.7k | 0 | 17 | ? | ? | also seems fine, but, not supported anymore. |
| rslock | 2.4k | 0 | 23 | 13 | ? | Good async interface. Decent support, but not popular? |
Summary Link to heading
ML Link to heading
Data prep - easy choice with polars. Seems like default choice for ML should be linfa, but, if that doesn’t support what we need to do, go look for specific algos.
Logging Link to heading
Log crate is fine, but tracing is much more flexible (3 for 1 deal - logging, tracing, and metrics), well supported by other libraries we’ve talked about (tower, hyper, tonic, axum, etc, etc), and has gained plenty of traction. Opentelemetry via tracing-opentelemetry is also recommended, which gets you all sorts of nice stuff.
Locking Link to heading
Seems to me we’ll want to wrap up our own helper(s) around this, none of the libraries I could find are “great” - so, we should pick one, and wrap it for the way(s) we want to use it. Perhaps we write a macro?