# Async Rust  ## Metadata - Author: Maxwell Flitton and Caroline Morton - Full Title: Async Rust - Category: #books ## Highlights - Asynchronous programming in Rust, often referred to as Async Rust, is a powerful paradigm that allows developers to write concurrent code that is more efficient and scalable. ([Location 27](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=27)) - concurrently, which is particularly useful when dealing with I/O-bound operations like network requests or file handling. ([Location 31](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=31)) - With async programming, we do not need to add another core to the CPU to get performance gains. Instead, with async, we can effectively juggle multiple tasks on a single thread if there is some dead time in those tasks, such as waiting for a response from a server. ([Location 169](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=169)) - Using async programming, we can free up CPU resources by not blocking the CPU with tasks that can wait. ([Location 330](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=330)) - Standard async programming in Rust does not use multiprocessing; however, we can achieve async behavior by using multiprocessing. For this to work, our async systems must sit within a process. ([Location 340](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=340)) - A process is an abstraction provided by an operating system that is executed by the CPU. Processes can be run by a program or application. ([Location 351](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=351)) - Processes differ from threads in that each process consists of its own memory space, and this is an essential part of how the CPU is managed because it prevents data from being corrupted or bleeding over into other processes. ([Location 359](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=359)) - A process has its own ID called a process ID (PID), which can be monitored and controlled by the computer’s operating system. ([Location 362](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=362)) - if two async tasks representing individual connections each rely on data from shared memory, we must introduce synchronization primitives such as locks. These synchronization primitives are at risk of adding complications such as deadlocks, which can end up grinding all connections that are relying on that lock to a halt. ([Location 376](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=376)) - While this reduces the number of moving parts, we still have the scaling issues of processes combined with the overhead of having to serialize data between the process and our main program. ([Location 610](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=610)) - Threads seem like a much better choice over processes for asynchronous programming due to the ease of sharing data between threads. ([Location 618](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=618)) - A thread of execution is the smallest sequence of programmed instructions that can be executed by the CPU. A thread can be independently managed by a scheduler. Inside a process, we can share memory across multiple threads ([Location 622](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=622)) - While both threads and async tasks are managed by a scheduler, they are different. Threads can run at the same time on different CPU cores, while async tasks usually wait their turn to use the CPU. ([Location 632](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=632)) - A JoinHandle allows you to wait for a thread to finish, pausing the program until that thread completes. Joining the thread means blocking the program until the thread is terminated. A JoinHandle implements the Send and Sync traits, which means that it can be sent between threads. However, a JoinHandle does not implement the Clone trait. This is because we need a unique JoinHandle for each thread. ([Location 672](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=672)) - If you have used other programming languages, you may have come across green threads. These threads are scheduled by something other than the operating system (for example, a runtime or a virtual machine). ([Location 685](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=685)) - Note that while Rust itself does not implement green threads, runtimes like Tokio do. ([Location 698](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=698)) - This stands for atomic reference counting, meaning that Arc keeps count of the references to the variable that is wrapped in an Arc. So, if we were to define an Arc<i32> and then reference it over four threads, the reference count would increase to four. The Arc<i32> would only be dropped when all four threads had finished referencing it, resulting in the reference count being zero. ([Location 740](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=740)) - Remember that Rust only allows us to have one mutable reference to a variable at any given time. A Mutex (mutual exclusion) is a smart pointer type that provides interior mutability by having the value inside the Mutex. ([Location 744](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=744)) - Acquiring the lock requires some overhead, as we might have to wait until it is released. ([Location 751](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=751)) - Short for conditional variable, this allows our threads to sleep and be woken when a notification is sent through the Condvar. We cannot send variables through the Condvar, but multiple threads can subscribe to a single Condvar. ([Location 753](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=753)) - At the point of waiting, the thread is said to be parked. ([Location 822](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=822)) - Notice that we have used the Relaxed term. It is critical to ensure that operations occur in a specific order to avoid data races and strange inconsistencies. This is where memory ordering comes into play. The Relaxed ordering, used with AtomicBool, ensures that the operations on the atomic variable are visible to all threads but does not enforce any particular order on the surrounding operations. ([Location 870](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=870)) - the main use cases involve operations that have a delay or potential delay in doing something or receiving something—for example, I/O calls to the filesystem or network requests. Async allows the program that calls these operations to continue without blocking, which could cause the program to hang and become less responsive. ([Location 904](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=904)) - A CPU can perform millions of operations in the time it takes for a single file to be opened. This is why I/O operations are often the bottleneck in a program’s performance. ([Location 920](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=920)) - In asynchronous programming, a task represents an asynchronous operation. The task-based asynchronous pattern (TAP) provides an abstraction over asynchronous code. You write code as a sequence of statements. ([Location 1252](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=1252)) - You may have noticed that await is used on the Tokio sleep functions that represent the steps that are not intensive and that we can wait on. We use the await keyword to suspend the execution of our step until the result is ready. When the await is hit, the async runtime can switch to another async task. ([Location 1346](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=1346)) - This is because that extra sleep function allowed the async runtime to switch context to other tasks and execute them until their await line was executed, and so on. This insertion of an artificial delay in the future to get the call rolling on other futures is informally referred to as cooperative multitasking. ([Location 1413](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=1413)) - A task is an asynchronous computation or operation that is managed and driven by an executor to completion. It represents the execution of a future, and it may involve multiple futures being composed or chained together. ([Location 1506](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=1506)) - One of the key features of async programming is the concept of a future. We’ve mentioned that a future is a placeholder object that represents the result of an asynchronous operation that has not yet completed. Futures allow you to start a task and continue with other operations while the task is being executed in the background. ([Location 1513](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=1513)) - You may have noticed that our poll function is not async. This is because an async poll function would return a circular dependency, as you would be sending a future to be polled in order to resolve a future being polled. With this, we can see that the future is the bedrock of the async computation. ([Location 1615](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=1615)) - The poll function takes a mutable reference of itself. However, this mutable reference is wrapped in a Pin, ([Location 1618](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=1618)) - Most normal primitives such as number, string, bool, structs, and enum implement the Unpin trait, enabling them to be moved around. ([Location 1626](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=1626)) - The *const String is a raw pointer to a string. This pointer directly references the memory address of the data. The pointer offers no safety guarantees. ([Location 1651](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=1651)) - A segmentation fault is an error caused by accessing memory that does not belong to the program. We can see that moving a struct with a reference to itself can be dangerous. Pinning ensures that the future remains at a fixed memory address. ([Location 1696](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=1696)) - A Context only serves to provide access to a waker to wake a task. A waker is a handle that notifies the executor when the task is ready to be run. ([Location 1703](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=1703)) - This is because our tasks are still idle, but there is no way to wake them up again to be polled and executed to completion. Futures need the Waker::wake() function so it can be called when the future should be polled again. ([Location 1740](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=1740)) - The process takes the following steps: The poll function for a future is called, and the result is that the future needs to wait for an async operation to complete before the future is able to return a value. The future registers its interest in being notified of the operation’s completion by calling a method that references the waker. The executor takes note of the interest in the future’s operation and stores the waker in a queue. At some later time, the operation completes, and the executor is notified. The executor retrieves the wakers from the queue and calls wake_by_ref on each one, waking up the futures. The wake_by_ref function signals the associated task that should be scheduled for execution. The way this is done can vary depending on the runtime. When the future is executed, the executor will call the poll method of the future again, and the future will determine whether the operation has completed, returning a value if completion is achieved. ([Location 1742](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=1742)) - We can also use a timeout on a thread of execution: the thread finishes when a certain amount of time has elapsed, so we do not end up with a program that hangs indefinitely. ([Location 1756](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=1756)) - could be cancelled if it doesn’t complete within the specified duration. This introduces the concept of cancel safety. ([Location 1798](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=1798)) - Cancel safety ensures that when a future is canceled, any state or resources it was using are handled correctly. ([Location 1799](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=1799)) - We can hold on the polling of the future by externally referencing the future’s waker and waking the future when we need to. ([Location 1822](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=1822)) - Although it can complicate things, we can share data between futures. We may want to share data between futures for the following reasons: Aggregating results Dependent computations Caching results Synchronization Shared state Task coordination and supervision Resource management Error propagation ([Location 1986](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=1986)) - We can see that we drop the guard before bothering to work out the return. This increases the time the guard is free for other futures and enables us to update the self.count. ([Location 2136](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=2136)) - If we run this, we get the exact same printout and behavior as our futures in the previous section. However, it’s clearly simpler and easier to write. There are trade-offs to both approaches. For instance, if we just wanted to write futures that have the behavior we have coded, it would make sense to use just an async function. However, if we needed more control over how a future was polled, or we do not have access to an async implementation but we have a blocking function that tries, then it would make sense to write the poll function ourselves. ([Location 2266](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=2266)) - This did not work for Rust because the callback model relies on dynamic dispatch, which means at runtime the exact function that was going to be called was determined at runtime as opposed to compile time. This produced additional overhead because the program had to work out what function to call at runtime. This violates the zero-cost approach and resulted in reduced performance. Rust opted for an alternative approach with the aim of optimizing runtime performance by using the Future trait, which uses polls. The runtime is responsible for managing when to call polls. It does not need to schedule callbacks and worry about working out what function to call, instead it can use polls to see if the future is completed. This is more efficient because futures can be represented as a state machine in a single heap allocation, and the state machine captures local variables that are needed to execute the async function. This means there is one memory allocation per task, without any concern that the memory allocation will be the incorrect size. ([Location 2278](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=2278)) - One common approach is by defining an async function using the async keyword before the function. However, as we’ve seen earlier, you can also manually create a future by implementing the Future trait yourself. When we call an async function, it returns a future. ([Location 2294](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=2294)) - We spawn a task with the future with await, which means we register with an executor. The executor then takes responsibility for taking the task to completion. To do this, it maintains a queue of tasks. ([Location 2297](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=2297)) - The executor processes the futures in the task by calling the poll method. ([Location 2299](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=2299)) - If the future is not pending (i.e., not ready), the executor places the task back into the queue to be executed in the future. ([Location 2302](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=2302)) - At some point, all the future in the task will complete, and the poll will return a ready. ([Location 2303](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=2303)) - async is not deterministic, meaning the execution order of async tasks is not set in stone, which, while initially daunting, opens a playground for optimization. ([Location 2561](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=2561)) - Cooperative multitasking isn’t just a trick; it’s a strategy to get the most out of our resources, something we’ve applied to accelerate our async operations. ([Location 2562](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=2562)) - task spawning serves as the entry point to the runtime. ([Location 2589](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=2589)) - flume provides unbounded channels that can hold an unlimited number of messages and implements lock-free algorithms. This makes flume particularly beneficial for highly concurrent programs, where the queue might need to handle a large number of messages in parallel, unlike the standard library channels that rely on a blocking mutex for synchronization. ([Location 2615](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=2615)) - The Future trait denotes that our future is going to result in either an error or the value T. Our future needs the Send trait because we are going to be sending our future into a different thread where the queue is based. The Send trait enforces constraints that ensure that our future can be safely shared among threads. ([Location 2667](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=2667)) - The static means that our future does not contain any references that have a shorter lifetime than the static lifetime. ([Location 2670](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=2670)) - Because we cannot guarantee when a task is finished, we must ensure that the lifetime of our task is static. ([Location 2673](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=2673)) - With static we are ensuring that our queue is living throughout the lifetime of the program. ([Location 2692](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=2692)) - Runnable is a handle for a runnable task. Every spawned task has a single Runnable handle, which exists only when the task is scheduled for running. ([Location 2697](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=2697)) - Once we have received our runnable, we run it in the catch_unwind function. We use this because we do not know the quality of the code being passed to our async runtime. Ideally, all Rust developers would handle possible errors properly, but in case they do not, catch_unwind runs the code and catches any error that’s thrown while the code is running, returning Ok or Err depending on the outcome. ([Location 2727](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=2727)) - Rust’s async syntax, which automatically handles the polling and scheduling of the task for us. Note that our sleep in async_fn is blocking because we want to see how the tasks are processed in our queue. ([Location 2813](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=2813)) - When a poll results in Pending, the task is then put back on the queue to be polled again. ([Location 2954](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=2954)) - To increase our number of threads working on the queue, we can add another thread consuming from the queue with a cloned receiver of our queue channel: ([Location 2975](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=2975)) - it is a useful approach to remember that certain problems can be solved by off-loading CPU-intensive tasks early in the program. ([Location 3038](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=3038)) - You may have come across warnings along the lines of “async is not for computationally heavy tasks.” Async is merely a mechanism, and you can use it for what you want as long as it makes sense. However, the warning is not without merit. ([Location 3039](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=3039)) - In task stealing, consuming threads steal tasks from other queues when their own queue is empty. ([Location 3203](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=3203)) - If we used the standard library for our Sender and Receiver, we would not be able to send the Sender or Receiver over to other threads. With flume, we make both of the channels static that are lazily evaluated inside our spawn_task function: ([Location 3219](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=3219)) - In production environments, you should aim to avoid sleeping in threads and instead use more responsive mechanisms like thread parking or condition variables. ([Location 3250](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=3250)) - Remember that the queues and channels are lazy in their evaluation. A task needs to be sent to the queue in order for the queue to start running. ([Location 3312](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=3312)) - Note that we may not want task stealing. For instance, if we put CPU-intensive tasks onto the high-priority queue and lightweight networking tasks on the low-priority queue, we would not want the low-priority queue stealing tasks from the high-priority queue. Otherwise, we run the risk of shutting down our network processing because of the low-priority-queue consumer threads being held up on CPU-intensive tasks. ([Location 3320](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=3320)) - We can also pass in async blocks and async functions because these are just syntactic sugar for futures. ([Location 3419](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=3419)) - We must remember that our tasks are being directly run. An error could occur in the execution of the task. To return a vector of results, we can create a try_join macro ([Location 3457](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=3457)) - You may remember that the queue is lazy: it will not start until it is called. This directly affects our task stealing. The example we gave was that if no tasks were sent to the high-priority queue, that queue would not start and therefore would not steal tasks from the low-priority queue if empty, and vice versa. Configuring a runtime to get things going and refine the number of consuming loops is not an unusual way of solving this problem. ([Location 3483](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=3483)) - Our run function defines the environment variables for the numbers and then spawns two tasks to both queues to set up the queues: ([Location 3561](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=3561)) - We use our join so that after the run function has been executed, both of our queues are ready to steal tasks. ([Location 3583](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=3583)) - Background processes are tasks that execute in the background periodically for the entire lifetime of the program. These processes can be used for monitoring and for maintenance tasks such as database cleanup, log rotation, and data updates to ensure that the program always has access to the latest information. ([Location 3619](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=3619)) - For instance, let’s say that a struct or function could create a background-running task. We do not need to try to juggle the task around the program so it does not get dropped, cancelling the background task. We can remove the need for juggling tasks to keep the background task running by using the detach method: ([Location 3671](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=3671)) - This method moves the pointer in the task into an unsafe loop that will poll the task and schedule it until it is finished. The pointer associated with the task in the main function is then dropped, dropping the need for keeping hold of the tasks in main. ([Location 3680](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=3680)) - An executor is responsible for running futures to completion. It is the part of the runtime that schedules tasks and makes sure they run (or are executed) when they are ready. We need an executor when we introduce networking into our runtime because without it, our futures such as HTTP requests would be created but never actually run. ([Location 3751](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=3751)) - A connector in networking is a component that establishes a connection between our application and the service we want to connect to. It handles activities like opening TCP connections and maintaining them through the lifetime of the request. ([Location 3756](https://readwise.io/to_kindle?action=open&asin=B0DMTWX6K4&location=3756))