Rust-Mio: mio — MIO is a lightweight IO library for Rust with a focus on adding as little overhead as possible over the OS abstractions

Mio – Metal IO

Mio is a fast, low-level I/O library for Rust focusing on non-blocking APIs and event notification for building high performance I/O apps with as little overhead as possible over the OS abstractions.

Crates.io MIT licensed Build Status Build Status

API documentation

This is a low level library, if you are looking for something easier to get started with, see Tokio.

Usage

To use mio, first add this to your Cargo.toml:

[dependencies]
mio = "0.7"

Next we can start using Mio. The following is quick introduction using TcpListener and TcpStream. Note that features = ["os-poll", "tcp"] must be specified for this example.

use std::error::Error;

use mio::net::{TcpListener, TcpStream};
use mio::{Events, Interest, Poll, Token};

// Some tokens to allow us to identify which event is for which socket.
const SERVER: Token = Token(0);
const CLIENT: Token = Token(1);

fn main() -> Result<(), Box<dyn Error>> {
    // Create a poll instance.
    let mut poll = Poll::new()?;
    // Create storage for events.
    let mut events = Events::with_capacity(128);

    // Setup the server socket.
    let addr = "127.0.0.1:13265".parse()?;
    let mut server = TcpListener::bind(addr)?;
    // Start listening for incoming connections.
    poll.registry()
        .register(&mut server, SERVER, Interest::READABLE)?;

    // Setup the client socket.
    let mut client = TcpStream::connect(addr)?;
    // Register the socket.
    poll.registry()
        .register(&mut client, CLIENT, Interest::READABLE | Interest::WRITABLE)?;

    // Start an event loop.
    loop {
        // Poll Mio for events, blocking until we get an event.
        poll.poll(&mut events, None)?;

        // Process each event.
        for event in events.iter() {
            // We can use the token we previously provided to `register` to
            // determine for which socket the event is.
            match event.token() {
                SERVER => {
                    // If this is an event for the server, it means a connection
                    // is ready to be accepted.
                    //
                    // Accept the connection and drop it immediately. This will
                    // close the socket and notify the client of the EOF.
                    let connection = server.accept();
                    drop(connection);
                }
                CLIENT => {
                    if event.is_writable() {
                        // We can (likely) write to the socket without blocking.
                    }

                    if event.is_readable() {
                        // We can (likely) read from the socket without blocking.
                    }

                    // Since the server just shuts down the connection, let's
                    // just exit from our event loop.
                    return Ok(());
                }
                // We don't expect any events with tokens other than those we provided.
                _ => unreachable!(),
            }
        }
    }
}

Features

  • Non-blocking TCP, UDP
  • I/O event queue backed by epoll, kqueue, and IOCP
  • Zero allocations at runtime
  • Platform specific extensions

Non-goals

The following are specifically omitted from Mio and are left to the user or higher-level libraries.

  • File operations
  • Thread pools / multi-threaded event loop
  • Timers

Platforms

Currently supported platforms:

  • Android
  • DragonFly BSD
  • FreeBSD
  • Linux
  • NetBSD
  • OpenBSD
  • Solaris
  • Windows
  • iOS
  • macOS

There are potentially others. If you find that Mio works on another platform, submit a PR to update the list!

Mio can handle interfacing with each of the event systems of the aforementioned platforms. The details of their implementation are further discussed in the Poll type of the API documentation (see above).

The Windows implementation for polling sockets is using the wepoll strategy. This uses the Windows AFD system to access socket readiness events.

Community

A group of Mio users hang out on Discord, this can be a good place to go for questions.

Contributing

Interested in getting involved? We would love to help you! For simple bug fixes, just submit a PR with the fix and we can discuss the fix directly in the PR. If the fix is more complex, start with an issue.

If you want to propose an API change, create an issue to start a discussion with the community. Also, feel free to talk with us in Discord.

Finally, be kind. We support the Rust Code of Conduct.

Comments

  • 0.7 test fails on s390x
    0.7 test fails on s390x

    Apr 29, 2020

    While trying to package mio-0.7 for fedora rawhide, it fails to pass the test: https://kojipkgs.fedoraproject.org//work/tasks/5377/43905377/build.log

    test tcp_listener_ipv6 ... FAILED
    failures:
    ---- tcp_listener_ipv6 stdout ----
    thread 'tcp_listener_ipv6' panicked at 'the following expected events were not found: [ExpectEvent { token: Token(0), readiness: Readiness(1) }]', tests/util/mod.rs:158:5
    note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
    failures:
        tcp_listener_ipv6
    

    and

    test tcp_stream_ipv6 ... FAILED
    failures:
    ---- tcp_stream_ipv6 stdout ----
    thread 'tcp_stream_ipv6' panicked at 'the following expected events were not found: [ExpectEvent { token: Token(0), readiness: Readiness(1) }]', tests/util/mod.rs:158:5
    note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
    failures:
        tcp_stream_ipv6
    
    Reply
  • Cargo.toml: exclude more CI files
    Cargo.toml: exclude more CI files

    Apr 30, 2020

                                                                                                                                                                                                           
    Reply
  • MIO should not report out-of-band data as 'is_readable'.
    MIO should not report out-of-band data as 'is_readable'.

    May 9, 2020

    As seen here.

    This is a vector for denial-of-service attacks. For background, read https://github.com/sandstorm-io/sandstorm-website/blob/ccee81abb2968e8a0301b1a63e189be719bc02d7/_posts/2015-04-08-osx-security-bug.md.

    Reply
  • crates.io has 0.6 readme for the 0.7.0 version
    crates.io has 0.6 readme for the 0.7.0 version

    May 26, 2020

    This is a little off. Is there a way to update this? https://crates.io/crates/mio image

    Reply
  • Solaris 11.4 does support SOCK_CLOEXEC and SOCK_NONBLOCK
    Solaris 11.4 does support SOCK_CLOEXEC and SOCK_NONBLOCK

    May 29, 2020

    https://github.com/tokio-rs/mio/blob/68805cbeb7797e0d2653e861e009cc10c05bd4f9/src/sys/unix/net.rs#L56 According to their document https://docs.oracle.com/cd/E88353_01/html/E37843/socket-3c.html but still not sure about previous versions.

    Reply
  • Add a check that a single Waker is active per Poll instance
    Add a check that a single Waker is active per Poll instance

    Jun 12, 2020

    Ensuring the API is used properly (at least during tests).

    See #1283

    Reply
  • Remove net2 dependency
    Remove net2 dependency

    Jul 11, 2019

    This initial comment only remove net2 from TcpStream::connect, on Unix platforms (expect for iOS, macOS and Solaris) this reduces the number of system calls from three to two.

    @carllerche this does increase the complexity a bit, do we still want to continue down this route?

    Closes #841. Closes #1045.

    Reply
  • tcp_stream test fails with TcpStream::set_ttl crash
    tcp_stream test fails with TcpStream::set_ttl crash

    Oct 30, 2019

    Problem: When running the unit tests on Windows 8.1 Pro, tcp_stream ttl test crash in TcpStrea::set_ttl :

    RUST_BACKTRACE=1 cargo test --test tcp_stream
        Finished dev [unoptimized + debuginfo] target(s) in 0.10s
         Running target\debug\deps\tcp_stream-7340f8b6dbab4dde.exe
    
    running 12 tests
    test is_send_and_sync ... ok
    test shutdown_write ... ignored
    test tcp_stream_ipv4 ... ignored
    test tcp_stream_ipv6 ... ignored
    test ttl ... FAILED
    test registering ... ok
    test deregistering ... ok
    test nodelay ... ok
    test reregistering ... ok
    test shutdown_both ... ok
    test shutdown_read ... ok
    test try_clone ... ok
    
    failures:
    
    ---- ttl stdout ----
    thread 'ttl' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 10022, kind: InvalidInput, message: "An invalid argument was supplied." }', src\libcore\result.rs:1084:5
    stack backtrace:
       0: backtrace::backtrace::trace_unsynchronized
                 at C:\Users\VssAdministrator\.cargo\registry\src\github.com-1ecc6299db9ec823\backtrace-0.3.34\src\backtrace\mod.rs:66
       1: std::sys_common::backtrace::_print
                 at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54\/src\libstd\sys_common\backtrace.rs:47
       2: std::sys_common::backtrace::print
                 at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54\/src\libstd\sys_common\backtrace.rs:36
       3: std::panicking::default_hook::{{closure}}
                 at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54\/src\libstd\panicking.rs:200
       4: std::panicking::default_hook
                 at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54\/src\libstd\panicking.rs:211
       5: std::panicking::rust_panic_with_hook
                 at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54\/src\libstd\panicking.rs:477
       6: std::panicking::continue_panic_fmt
                 at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54\/src\libstd\panicking.rs:384
       7: std::panicking::rust_begin_panic
                 at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54\/src\libstd\panicking.rs:311
       8: core::panicking::panic_fmt
                 at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54\/src\libcore\panicking.rs:85
       9: core::result::unwrap_failed
                 at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54\/src\libcore\result.rs:1084
      10: core::result::Result<(), std::io::error::Error>::unwrap<(),std::io::error::Error>
                 at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54\src\libcore\result.rs:852
      11: tcp_stream::ttl
                 at .\tests\tcp_stream.rs:177
      12: tcp_stream::ttl::{{closure}}
                 at .\tests\tcp_stream.rs:168
      13: core::ops::function::FnOnce::call_once<closure-0,()>
                 at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54\src\libcore\ops\function.rs:235
      14: alloc::boxed::{{impl}}::call_once<(),FnOnce<()>>
                 at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54\src\liballoc\boxed.rs:787
      15: panic_unwind::__rust_maybe_catch_panic
                 at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54\/src\libpanic_unwind\lib.rs:80
      16: std::panicking::try
                 at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54\src\libstd\panicking.rs:275
      17: std::panic::catch_unwind
                 at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54\src\libstd\panic.rs:394
      18: test::run_test::run_test_inner::{{closure}}
                 at /rustc/625451e376bb2e5283fc4741caa0a3e8a2ca4d54\/src\libtest\lib.rs:1408
    note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
    
    
    failures:
        ttl
    
    test result: FAILED. 8 passed; 1 failed; 3 ignored; 0 measured; 0 filtered out
    

    Cause: According to @PerfectLaugh:

    I figured it out that the windows tcpstream::connect was not allowed to use set_ttl when you aren't not connected yet since we used the non_blocking on connect
    

    Solution: ?

    windows 
    Reply
  • Support Windows
    Support Windows

    Apr 20, 2015

    Overview

    Currently, Mio currently only supports Linux and Darwin platforms (though *BSD support could happen relatively easily). It uses epoll and kqueue respectively to provide a readiness API to consumers. Windows offers a completion based API (completion ports) which is significantly different from epoll & kqueue. The goal would be to tweak Mio in order to support Windows while still maintaining low overhead that mio strives for across all platforms.

    History

    I have wavered a bunch on the topic of how to best support Windows. At first, I had originally planned to do whatever was needed to support windows even if the implementation was less than ideal. Then, started towards not supporting windows with Mio and instead provide a standalone IO library that supported windows only. I started investigating the IOCP APIs in more depth and thinking about what a windows library would look like and it was very similar to what mio already is.

    Completion Ports

    There are a number of details related to using completion ports, but what matters is that instead of being notified when an operation (read, write, accept, ...) is ready to be performed and then performing the operation, an operation is submitted and then completion is signaled by reading from a queue.

    For example, when reading, a byte buffer is provided to the operating system. The operating system then takes ownership of the buffer until the operation completes. When the operation completes, the application is notified by reading off of the completion status queue

    Strategy

    The strategy would be to, on windows, internally manage a pool of buffers. When a socket is registered with the event loop with readable interest, a read a system read would be initiated supplying an available buffer. When the read completes, the internal buffer is now full. The the event loop would notify readiness and the user will then be able to read from the socket. The read would copy data from the internal buffer to the user's buffer and the read would be complete.

    On write, the user's data would be copied to a an internal buffer immediately and then the internal buffer submitted to the OS for the system write call.

    Mio API changes

    In order to implement the above strategy, Mio would not be able to rely on IO types from std::net anymore. As such, I propose to bring back TcpStream and TcpListener implemented in mio::net. Since Mio will then own all IO types, there will be no more need to have the NonBlock wrapper. Also, it seems thatNonBlock` can be confusing (see #154). So, all IO types in mio will always be blocking.

    I believe that this will be the only required API change.

    windows 
    Reply
  • Does deregister clear events already pending delivery? Should it?
    Does deregister clear events already pending delivery? Should it?

    Jul 25, 2015

    Let's say I have two event sources registered, both of them fired at the same time and landed in pending events list for current iteration.

    While handling the first event, I do deregister on the other one. Will I still get it? It seems to me that I will, which is very inconvenient, as it makes hard (impossible?) to safely deregister events other than the one that just fired.

    A spin of the above case is: What if I do deregister and right after used register_opt on different event source, using the same token.

    Another spinoff is: What if I do reregister using different interest.

    If I am not missing anything, it seems to me that either: the end of each event delivery iteration should have a notification, during which calls like deregister and reregister can be safely used, or deregister and reregister should make sure to remove any events that are not to be delivered anymore.

    Reply
  • Proposal: Unify Sockets, Timers, and Channels
    Proposal: Unify Sockets, Timers, and Channels

    Feb 23, 2016

    Unify Sockets, Timers, and Channels

    Currently, there are two runtime APIs: Poll and EventLoop. Poll is the abstraction handling readiness for sockets and waiting for events on these sockets. EventLoop is a wrapper around Poll providing a timeout API and a cross thread notification API as well as the loop / reactive (via Handler) abstraction.

    I am proposing to extract the timer and the notification channel features into standalone types that implement Evented and thus can be used directly with Poll. For example:

    let timer = mio::Timer::new();
    let poll = mio::Poll::new();
    
    poll.register(&timer, Token(0), EventSet::readable(), PollOpt::edge());
    
    timer.timeout("hello", Duration::from_millis(1_000));
    
    poll.poll();
    

    The insight is that a lot of the code that is currently windows specific is useful to all platforms. Stabilizing the API and providing it to all platforms allows implementing Evented for arbitrary types.

    Advantages

    • Mio would have a unified API.
    • No change in overhead for any platform.
    • Move (and improve) some of the currently windows-only code to all platforms.
    • The notification channel backend could be configurable:
    let (tx, rx) = mio::Channel::new(mpsc::channel());
    poll.register(&rx, Token(0), EventSet::readable(), PollOpt::edge());
    

    Disadvantages

    • Unsafe code
    • More code (lock-free algorithms)

    The primary disadvantage that I can think of is that the code path around timers & the notification channel become slightly more complicated. I don't believe that the change would have a meaningful performance impact.

    There is also additional code complexity for all platforms. However, this code complexity already exists for Windows.

    Behavior

    An Evented would mirror the behavior of a socket registered with epoll. Specifically, in a single threaded environment:

    • A value registered will trigger at most one notification per call to poll.
    • A value registered with readable interest & edge triggers a notification once when it becomes readable.
    • A value registered with readable interest & level triggers a notification every call to poll as long as the value is still readable.
    • A value registered (or reregistered) with readable interest triggers a notification immediately if it is currently readable.
    • If a value is registered with readable interest only and already has a pending writable notification, the event is discarded
    • If a value has any pending notifications and is deregistered, the pending notifications are cleared.
    • When a value is dropped, it will no longer trigger any further notifications.
    • Poll is permitted to fire of spurious readiness events except if the value has been dropped.

    In the presence of concurrency, specifically readiness being modified on a different thread than Poll, a best effort is made to preserve these semantics.

    Implementation

    This section will describe how to implement a custom Evented type as well as Mio's internals to handle it. For simplicity and performance, custom Evented types will only be able to be registered with a single Poll.

    It is important to note that the implementation is not intended to replace FD polling on epoll & kqueue. It is meant to work in conjunction with the OS's event queue to support types that cannot be implemented using a socket or other system type that is compatible with the system's event queue.

    Readiness Queue

    Poll will maintain an internal readiness queue, represented as a linked list. The linked list head pointer is an AtomicPtr. All of the nodes in the linked list are owned by the Poll instance.

    The type declarations are for illustration only. The actual implementations will have some additional memory safety requirements.

    struct Poll {
        readiness_queue: Arc<PollReadinessQueue>,
    }
    
    struct PollReadinessQueue {
        // All readiness nodes owned by the `Poll` instance. When the `Poll`
        // instance is freed, the list is walked and each Arc ref count is
        // decremented.
        head_all_nodes: Box<ReadinessNode>,
    
        // linked list of nodes that are pending some processing
        head_readiness: AtomicPtr<ReadinessNode>,
    
        // Hashed wheel timer for delayed readiness notifications
        readiness_wheel: Vec<AtomicPtr<ReadinessNode>>,
    }
    
    struct ReadinessNode {
        // Next node in ownership tracking queue
        next_all_nodes: Box<ReadinessNode>,
        // Used when the node is queued in the readiness linked list OR the
        // linked list for a hashed wheel slot.
        next_readiness: *mut ReadinessNode,
        // The Token used to register the `Evented` with `Poll`. This can change,
        // but only by calling `Poll` functions, so there will be no concurrency.
        token: Token,
        // The set of events to include in the notification on next poll
        events: AtomicUsize,
        // Tracks if the node is queued for readiness using the MSB, the
        // rest of the usize is the readiness delay.
        queued: AtomicUsize,
        // Both interest and opts can be mutated
        interest: Cell<EventSet>,
        // Poll opts
        opts: Cell<PollOpt>,
    }
    
    // Implements `Sync`, aka all functions are safe to call concurrently
    struct Registration {
        node: *mut ReadinessNode,
        queue: Arc<PollReadinessQueue>,
    }
    
    struct MyEvented {
        registration: Option<Registration>,
    }
    

    Registration

    When a MyEvented value is registered with the event loop, a new Registration value is obtained:

    my_evented.registration = Some(Registration::new(poll, token, interest));
    

    Registration will include the internal EventSet::dropped() event to the interest.

    Re-registration

    A Registration's interest & PollOpt can be changed by calling Registration::update:

    // poll: &Poll
    my_evented.registration.as_ref().unwrap()
        .update(poll, interest, opts);
    

    The Poll reference will not be used but will ensure that update is only called from a single thread (the thread that owns the Poll reference). This allows safe mutation of interest and opts without synchronization primitives.

    Registration will include the internal EventSet::dropped() event to the interest.

    Triggering readiness notifications

    Readiness can be updated using Registration::set_readiness and Registration::unset_readiness. These can be called concurrently. set_readiness adds the given events with the existing Registration readiness. unset_readiness subtracts the given events from the existing Registration.

    my_evented.registration.as_ref().unwrap().set_readiness(EventSet::readable());
    my_evented.registration.as_ref().unwrap().unset_readiness(EventSet::readable());
    

    Registration::set_readiness ensures that the registration node is queued for processing.

    Delaying readiness

    In order to support timeouts, Registration has the ability to schedule readiness notifications using Registration::delay_readiness(events, timeout).

    There is a big caveat. There is precise timing guarantee. A delayed readiness event could be triggered much earlier than requested. Also, the readiness timer is coarse grained, so by default will be rounded to 100ms or so. The one guarantee is that the event will be triggered no later than the requested timeout + the duration of a timer tick (100ms by default).

    Queuing Registration for processing

    First, atomically update Registration.queued. Attempt to set the MSB. Check the current delay value. If the requested delay is less than the current, update the delayed portion of queued.

    If the MSB was successfully set, then the current thread is responsible for queuing the registration node (pseudocode):

    loop {
        let ptr = PollReadinessQueue.readiness_head.get();
        ReadinessNode.next_readiness = ptr;
    
        if PollReadinessQueue.readiness_head.compare_and_swap(ptr, &ReadinessNode) {
            return;
        }
    }
    

    Dropping Registration

    Processing a drop is handled by setting readiness to an internal Dropped event:

    fn drop(&mut self) {
        self.registration.as_ref().unwrap()
            .set_readiness(EventSet::dropped());
    }
    

    The Registration value itself does not own any data, so there is nothing else to do.

    Polling

    On Poll::poll() the following happens:

    Reset the events on self

    self.events.clear();
    

    Atomically take ownership of the readiness queue:

    let ready_nodes = PollReadinessQueue.readiness_head.swap(ptr::null());
    

    The dequeued nodes are processed.

    for node in ready_nodes {
        // Mask the readiness info by the node's interest. This is needed to
        // support concurrent setting of readiness. Another thread may not
        // be aware of the latest interest value.
        let mut events = node.events.get() & node.interest;
    
        // Used to read the delay component of `Registration::queued`.
        let delay;
    
        if opts.is_edge() || events.is_empty() {
            // If the registration is edge, the node is always dequeued. If
            // it is level, we only dequeue the event when there are no
            // events (aka, no readiness). By not dequeing the event it will
            // be processed again next call to `poll`
            delay = unset_msb_and_read_delay_component(&node.queued);
    
            // Reload the events to ensure that we don't "lose" any
            // readiness notifications. Remember, it's ok to have
            // spurious notifications. 
            events = node.events.get() | node.interest;
        } else if !events.is_drop() {
            // Push the node back into the queue. This is done via a compare
            // and swap on `readiness_head`, pushing the node back to the
            // front.
            prepend(&ready_nodes, node);
    
            delay = read_delay_component(&node.queued);
        }
    
        if delay > 0 {
            node.update_delay_in_hashed_wheel(delay);
        } else {
            // The event will be fired immediately, if the node is currently
            // being delayed, remove it from the hashed wheel.
            if node.is_currently_in_hashed_wheel() {
                node.remove_from_hashed_wheel();
            }
    
            if events.is_drop() {
                // The registration is dropped. Unlink the node, freeing the
                // memory.
                node.unlink();
                continue;
            }
    
            if !events.is_empty() {
                // Track the event
                self.events.push_event(node.token, events);
            }
        }
    
    }
    

    The next step is to process all delayed readiness nodes that have reached their timeout. The code for this is similar to the current timer code.

    Integrating with Selector

    The readiness queue described above is not to replace socket notifications on epoll / kqueue / etc... It is to be used in conjuction.

    To handle this, PollReadinessQueue will be able to wakup the selector. This will be implemented in a similar fashion as the current channel implementation. A pipe will be used to force the selector to wakeup.

    The full logic of poll will look something like:

    let has_pending = !readiness_queue.is_empty();
    
    if has_pending {
        // Original timeout value is passed to the function...
        timeout = 0;
    }
    
    // Poll selector
    selector.poll(&mut self.events, timeout);
    
    // Process custom evented readiness queue as specified above.
    

    Implementing mio::Channel

    Channel is a mpsc queue such that when messages are pushed onto the channel, Poll is woken up and returns a readable readiness event for the Channel. The specific queue will be supplied on creation of Channel, allowing the user to choose the behavior around allocation and capacity.

    Channel will look something like:

    struct Channel<Q> {
        queue: Q,
    
        // Poll registration
        registration: Option<Registration>,
    
        // Tracks the number of pending messages.
        pending: AtomicUsize,
    }
    

    When a new message is sent over the channel:

    self.queue.push(msg);
    
    let prev = self.pending.fetch_add(1);
    
    if prev == 0 {
        // set readiness
        self.registration.as_ref().unwrap()
            .set_readiness(EventSet::readable());
    }
    

    When readiness is set, Poll will wake up with a readiness notification. The user can now "poll" off of the channel. The implementation of poll is something like:

    self.queue.poll().map(|msg| {
        let first = self.pending.get();
    
        if first == 1 {
            self.registration.as_ref().unwrap()
                .unset_readiness(EventSet::readable());
        }
    
        let second = self.pending.fetch_sub(1);
    
        if first == 1 && second > 0 {
            // There still are pending messages, reset readiness
            self.registration.as_ref().unwrap()
                .set_readiness(EventSet::readable());
        }
    
        msg
    })
    

    Implemented Timer

    Timer is a delay queue. Messages are pushed onto it with a delay after which the message can be "popped" from the queue. It is implemented using a hashed wheel timer strategy which is ideal in situations where large number of timeouts are required and the timer can use coarse precision (by default, 100ms ticks).

    The implementation is fairly straight forward. When a timeout is requested, the message is stored in the Timer implementation and Registration::delay_readiness is called with the timeout. There are some potential optimizations, but those are out of scope for this proposal.

    Windows

    The readiness queue described in this proposal would replace the current windows specific implementation. The proposal implementation would be more efficient as it avoids locking as well as uses lighter weight data structures (mostly, linked lists vs. vecs).

    Outstanding questions

    The biggest outstanding question would be what to do about EventLoop. If this proposal lands, then EventLoop becomes entirely a very light shim around Poll that dispatches events to the appropriate handler function.

    The entire implementation would look something like:

    pub fn run(&mut self, handler: &mut H) -> io::Result<()> {
        self.run = true;
    
        while self.run {
            self.poll.poll();
    
            for event in self.poll.events() {
                handler.ready(self, event.token(), event.kind());
            }
    
            handler.tick(self);
        }
    }
    

    It will also not be possible to maintain API compatibility. Handler::notify and Handler::timeout will no longer exist as EventLoop does not know the difference between those two types and other Evented types that have notifications called through ready.

    The options are:

    • Update EventLoop to follow the new API and keep the minimal impelmentation.
    • Get rid of EventLoop and make Poll the primary API
    • Provide a hire level API via EventLoop that accepts allocations (though this would be post 1.0).

    Alternatives

    It is possible to implement Timer and Channel as standalone types without having to implement the readiness queue. For Timer, it would require using timerfd on linux and a timer thread on other platforms. The disadvanage here is minor for linux as syscalls can be reduced significantly by only using timerfd to track the next timeout in the Timer vs. every timeout in Timer.

    However, on platforms that don't have timerfd available, a polyfill will be needed. This can be done by creating a pipe and spawning a thread. When a timeout is needed, send a request to the thread. The thread writes a byte to the pipe after the timeout has expired. This has overhead, but again it can be amortized by only using the thread/pipe combo for the next timeout in Timer vs. every timeout. Though, there may be some complication with this amoritization when registering the Timer using level triggered notifications.

    On the other hand. For Channel, a syscall would be needed for each message enqueued and dequeued. The implementation would be to have a pipe associated with the Chanenl. Each time a message is enqueued, write a byte to the pipe. Whenever a message is dequeued, read a byte.

    api behavior windows 
    Reply
  • Confusing behavior when multiple events for a single socket are fired in a single event loop iteration
    Confusing behavior when multiple events for a single socket are fired in a single event loop iteration

    May 28, 2015

    Issue representing the confusion experienced in #172 and #179.

    My current understanding of the confusion is, given epoll & a socket that received a HUP and is writable, mio will first call the Handler::readable w/ ReadHint set to Hup followed by the Handler::writable event. The confusion arises when the socket is closed during the Handler::readable call, or perhaps the token associated with that token is reused. The next call to Handler::writable for the "finalized" socket is unexpected.

    Part of the original API design consideration is how to unify epoll w/ kqueue. With kqueue, the readable & writable notifications are provided separately. As far as I am aware, there is no guarantee as to which order (readable or writable) the events will be seen.

    I don't believe that either solutions described by @rrichardson in #179 would be possible to implement (efficiently) for kqueue. The reason why hup / error are provided as a ReadHint is that it is not 100% portable for mio to detect them. However, the hup / error state can be discovered by the mio by reading from the socket, which is why they are considered readable events. In other words, to correctly use mio, hup / error states must be discovered by listening for Handler::readable, reading from the socket, and seeing an error. As such, doing anything with the ReadHint argument passed to Handler::readable is simply an optimization. I also don't see a way to implement passing ReadHint (renamed to EventHint) to Handler::writable callback in a portable fashion.

    Reply