Rust-Rexpect: rexpect — automate interactive applications such as ssh, ftp, passwd, etc

rexpect

Build Status crates.io Released API docs Master API docs

Spawn, control, and respond to expected patterns of child applications and processes, enabling the automation of interactions and testing. Components include:

  • session: start a new process and interact with it; primary module of rexpect.
  • reader: non-blocking reader, which supports waiting for strings, regex, and EOF.
  • process: spawn a process in a pty.

The goal is to offer a similar set of functionality as pexpect.

Examples

For more examples, check the examples directory.

Basic usage

Add this to your Cargo.toml

[dependencies]
rexpect = "0.4"

Simple example for interacting via ftp:

extern crate rexpect;

use rexpect::spawn;
use rexpect::errors::*;

fn do_ftp() -> Result<()> {
    let mut p = spawn("ftp speedtest.tele2.net", Some(30_000))?;
    p.exp_regex("Name \\(.*\\):")?;
    p.send_line("anonymous")?;
    p.exp_string("Password")?;
    p.send_line("test")?;
    p.exp_string("ftp>")?;
    p.send_line("cd upload")?;
    p.exp_string("successfully changed.\r\nftp>")?;
    p.send_line("pwd")?;
    p.exp_regex("[0-9]+ \"/upload\"")?;
    p.send_line("exit")?;
    p.exp_eof()?;
    Ok(())
}


fn main() {
    do_ftp().unwrap_or_else(|e| panic!("ftp job failed with {}", e));
}

Example with bash and reading from programs

extern crate rexpect;
use rexpect::spawn_bash;
use rexpect::errors::*;


fn do_bash() -> Result<()> {
    let mut p = spawn_bash(Some(2000))?;
    
    // case 1: wait until program is done
    p.send_line("hostname")?;
    let hostname = p.read_line()?;
    p.wait_for_prompt()?; // go sure `hostname` is really done
    println!("Current hostname: {}", hostname);

    // case 2: wait until done, only extract a few infos
    p.send_line("wc /etc/passwd")?;
    // `exp_regex` returns both string-before-match and match itself, discard first
    let (_, lines) = p.exp_regex("[0-9]+")?;
    let (_, words) = p.exp_regex("[0-9]+")?;
    let (_, bytes) = p.exp_regex("[0-9]+")?;
    p.wait_for_prompt()?; // go sure `wc` is really done
    println!("/etc/passwd has {} lines, {} words, {} chars", lines, words, bytes);

    // case 3: read while program is still executing
    p.execute("ping 8.8.8.8", "bytes of data")?; // returns when it sees "bytes of data" in output
    for _ in 0..5 {
        // times out if one ping takes longer than 2s
        let (_, duration) = p.exp_regex("[0-9. ]+ ms")?;
        println!("Roundtrip time: {}", duration);
    }
    p.send_control('c')?;
    Ok(())
}

fn main() {
    do_bash().unwrap_or_else(|e| panic!("bash job failed with {}", e));
}

Example with bash and job control

One frequent bitfall with sending ctrl-c and friends is that you need to somehow ensure that the program has fully loaded, otherwise the ctrl-* goes into nirvana. There are two functions to ensure that:

  • execute where you need to provide a match string which is present on stdout/stderr when the program is ready
  • wait_for_prompt which waits until the prompt is shown again
extern crate rexpect;
use rexpect::spawn_bash;
use rexpect::errors::*;


fn do_bash_jobcontrol() -> Result<()> {
    let mut p = spawn_bash(Some(1000))?;
    p.execute("ping 8.8.8.8", "bytes of data")?;
    p.send_control('z')?;
    p.wait_for_prompt()?;
    // bash writes 'ping 8.8.8.8' to stdout again to state which job was put into background
    p.execute("bg", "ping 8.8.8.8")?;
    p.wait_for_prompt()?;
    p.send_line("sleep 0.5")?;
    p.wait_for_prompt()?;
    // bash writes 'ping 8.8.8.8' to stdout again to state which job was put into foreground
    p.execute("fg", "ping 8.8.8.8")?;
    p.send_control('c')?;
    p.exp_string("packet loss")?;
    Ok(())
}

fn main() {
    do_bash_jobcontrol().unwrap_or_else(|e| panic!("bash with job control failed with {}", e));
}

Project Status

Rexpect covers more or less the features of pexpect. If you miss anything I'm happy to receive PRs or also Issue requests of course.

The tests cover most of the aspects and it should run out of the box for rust stable, beta and nightly on both Linux or Mac.

Design decisions

  • use error handling of error-chain
  • use nix (and avoid libc wherever possible) to keep the code safe and clean
  • sadly, expect is used in rust too prominently to unwrap Options and Results, use exp_* instead

Licensed under MIT License

Comments

  • add child.interact() e.g. to resize window when got SIGWINCH signal?
    add child.interact() e.g. to resize window when got SIGWINCH signal?

    Dec 10, 2019

                                                                                                                                                                                                            enhancement 
    Reply
  • Windows Support
    Windows Support

    Feb 11, 2020

    I am interested in adding Windows support for this crate. Would you be willing to accept a PR if I can get this working?

    Specifically, I want to look into adding Windows support via PseudoTerminal. Doing so will likely require some additions to this crate, including depending on winapi and some other platform-specific code for Windows builds.

    I don't intend on changing the API, as I like the simplicity, and would try not to change the internals much. That said, I haven't looked closely at this codebase yet so I don't really know what will be required.

    If this is something you see as possible, I can start working on it. It could take me a couple weeks (or more) to get something working.

    Reply
  • get the index of element that was matched by exp_any
    get the index of element that was matched by exp_any

    Mar 22, 2020

    find fn returns a tuple with the unread buffer and the matched string by needle. The question is if it could also return the position of the expression used in ReadUntil::Any(vec)? Right now I'm looping through the passed expressions and checking which one matches the returned matched string.

    Think "did I match username or password prompt, what to send back to the process".

    Reply
  • Set default terminal size to 80x25
    Set default terminal size to 80x25

    May 12, 2020

    It looks like otherwise the default is 0, which crashes some programs.

    I would make it customizable, but I'm not sure how to make API for that.

    Also see #10

    Reply
  • examples
    examples "ftp" and "bash_read" are broken

    May 25, 2020

    just a mental note to fix those. Also: might be good to mention the tcp example in the README, or maybe not…

    Reply
  • Strip ANSI escape codes
    Strip ANSI escape codes

    Jun 2, 2020

    Is it possible to leverage https://docs.rs/strip-ansi-escapes/0.1.0/strip_ansi_escapes somehow to strip ANSI escape codes from the output?

    Reply
  • rexpect 0.3 ready ?
    rexpect 0.3 ready ?

    Feb 25, 2018

    Hi @philippkeller In README.md, the example is referring to rexpect 0.3.0 while the version on crates.io is only upto 0.2.0.

    I tried to test out the examples in README.md using 0.2.0 but failed with following message.

    [[email protected] src]$cargo build
       Compiling cmd1 v0.1.0 (file:///home/tjyang/cmd1)
    error[E0061]: this function takes 1 parameter but 2 parameters were supplied
     --> src/main.rs:8:7
      |
    8 |     p.execute("ping 8.8.8.8", "bytes of data")?;
      |       ^^^^^^^ expected 1 parameter
    
    error[E0061]: this function takes 1 parameter but 2 parameters were supplied
      --> src/main.rs:12:7
       |
    12 |     p.execute("bg", "ping 8.8.8.8")?;
       |       ^^^^^^^ expected 1 parameter
    
    error[E0061]: this function takes 1 parameter but 2 parameters were supplied
      --> src/main.rs:17:7
       |
    17 |     p.execute("fg", "ping 8.8.8.8")?;
       |       ^^^^^^^ expected 1 parameter
    
    error: aborting due to 3 previous errors
    
    error: Could not compile `cmd1`.
    
    To learn more, run the command again with --verbose.
    [[email protected] src]$
    
    
    Reply
  • Send signals to underlying process?
    Send signals to underlying process?

    Aug 8, 2017

    I notice you can send a control character like <ctrl-c> for keyboard interrupt, or <ctrl-d> for EOF. Is there any way to send a signal like SIGTERM or SIGHUP (perhaps using a nix::sys::signal::Signal) to the underlying process?

    An alternative might be adding a method to the PtySession which gives you a mutable reference to the underlying PtyProcess so you can call signal() and friends directly on that.

    Reply
  • Mechanism to access buffer after an expectation
    Mechanism to access buffer after an expectation

    Sep 14, 2017

    A common pattern I've done in pexpect is something like the following:

    proc = pexpect.spawnu("foo")
    proc.setecho(False)
    proc.sendline("some input")
    proc.expect("> ")
    print(proc.before)
    

    The proc.before part allows me to access the result of the input. Is there currently an equivalent in rexpect?

    Reply
  • draft of a possible solution to the #14 issue and changes in public interface
    draft of a possible solution to the #14 issue and changes in public interface

    May 26, 2020

    Hi @philippkeller, In the #22 I put an example in what way may be changed the interface of ReadUntil. And today I went through the use cases of exp function and noticed that in all cases there's no need for all the data find function gives so I went slightly further and check what can be done.

    The main idea behind this PR is to provide a cleaner calls of read_until function. The example when we match string and return the buffer before and the string itself but we know the string already since we did a match by that so the second part is always needless.

    A bunch of examples

    let mut p = spawn("cat", Some(1000)).expect("cannot run cat");
    p.send_line("lorem ipsum dolor sit amet")?;
    assert_eq!("lorem ipsum dolor sit ", p.exp_string("amet")?);
    
    let f = io::Cursor::new("2014-03-15");
    let mut r = NBReader::new(f, None);
    let re = Regex::new(r"-\d{2}-").unwrap();
    assert_eq!(
        ("2014".to_string(), "-03-".to_string()),
        r.read_until(&re).expect("regex doesn't match")
    );
    
    let f = io::Cursor::new("abcdef");
    let mut r = NBReader::new(f, None);
    assert_eq!(
        "ab".to_string(),
         r.read_until(&NBytes(2)).expect("2 bytes")
    );
    

    The resolution of #14 could looks like

    we have an index of the ReadUntil element by which we get successful match, and the copy of the buffer on which it be called so if the user want he can repeat the match on the buffer.

    let until = vec![ReadUntil::NBytes(30), ReadUntil::String("Hi".to_string())];
    if let Ok(buffer, index)= = p.exp(until.as_slice()) {
       let res = &until[index]
          .clone()
          .string_needle()
          .unwrap()
          .find(&buffer, true)
          .unwrap();
    }
    

    This is a draft and it's affected by a linter which I am sorry about.

    All tests are passed.

    Reply
  • Need help on converting a bash script into rexpect
    Need help on converting a bash script into rexpect

    Feb 25, 2018

    Hi @philippkeller

    • And this is the command and logs from realm join command
    #realm join -v --computer-ou="ou=servers,dc=test,dc=com" test.com -U admin_ad
     * Resolving: _ldap._tcp.test.com
     * Performing LDAP DSE lookup on: 100.76.1.18
     * Performing LDAP DSE lookup on: 100.76.1.19
     * Performing LDAP DSE lookup on: 100.64.0.65
     * Successfully discovered: test.com
    Password for admin_ad: admin_ad_password
    
    • This is one-liner bash script I used to join a centos 7 to MS AD domain.
    realm join -v --computer-ou="ou=ma,ou=servers,dc=test,dc=com" test.com -U admin_ad << MYPASSWD
    admin_ad_password
    MYPASSWD
    
    • And this is using rexpect 0.3
    //Run this code with cargo
    // Join test.com AD domain on CentOS 7 with SSSD enabled.
    //  Following is the response from realm command
    //!  realm join -v --computer-ou="ou=unit1,ou=servers,dc=test,dc=com" test.com -U admin_ad
    //!  * Resolving: _ldap._tcp.test.com
    //!  * Performing LDAP DSE lookup on: 100.76.1.18
    //!  * Performing LDAP DSE lookup on: 100.76.1.19
    //!  * Performing LDAP DSE lookup on: 100.58.0.65
    //!  * Successfully discovered: test.com
    //! Password for admin_ad:
    
    extern crate rexpect;
    
    use rexpect::spawn;
    use rexpect::errors::*;
    
    fn  join() -> Result<()> {
        let mut p = spawn("realm join -v --computer-ou=\"ou=ma,ou=servers,dc=test,dc=com\" test.com -U admin_ad", Some(30_000))?;
        p.exp_regex("^Password for .*: ")?;
        p.send_line("admin_ad_password")?;
        p.exp_eof()?;
        Ok(())
    }
    
    
    fn main() {
        join().unwrap_or_else(|e| panic!("Failed to joing test.com AD domain {}", e));
    }
    
    • The error Is there a way to skip first 5 lines from realm command output ?
    >target/debug/joinad
    thread 'main ' panicked at 'Failed to joing test.com AD domain EOF (End of File): Expected Regex: "^Password for .*: " but got EOF after reading " * Resolving: _ldap._tcp.test.com
    
    • I inserted "p.wait_for_prompt()?;" before " p.exp_regex" but I got compilation error saying no wait_for_promp method for ptysession ...
    Reply
  • Rare inconsistent result.
    Rare inconsistent result.

    Mar 15, 2018

    While testing gluon with rexpect, there has been one inconsistent test, specicially the hello_world test. It may be a problem of how I used the api or a rare bug with the repl. Rexpect has worked consistently with every other test that's been thrown at it.

    There are a few different variations tried in here, including the original without expecting anything after the first sent import line, expecting a regex, and using execute -- https://github.com/gluon-lang/gluon/pull/478.

    It would be great to get your eyes on this if you spot anything I'm missing while using rexpect :)

    Reply