Rust-Rust rdkafka: A fully asynchronous, futures-enabled Apache Kafka client library for Rust based on librdkafka.

rust-rdkafka

crates.io docs.rs Build Status coverate Join the chat at https://gitter.im/rust-rdkafka/Lobby

A fully asynchronous, futures-enabled Apache Kafka client library for Rust based on librdkafka.

The library

rust-rdkafka provides a safe Rust interface to librdkafka. The master branch is currently based on librdkafka 1.4.2.

Documentation

Features

The main features provided at the moment are:

  • Support for all Kafka versions since 0.8.x. For more information about broker compatibility options, check the librdkafka documentation.
  • Consume from single or multiple topics.
  • Automatic consumer rebalancing.
  • Customizable rebalance, with pre and post rebalance callbacks.
  • Synchronous or asynchronous message production.
  • Customizable offset commit.
  • Create and delete topics and add and edit partitions.
  • Alter broker and topic configurations.
  • Access to cluster metadata (list of topic-partitions, replicas, active brokers etc).
  • Access to group metadata (list groups, list members of groups, hostnames, etc.).
  • Access to producer and consumer metrics, errors and callbacks.

One million messages per second

rust-rdkafka is designed to be easy and safe to use thanks to the abstraction layer written in Rust, while at the same time being extremely fast thanks to the librdkafka C library.

Here are some benchmark results using the BaseProducer, sending data to a single Kafka 0.11 process running in localhost (default configuration, 3 partitions). Hardware: Dell laptop, with Intel Core i7-4712HQ @ 2.30GHz.

  • Scenario: produce 5 million messages, 10 bytes each, wait for all of them to be acked

    • 1045413 messages/s, 9.970 MB/s (average over 5 runs)
  • Scenario: produce 100000 messages, 10 KB each, wait for all of them to be acked

    • 24623 messages/s, 234.826 MB/s (average over 5 runs)

For more numbers, check out the kafka-benchmark project.

Client types

rust-rdkafka provides low level and high level consumers and producers.

Low level:

  • BaseConsumer: a simple wrapper around the librdkafka consumer. It must be periodically poll()ed in order to execute callbacks, rebalances and to receive messages.
  • BaseProducer: a simple wrapper around the librdkafka producer. As in the consumer case, the user must call poll() periodically to execute delivery callbacks.
  • ThreadedProducer: a BaseProducer with a separate thread dedicated to polling the producer.

High level:

For more information about consumers and producers, refer to their module-level documentation.

Warning: the library is under active development and the APIs are likely to change.

Asynchronous data processing with Tokio

Tokio is a platform for fast processing of asynchronous events in Rust. The interfaces exposed by the StreamConsumer and the FutureProducer allow rust-rdkafka users to easily integrate Kafka consumers and producers within the Tokio platform, and write asynchronous message processing code. Note that rust-rdkafka can be used without Tokio.

To see rust-rdkafka in action with Tokio, check out the asynchronous processing example in the examples folder.

At-least-once delivery

At-least-once delivery semantics are common in many streaming applications: every message is guaranteed to be processed at least once; in case of temporary failure, the message can be re-processed and/or re-delivered, but no message will be lost.

In order to implement at-least-once delivery the stream processing application has to carefully commit the offset only once the message has been processed. Committing the offset too early, instead, might cause message loss, since upon recovery the consumer will start from the next message, skipping the one where the failure occurred.

To see how to implement at-least-once delivery with rdkafka, check out the at-least-once delivery example in the examples folder. To know more about delivery semantics, check the message delivery semantics chapter in the Kafka documentation.

Users

Here are some of the projects using rust-rdkafka:

If you are using rust-rdkafka, please let us know!

Installation

Add this to your Cargo.toml:

[dependencies]
rdkafka = { version = "0.23", features = ["cmake-build"] }

This crate will compile librdkafka from sources and link it statically to your executable. To compile librdkafka you'll need:

  • the GNU toolchain
  • GNU make
  • pthreads
  • zlib: optional, but included by default (feature: libz)
  • cmake: optional, not included by default (feature: cmake-build)
  • libssl-dev: optional, not included by default (feature: ssl)
  • libsasl2-dev: optional, not included by default (feature: gssapi)
  • libzstd-dev: optional, not included by default (feature: zstd-pkg-config)

Note that using the CMake build system, via the cmake-build feature, is encouraged if you can take the dependency on CMake.

By default a submodule with the librdkafka sources pinned to a specific commit will be used to compile and statically link the library. The dynamic-linking feature can be used to instead dynamically link rdkafka to the system's version of librdkafka. Example:

[dependencies]
rdkafka = { version = "0.23", features = ["dynamic-linking"] }

For a full listing of features, consult the rdkafka-sys crate's documentation. All of rdkafka-sys features are re-exported as rdkafka features.

Asynchronous runtimes

Some features of the StreamConsumer and FutureProducer depend on Tokio, which can be a heavyweight dependency for users who only intend to use the low-level consumers and producers. The Tokio integration is enabled by default, but can be disabled by turning off default features:

[dependencies]
rdkafka = { version = "0.23", default-features = false }

If you would like to use an asynchronous runtime besides Tokio, you can integrate it with rust-rdkafka by providing a shim that implements the AsyncRuntime trait. See the smol runtime example for an example integration with smol.

Examples

You can find examples in the examples folder. To run them:

cargo run --example <example_name> -- <example_args>

Debugging

rust-rdkafka uses the log and env_logger crates to handle logging. Logging can be enabled using the RUST_LOG environment variable, for example:

RUST_LOG="librdkafka=trace,rdkafka::client=debug" cargo test

This will configure the logging level of librdkafka to trace, and the level of the client module of the Rust client to debug. To actually receive logs from librdkafka, you also have to set the debug option in the producer or consumer configuration (see librdkafka configuration).

To enable debugging in your project, make sure you initialize the logger with env_logger::init(), or the equivalent for any log-compatible logging framework.

rdkafka-sys

See rdkafka-sys.

Contributors

Thanks to:

Alternatives

  • kafka-rust: a pure Rust implementation of the Kafka client.

Comments

  • RDkafka dropping  messages
    RDkafka dropping messages

    May 1, 2020

    Hi,

    I have been hitting my head against the wall for a while on this one.

    Essentially, I'm producing 64 messages in a test (confirmed by going directly to kafka using kafkacat), but my consumer only sees 7-16 messages out of the 64, on average, per test.

    I'm not exactly sure that is wrong. Here is the consumer:

    pub async fn txn_consumer() {
      use std::env;
    
      use common::postgres::{PostgresPool, connection_pool};
      use futures::{
        compat::Stream01CompatExt,
        stream::StreamExt,
      };
    
      use rdkafka::{
          config::{ClientConfig, RDKafkaLogLevel},
          consumer::{Consumer, CommitMode, DefaultConsumerContext, MessageStream, StreamConsumer},
          error::KafkaResult,
          message::{BorrowedMessage, Message as RdMessage},
      };
    
      use prost::Message;
    
      let topic = #topic_name;
      let group = #group_name;
      let pool = connection_pool();
      println!("Listening to topic: {:}, group: {:}", topic, group);
    
      let consumer_url = env::var("KAFKA_CONSUMER").expect("KAFKA_CONSUMER must be set");
      let consumer:StreamConsumer = ClientConfig::new()
        .set("group.id", group)
        .set("bootstrap.servers", &consumer_url)
        .set("enable.partition.eof", "false")
        .set("session.timeout.ms", "6000")
        .set("enable.auto.commit", "true")
        .set("enable.auto.offset.store", "false")
        //.set("enable.auto.commit", "true")
        .set_log_level(RDKafkaLogLevel::Debug)
        .create()
        .expect("Consumer creation failed");
    
      consumer
        .subscribe(&[topic])
        .expect("Can't subscribe to specified topic");
    
      let mut counter = 0;
    
      while let Some(buf) = consumer.start().compat().next().await {
    
        println!("{:}: received {:}", counter, topic);
        counter += 1;
    
        match buf {
          Ok(Ok(borrowed)) => {
            let msg = borrowed.detach();
    
            let result = match msg.payload_view::<[u8]>() {
              None => {
                // move the needle forward for now
                println!("No idea");
                let _ = consumer.store_offset(&borrowed);
                continue;
              },
              Some(Err(e)) => {
                // move the needle forward for now
                println!("Error: ({:}, {:})\n{:#?}", topic, group, e);
                let _ = consumer.store_offset(&borrowed);
                continue;
              },
              Some(Ok(payload)) => match Txn::decode(payload) {
                Ok(val) => val,
                Err(e) => {
                  println!("Could not decode payload: {:#?}", e);
                  // move the needle forward for now
                  let _ = consumer.store_offset(&borrowed);
                  continue;
                },
              },
            };
    
            let conn = match pool.get() {
              Ok(val) => val,
              Err(e) => {
                println!("Could not get a database connection: {:#?}", e);
                // do not move the needle forward for now
                // we might want to retry this later
                continue;
              },
            };
    
            let get_result = /* async function goes here */;
    
            match get_result().await {
              Ok(_) => {
                let _ = consumer.store_offset(&borrowed);
              },
              Err(error) => println!("Error: ({:}, {:})\n{:#?}", topic, group, error),
            };
    
          }
          _ => {}
        }
      }
    }
    
    Reply
  • New release
    New release

    May 13, 2020

    Has been a while since the last release (January).

    Reply
  • Reparallelize CI tests
    Reparallelize CI tests

    May 21, 2020

    CI was flaky when running multiple tests in parallel. Re-enable to start to track down the bug.

    Reply
  • Cross-Compile dynamic linked issues
    Cross-Compile dynamic linked issues

    Jun 7, 2020

    Hello @fede1024,

    I need some or guidance, I'm currently trying to compile/build targets "x86_64-unknown-linux-musl" cargo build --target x86_64-unknown-linux-musl --release --features vendored and just for clairification the normal build without any problem, and I have the librdkafka installed.

    The error keeps point to something I would say relatively with the dynamically linked issues: `_error: failed to run custom build command for rdkafka-sys v1.3.1

    Caused by: process didn't exit successfully: /home/w/rust/rust-grpc-tonic/target/release/build/rdkafka-sys-b8f266a3d2a85c7e/build-script-build(exit code: 1) --- stderr librdkafka will be linked dynamically librdkafka 1.3.1 cannot be found on the system Dynamic linking failed. Exiting._`

    I have tried this couple day now, could you kindly point me to some right direction. And some of the suggestions I have tried also in the issues list, which still not working.

    Thank you in advance for the help here. Appreciated.

    Reply
  •  librdkafka when to release version 1.4.2
    librdkafka when to release version 1.4.2

    Jun 8, 2020

    A problem encountered in version 1.3.1,So I want to upgrade and try to solved it.

    Error: Disconnected (after XYZms in state UP)

    Reply
  • Add Offset::OffsetTail
    Add Offset::OffsetTail

    Jun 9, 2020

    This allows specifying negative offsets reading from the end of a partition. It mirrors what the RD_KAFKA_OFFSET_TAIL macro does in the C lib.

    Unfortunately I wasn't able to fully run all tests because test_topics topic fails for me (also on master without my changes applied) with a timeout error despite me having a Kafka instance listening on localhost:9092.

    Let me know if you want me to change anything!

    Reply
  • 0.21.0 breaks build on Rust Docker image
    0.21.0 breaks build on Rust Docker image

    May 9, 2019

    Problem

    Using the official Rust docker image, I am unable to build version 0.21.0, whereas 0.20.0 worked without issue. The error I see is:

    --- stderr
    Building and linking librdkafka statically
    Running command: "./configure --disable-sasl --enable-ssl --disable-lz4" in dir: librdkafka
    Running command: "make -j 4 libs" in dir: librdkafka
    thread 'main' panicked at 'Unable to find libclang: "couldn\'t find any valid shared libraries matching: [\'libclang.so\', \'libclang-*.so\', \'libclang.so.*\'], set the `LIBCLANG_PATH` environment variable to a path where one of these files can be found (invalid: [])"', src/libcore/result.rs:997:5
    note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
    

    Solution

    I am able to build successfully after installing some additional dependencies in the image (copied from the Dockerfile in this repo): apt-get install -y llvm-3.9-dev libclang-3.9-dev clang-3.9

    Based on the description in d9321f2b1f132d554d2c5854803fcb2869c6495c, it looks like the new dependencies are intended. Perhaps the new dependencies could be mentioned in the changelog to help anyone with broken builds?

    Let me know if there is additional information I can provide.

    Thanks!

    Reply
  • Is this function safe to use and support futures for async processing ? Need simple Producer usage
    Is this function safe to use and support futures for async processing ? Need simple Producer usage

    Aug 21, 2019

    Hi,

    I need to use simple function to produce Kafka messages with a solid function, that process producing on demand when requested by users:

    //Send messages to Kafka
    fn produce(brokers: &str, topic_name: &str, messages: String) {
        let producer: FutureProducer = ClientConfig::new()
            .set("bootstrap.servers", brokers)
            .set("produce.offset.report", "true")
            .set("message.timeout.ms", "5000")
            .create()
            .expect("Producer creation error");
    
    
        producer.send(
            FutureRecord::to(topic_name)
                .payload(&format!("Message {}", messages))
                .key(&format!("Key {}", messages))
                .headers(OwnedHeaders::new()
                    .add("header_key", "header_value")),
            0
        );
    }
    

    Do this function is safe and performente ? support futures ?

    Thanks in advance.

    Reply
  • Compilation fails w/ rust-musl-builder
    Compilation fails w/ rust-musl-builder

    Jun 26, 2018

    When compiling a rust application using rdkafka on rust-musl-builder the build fails when linking:

    My guess is that the rdkafka lib is build with glibc and then fails w/ musl.

    Is there a way to specify what to build it against the same way that dynamic linking can be specified?

    error: linking with `cc` failed: exit code: 1
      |
      = note: "cc" "-Wl,--as-needed" "-Wl,-z,noexecstack" "-nostdlib" "-Wl,--eh-frame-hdr" "-Wl,-(" "-m64" "/home/rust/.rustup/toolchains/1.26.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/crt1.o" "/home/rust/.rustup/toolchains/1.26.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/crti.o" "-L" "/home/rust/.rustup/toolchains/1.26.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/mshello-2054ff820a64761c.mshello0-66b6f0d31720f4bd44d6f265903cd36d.rs.rcgu.o" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/mshello-2054ff820a64761c.mshello1-66b6f0d31720f4bd44d6f265903cd36d.rs.rcgu.o" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/mshello-2054ff820a64761c.mshello2-66b6f0d31720f4bd44d6f265903cd36d.rs.rcgu.o" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/mshello-2054ff820a64761c.mshello3-66b6f0d31720f4bd44d6f265903cd36d.rs.rcgu.o" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/mshello-2054ff820a64761c.mshello4-66b6f0d31720f4bd44d6f265903cd36d.rs.rcgu.o" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/mshello-2054ff820a64761c.mshello5-66b6f0d31720f4bd44d6f265903cd36d.rs.rcgu.o" "-o" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/mshello-2054ff820a64761c" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/mshello-2054ff820a64761c.crate.allocator.rcgu.o" "-Wl,--gc-sections" "-no-pie" "-Wl,-z,relro,-z,now" "-Wl,-O1" "-nodefaultlibs" "-L" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps" "-L" "/home/rust/src/target/release/deps" "-L" "/home/rust/.cargo/registry/src/github.com-1ecc6299db9ec823/rdkafka-sys-0.11.4-0/librdkafka/src" "-L" "/home/rust/src/target/x86_64-unknown-linux-musl/release/build/libz-sys-9e0dfdf6ce77c312/out/lib" "-L" "/home/rust/.rustup/toolchains/1.26.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib" "-Wl,-Bstatic" "/home/rust/.rustup/toolchains/1.26.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/liballoc_jemalloc-f68d8e8245e507c6.rlib" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka-a45674fad582d6de.rlib" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/liblibz_sys-d28970724bb40e9f.rlib" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/liblibc-eef0eb2737b72223.rlib" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/libfutures-f1a25787e6cfbe44.rlib" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/libserde_json-fded606be9e377b3.rlib" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/libitoa-539d7e09fab6c05d.rlib" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/libdtoa-19c34238b45a98bc.rlib" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/libserde-30812860af6099f8.rlib" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/liblog-4de18e7534a813d8.rlib" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/liblog-9a6200d5626a5e8a.rlib" "/home/rust/src/target/x86_64-unknown-linux-musl/release/deps/libcfg_if-5f123802e50df72d.rlib" "/home/rust/.rustup/toolchains/1.26.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libstd-8d847bbe97fc9dc5.rlib" "/home/rust/.rustup/toolchains/1.26.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libpanic_unwind-008c42aad548e7dd.rlib" "/home/rust/.rustup/toolchains/1.26.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libunwind-0fe53371ec419e32.rlib" "/home/rust/.rustup/toolchains/1.26.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/liballoc_system-9dec1cbd51097ce1.rlib" "/home/rust/.rustup/toolchains/1.26.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/liblibc-6a4fb915dd86d140.rlib" "/home/rust/.rustup/toolchains/1.26.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/liballoc-7ebba6af2d3cc324.rlib" "/home/rust/.rustup/toolchains/1.26.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libstd_unicode-7a26f8b3cf380464.rlib" "/home/rust/.rustup/toolchains/1.26.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libcore-b0c2d164a9741309.rlib" "/home/rust/.rustup/toolchains/1.26.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/libcompiler_builtins-66d072e25a9acee3.rlib" "-static" "-Wl,-Bdynamic" "/home/rust/.rustup/toolchains/1.26.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-musl/lib/crtn.o" "-Wl,-)"
      = note: /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka.o): In function `fprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka.o):/usr/include/x86_64-linux-gnu/bits/stdio2.h:97: more undefined references to `__fprintf_chk' follow
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka.o): In function `vsnprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:77: undefined reference to `__vsnprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:77: undefined reference to `__vsnprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka.o): In function `fprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka.o): In function `snprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka.o): In function `fprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka.o):/usr/include/x86_64-linux-gnu/bits/stdio2.h:97: more undefined references to `__fprintf_chk' follow
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka.o): In function `snprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka.o):/usr/include/x86_64-linux-gnu/bits/stdio2.h:64: more undefined references to `__snprintf_chk' follow
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka.o): In function `fprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka.o):/usr/include/x86_64-linux-gnu/bits/stdio2.h:97: more undefined references to `__fprintf_chk' follow
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka.o): In function `snprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka.o): In function `vsnprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:77: undefined reference to `__vsnprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka.o): In function `snprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka.o): In function `fprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka.o):/usr/include/x86_64-linux-gnu/bits/stdio2.h:97: more undefined references to `__fprintf_chk' follow
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka.o): In function `snprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka.o): In function `fprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka.o): In function `rd_strndup':
              /home/rust/.cargo/registry/src/github.com-1ecc6299db9ec823/rdkafka-sys-0.11.4-0/librdkafka/src/rd.h:127: undefined reference to `__strndup'
              /home/rust/.cargo/registry/src/github.com-1ecc6299db9ec823/rdkafka-sys-0.11.4-0/librdkafka/src/rd.h:127: undefined reference to `__strndup'
              /home/rust/.cargo/registry/src/github.com-1ecc6299db9ec823/rdkafka-sys-0.11.4-0/librdkafka/src/rd.h:127: undefined reference to `__strndup'
              /home/rust/.cargo/registry/src/github.com-1ecc6299db9ec823/rdkafka-sys-0.11.4-0/librdkafka/src/rd.h:127: undefined reference to `__strndup'
              /home/rust/.cargo/registry/src/github.com-1ecc6299db9ec823/rdkafka-sys-0.11.4-0/librdkafka/src/rd.h:127: undefined reference to `__strndup'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka.o):/home/rust/.cargo/registry/src/github.com-1ecc6299db9ec823/rdkafka-sys-0.11.4-0/librdkafka/src/rd.h:127: more undefined references to `__strndup' follow
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka.o): In function `fprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka.o): In function `syslog':
              /usr/include/x86_64-linux-gnu/bits/syslog.h:31: undefined reference to `__syslog_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_broker.o): In function `snprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_broker.o): In function `fprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_broker.o): In function `snprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_broker.o): In function `fprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_broker.o): In function `snprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_broker.o): In function `vsnprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:77: undefined reference to `__vsnprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_broker.o): In function `snprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_broker.o): In function `fprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_broker.o):/usr/include/x86_64-linux-gnu/bits/stdio2.h:97: more undefined references to `__fprintf_chk' follow
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_broker.o): In function `snprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_broker.o): In function `fprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_broker.o):/usr/include/x86_64-linux-gnu/bits/stdio2.h:97: more undefined references to `__fprintf_chk' follow
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_broker.o): In function `snprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_msg.o): In function `fprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_msg.o):/usr/include/x86_64-linux-gnu/bits/stdio2.h:97: more undefined references to `__fprintf_chk' follow
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_conf.o): In function `snprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_conf.o):/usr/include/x86_64-linux-gnu/bits/stdio2.h:64: more undefined references to `__snprintf_chk' follow
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_conf.o): In function `fprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_conf.o):/usr/include/x86_64-linux-gnu/bits/stdio2.h:97: more undefined references to `__fprintf_chk' follow
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_conf.o): In function `snprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_conf.o): In function `fprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_offset.o): In function `fprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_offset.o):/usr/include/x86_64-linux-gnu/bits/stdio2.h:97: more undefined references to `__fprintf_chk' follow
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_offset.o): In function `snprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_offset.o): In function `fprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_offset.o):/usr/include/x86_64-linux-gnu/bits/stdio2.h:97: more undefined references to `__fprintf_chk' follow
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_offset.o): In function `snprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_transport.o): In function `snprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_transport.o):/usr/include/x86_64-linux-gnu/bits/stdio2.h:64: more undefined references to `__snprintf_chk' follow
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_transport.o): In function `poll':
              /usr/include/x86_64-linux-gnu/bits/poll2.h:41: undefined reference to `__poll_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_buf.o): In function `fprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_buf.o): In function `snprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_buf.o): In function `fprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_queue.o):/usr/include/x86_64-linux-gnu/bits/stdio2.h:97: more undefined references to `__fprintf_chk' follow
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_op.o): In function `vsnprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:77: undefined reference to `__vsnprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_op.o): In function `fprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_op.o):/usr/include/x86_64-linux-gnu/bits/stdio2.h:97: more undefined references to `__fprintf_chk' follow
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_request.o): In function `snprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_request.o): In function `fprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_request.o):/usr/include/x86_64-linux-gnu/bits/stdio2.h:97: more undefined references to `__fprintf_chk' follow
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_request.o): In function `snprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_request.o): In function `rd_strndup':
              /home/rust/.cargo/registry/src/github.com-1ecc6299db9ec823/rdkafka-sys-0.11.4-0/librdkafka/src/rd.h:127: undefined reference to `__strndup'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_request.o): In function `fprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_cgrp.o):/usr/include/x86_64-linux-gnu/bits/stdio2.h:97: more undefined references to `__fprintf_chk' follow
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_cgrp.o): In function `snprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_cgrp.o): In function `fprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_cgrp.o):/usr/include/x86_64-linux-gnu/bits/stdio2.h:97: more undefined references to `__fprintf_chk' follow
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_cgrp.o): In function `rd_strndup':
              /home/rust/.cargo/registry/src/github.com-1ecc6299db9ec823/rdkafka-sys-0.11.4-0/librdkafka/src/rd.h:127: undefined reference to `__strndup'
              /home/rust/.cargo/registry/src/github.com-1ecc6299db9ec823/rdkafka-sys-0.11.4-0/librdkafka/src/rd.h:127: undefined reference to `__strndup'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_cgrp.o): In function `fprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_pattern.o): In function `snprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_partition.o): In function `fprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_partition.o):/usr/include/x86_64-linux-gnu/bits/stdio2.h:97: more undefined references to `__fprintf_chk' follow
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_partition.o): In function `snprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_partition.o): In function `fprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_partition.o): In function `rd_strndup':
              /home/rust/.cargo/registry/src/github.com-1ecc6299db9ec823/rdkafka-sys-0.11.4-0/librdkafka/src/rd.h:127: undefined reference to `__strndup'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_partition.o): In function `fprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_partition.o):/usr/include/x86_64-linux-gnu/bits/stdio2.h:97: more undefined references to `__fprintf_chk' follow
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_partition.o): In function `snprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_assignor.o): In function `snprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdstring.o):/usr/include/x86_64-linux-gnu/bits/stdio2.h:64: more undefined references to `__snprintf_chk' follow
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_metadata.o): In function `fprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_metadata.o): In function `rd_strndup':
              /home/rust/.cargo/registry/src/github.com-1ecc6299db9ec823/rdkafka-sys-0.11.4-0/librdkafka/src/rd.h:127: undefined reference to `__strndup'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdbuf.o): In function `fprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdbuf.o):/usr/include/x86_64-linux-gnu/bits/stdio2.h:97: more undefined references to `__fprintf_chk' follow
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(snappy.o): In function `snprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rddl.o): In function `snprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rddl.o):/usr/include/x86_64-linux-gnu/bits/stdio2.h:64: more undefined references to `__snprintf_chk' follow
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(crc32c.o): In function `fprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(crc32c.o):/usr/include/x86_64-linux-gnu/bits/stdio2.h:97: more undefined references to `__fprintf_chk' follow
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdaddr.o): In function `snprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdaddr.o): In function `strncpy':
              /usr/include/x86_64-linux-gnu/bits/string3.h:126: undefined reference to `__strncpy_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdaddr.o): In function `strcpy':
              /usr/include/x86_64-linux-gnu/bits/string3.h:110: undefined reference to `__memcpy_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdlist.o): In function `printf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:104: undefined reference to `__printf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:104: undefined reference to `__printf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_metadata_cache.o): In function `fprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_sasl.o): In function `snprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_msgset_reader.o): In function `fprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_msgset_reader.o):/usr/include/x86_64-linux-gnu/bits/stdio2.h:97: more undefined references to `__fprintf_chk' follow
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_msgset_reader.o): In function `snprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdvarint.o): In function `fprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdvarint.o):/usr/include/x86_64-linux-gnu/bits/stdio2.h:97: more undefined references to `__fprintf_chk' follow
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdkafka_plugin.o): In function `snprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdlog.o): In function `fprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdlog.o): In function `snprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:64: undefined reference to `__snprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdlog.o): In function `fprintf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:97: undefined reference to `__fprintf_chk'
              /home/rust/src/target/x86_64-unknown-linux-musl/release/deps/librdkafka_sys-1dabf5d6227192b6.rlib(rdlog.o): In function `printf':
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:104: undefined reference to `__printf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:104: undefined reference to `__printf_chk'
              /usr/include/x86_64-linux-gnu/bits/stdio2.h:104: undefined reference to `__printf_chk'
    
    Reply
  • Running StreamConsumer with tokio's executor
    Running StreamConsumer with tokio's executor

    Jun 3, 2018

    Currently how StreamConsumer stores the JoinHandle in a Cell prevents us of using the consumer in tokio's thread_pool executor, which expects the consumer to implement Send. In other hand using the current_thread executor doesn't comply with missing static lifetimes somewhere in the consumer code I haven't been able to locate yet.

    Is there plans to support the new executors, especially when the development of tokio-core and futures-cpupool seems to be stalled in favor of tokio?

    Reply
  • Directly return a DeliveryFuture from FutureProducer::send_copy
    Directly return a DeliveryFuture from FutureProducer::send_copy

    Jun 29, 2017

    Returning a Result made it very difficult to use send_copy() in a chain of futures, and, since a Future is just an asynchronous Result, this commit changes the DeliveryFuture to yield a KafkaError on failure. Previously, the DeliveryFuture would only yield Canceled, and, to continue supporting this, a new variant of KafkaError was added, FutureCanceled.

    This makes the FutureProducer API considerably more ergonomic. For example, in a function which returns a BoxFuture<(), ::std::io::Error>, this is what was necessary before:

    return match kafka_producer.send_copy::<_, ()>(
        "test", None, Some(&*format!("msg on {}", topic)), None, None) {
        Ok(report) => {
            report
                .map_err(|_| io::Error::new(io::ErrorKind::Other,
                                            "failed to send kafka message"))
                .and_then(|_| future::ok(()))
                .boxed()
        },
    
        Err(_) => {
            future::err(io::Error::new(
                    io::ErrorKind::InvalidData, "failed to send kafka message"))
                .boxed()
        }
    }
    

    This becomes even more complicated with the fact that match arms must return the same type and the types of chained Futures can easily get very complex.

    Compare that to the code one can write with this PR applied:

    return kafka_producer
        .send_copy::<_, ()>("test", None, Some(&*format!("msg on {}", topic)), None, None)
        .map_err(|_| io::Error::new(io::ErrorKind::Other,
                                    "failed to send kafka message"))
        .and_then(|_| future::ok(()))
        .boxed();
    

    Much less verbose, and, more importantly, it flows naturally as part of a chain of futures.

    Full disclosure: there is a compilation warning for the unused Result returned by tx.send() in two places, but this warning was present before the change and I'm also not sure what the correct way to handle an error arising in such a situation actually is.

    Reply
  • First time user
    First time user

    Aug 22, 2017

    Hello! I've recently tried to use rust-rdkafka, and thought I would share some experiences. I think there may be some "issues" here, but there could also just be some helpful information about what people (person) stumble over.

    First off, thank you! This is in support of a timely dataflow Kafka adapter, which I wouldn't want to try and write without a crate. :)

    I wanted to try sending a message through kafka and back, so I wrote what seemed like a pretty simple program, following your examples but using the Base* variants of producer and consumer.

    fn round_trip(brokers: &str, topic_name: &str) -> Result<(), rdkafka::error::KafkaError> {
    
        let mut topic_config = TopicConfig::new();
        topic_config
            .set("produce.offset.report", "true")
            .finalize();
    
        let mut client_config = ClientConfig::new();
        client_config
            .set("group.id", "example")
            .set("bootstrap.servers", brokers)
            .set_default_topic_config(topic_config);
    
        let producer: BaseProducer<_> = try!(client_config.create());
        let consumer: BaseConsumer<_> = try!(client_config.create());
    
        try!(consumer.subscribe(&[topic_name]));
    
        let text = format!("hi there at {:?}", ::std::time::Instant::now());
        try!(producer.send_copy::<str,()>(topic_name, None, Some(text.as_str()), None,  None, None));
    
        for _ in 0 .. 100 {
    
            producer.poll(1);
    
            if let Some(result) = try!(consumer.poll(1)) {
                println!("{:?}:\t{:?}", ::std::time::Instant::now(), result.payload_view::<str>());
                try!(consumer.commit(None, rdkafka::consumer::CommitMode::Sync));
            }
            else {
                println!("{:?}:\tgot nothing", ::std::time::Instant::now());
            }
        }
    
        Ok(())
    }
    

    The behavior was a bit surprising, and it was tricky to find documentation to explain if this was expected.

    1. This seems to run to completion just fine, printing "got nothing" 100 times and returning Ok(()) when neither ZK nor Kafka are up and running. Where is the moment in the code it is supposed to complain if they are not installed or up and running? (when they are up and running, it does seem to connect and occasionally round-trip data; mysterious things happen).

    2. I'm not a Kafka expert, so I was surprised to see things like "nothing more to read now" surface as an error (PartitionEOF) rather than the absence of a message. I imagine that is just what rdkafka thinks is best and returns to you, so you want to surface it up?

    3. The BaseProducer documentation speaks of a BaseProducerTopic that I should use, but I couldn't find this. Is it just stale documentation?

    4. I was hoping to have a non-blocking poll, but I'm not sure if this is what an argument of 0 would mean. Sometimes 0 means "block". Is that something you could clarify in the documentation? It could be that rdkafka does what it does and won't let us fcntl the socket to non-blocking mode, or something like that, but would be good to know!

    Maybe as a meta comment, if BaseProducer and BaseConsumer are meant to be a very thin layer on top of rdkafka, with the caveat that you had best understand the details of rdkafka (which I don't yet), you could say that in the documentation (I'm still not sure). I'm sure it is still useful, but would keep me from scratching / banging my head.

    I'm going to try out the "higher-level" stuff next, but I'd have preferred to just have a non-blocking polling interface. If that should work out, and I just need more Kafka protocol expertise, I can push on it more and come back with comments on that.

    Thanks again!

    Reply