• Skip to primary navigation
  • Skip to content
  • Skip to primary sidebar

Red Sift Blog

Red Sift Blog
  • redsift.com
  • Featured
  • Who are we?
  • Get in touch
You are here: Home / Labs / Putting Rust in the kernel with eBPF

Putting Rust in the kernel with eBPF

by Peter Parkanyi
November 7, 2019August 18, 2022Filed under:
  • Labs
  • Work at Red Sift

It was a little more than a year ago that I kicked off development on what became InGRAINd. A lot has happened in that year, so, let’s take a step back before we kick off with the story.

At Red Sift, we’re building our own data analytics platform for cybersecurity. We built our existing products, OnDMARC and OnINBOX, on top of this platform.

During the product development process monitoring our pipelines proved challenging, and we wanted more visibility into our containers. After a short period of exploration, we found that eBPF would address most of the pain points and dark spots we were encountering.

There was one catch: no eBPF tooling would help us deploy and maintain new probes within our small, but focused ops team. BCC, while great for tinkering, requires significant effort to roll out to production. It also makes it difficult to integrate our toolkit into our usual CI/CD deployment models.

Faced with this dilemma, we decided the only option was for us to write our own Rust-based agent that integrated well with our testing and deployment strategies. I had the opportunity to share our experience at the Linux Plumbers Conference in Vancouver last year, and talk about InGRAINd in depth. Doing away with BCC means that we can deploy a 15MB-ish binary, built and tested by our CI, to servers instead of several 100 megabytes of dependencies.

This past year, we spent a lot of effort making eBPF more accessible while we expanded our use of the InGRAINd agent in our fleet. As a culmination of this process, I am extremely happy to announce the early versions of our agent that can run Rust in the kernel, thanks mostly to the work of the amazing Alessandro Decina!

This is how parsing a network buffer works in InGRAINd now. The code below runs on both my NextCloud Raspberry Pi at home and our AMD64 servers in the data center:

#[map("events")]
static mut events: PerfMap<Event> = PerfMap::new();

#[xdp("dns_queries")]
pub extern "C" fn probe(ctx: XdpContext) -> XdpAction {
    let (ip, transport) = match (ctx.ip(), ctx.transport()) {
        (Some(i), Some(t)) => (unsafe { *i }, t),
        _ => return XdpAction::Pass
    };
    let data = match ctx.data() {
        Some(data) => data,
        None => return XdpAction::Pass
    };

    let header = match data.slice(12) {
        Some(s) => s,
        None => return XdpAction::Pass
    };

...

The Journey to Rust

But the process of getting to this stage is just as exciting as the results. We found that mixing C and Rust made it difficult to share data between the kernel and userspace for more complex cases.

Moreover, maintaining our toolchain to compile and work well on newer Linux versions turned into an uphill battle. We needed more flexibility in the toolchain itself, and more useful tools while building probes.

Third, the mix of two languages meant that we experienced a barrier of entry that made it tedious to extract new metrics and tinker with existing ones. You not only need to know how to write code in eBPF, a highly specialized environment, you also had to have a working knowledge of C and Rust, without much documentation or support.

Since it seems like the idea of putting Rust into the kernel is seeing a warm reception, we decided to address these issues by doing just that. There’s definitely a lot more to refine on our interfaces, but we have already made a lot of progress. I hope everyone will agree that having RustDoc eBPF is a lot more approachable than reading the source for Linux.

To get comparisons out of the way, this is what the same probe above looks like in C.

struct bpf_map_def SEC("maps/dns_queries") dns_queries = {
    .type = BPF_MAP_TYPE_PERF_EVENT_ARRAY,
    .key_size = sizeof(u32),
    .value_size = sizeof(u32),
    .max_entries = 1024,
    .pinning = 0,
    .namespace = "",
};

__inline_fn
s8 parse_dns_packet(struct xdp_md *ctx, void *buffer, void *data_end, struct _data_dns_query *query) {
  struct ethhdr *eth = (struct ethhdr *) buffer;
  struct iphdr *ip;
  struct udphdr *udp;
  void *dns;

  ip = (struct iphdr *) (buffer + sizeof(struct ethhdr));
  udp = (struct udphdr *) (ip + 1);

  /* dns header */
  if (udp + 1 > data_end) {
    return -7;
  }

  if (!(eth->h_proto == bpf_htons(ETH_P_IP)
        && ip->protocol == IPPROTO_UDP
        )) {
    return -6;
  }

  dns = buffer + sizeof(struct ethhdr)
    + sizeof(struct udphdr)
    + (ip->ihl * 4);
  if (dns + 12 > data_end) {
    return -5;
  }

...

Overall, we found that the C compiler is not nearly as friendly as the Rust compiler when it’s reporting errors.

The high-level abstractions for dealing with low-level kernel structures make all the difference! On top of that, the toolchain allows us to target any kernel version with minimal maintenance costs. We can unit test eBPF code with ease!

Next Steps

In the future, we are looking at integrating specific checks into the build process so you can verify your bytecode before it gets rejected by the kernel’s verifier. We are also expanding the number of idiomatic wrappers around eBPF constructs to cover more program types.

All the small improvements do add up. As the mission of Red Sift is to democratize cybersecurity, we are more than happy to contribute to the eBPF ecosystem to make this exciting technology more approachable.

Since the power of eBPF is in how adaptable it is, we are keen on helping ops teams feel empowered to write and deploy their own monitoring modules more easily, and deploy with less risk and more automation than other approaches.

We’re so excited about this work that we’re bringing a few members of our team to the Barcelona RustFest this weekend. If you’re attending and would like to play around with Raspberry Pis and eBPF we have just the workshop for you!

If you can’t make it to RustFest, make sure to check out the InGRAINd repo, and have a play. You can run InGRAINd, as we do it in production, and save the collected data to your favorite StatsD compatible service, or S3 buckets. We would love to hear what you think!

Get in touch

Share this:

  • Click to share on Twitter (Opens in new window)
  • Click to share on LinkedIn (Opens in new window)

Related

Tagged:
  • Development
  • Linux
  • rust
  • Security

Post navigation

Previous Post Byline: Today’s Conveyancer – Increasing Risk and Cyber Threat in Conveyancing Sector
Next Post Research: Threatpost – U.S. Universities Get Failing Grades for DMARC Adoption

Primary Sidebar

Subscribe to our blog and be the first to get updates!

Categories

  • AI
  • BEC
  • BIMI
  • Brand Protection
  • Coronavirus
  • Cybersecurity
  • Deliverability
  • DMARC
  • DORA
  • Email
  • Finance
  • Labs
  • News
  • OnINBOX
  • Partner Program
  • Red Sift Tools
  • Work at Red Sift
  • February 2023
  • January 2023
  • December 2022
  • November 2022
  • October 2022
  • September 2022
  • August 2022
  • July 2022
  • June 2022
  • May 2022
  • April 2022
  • March 2022
  • February 2022
  • January 2022
  • December 2021
  • November 2021
  • October 2021
  • September 2021
  • August 2021
  • July 2021
  • June 2021
  • May 2021
  • April 2021
  • March 2021
  • February 2021
  • January 2021
  • December 2020
  • November 2020
  • October 2020
  • September 2020
  • July 2020
  • June 2020
  • May 2020
  • April 2020
  • March 2020
  • February 2020
  • January 2020
  • December 2019
  • November 2019
  • October 2019
  • September 2019
  • August 2019
  • July 2019
  • June 2019
  • May 2019
  • April 2019
  • March 2019
  • February 2019
  • January 2019
  • December 2018
  • November 2018
  • October 2018
  • September 2018
  • August 2018
  • July 2018
  • June 2018
  • May 2018
  • April 2018
  • March 2018
  • February 2018
  • January 2018
  • December 2017
  • November 2017
  • October 2017
  • September 2017
  • July 2017
  • June 2017
  • May 2017
  • April 2017
  • March 2017
  • October 2016

Copyright © 2023 · Milan Pro on Genesis Framework · WordPress · Log in