Rust in R beyond vectors

I have blogged about using Rust in R twice before. Once about a phone number parser called dialrs and once about a datetime parser called timeless. There is a actually third post about the archival of timeless on CRAN due to some issues with the Rust policies. Both packages are really rudimentary in terms of the included Rust code. After all, I am still a beginner and quite happy if I get anything to run at all. So this should also serve as a disclaimer for this post. Take any Rust code you see here with a grain of salt. Just because it works, doesn’t mean it is the best way (or even correct way). If you have suggestions or spotted big errors, please leave a comment.

Returning vectors in Rust functions

What I meant with “rudimentary” above is that both packages only wrap one or two crates and the most complex return values are vectors of strings, ‘Vec’ in Rust terminology.

Here is for example a function that parses international phone numbers from dialrs

fn parse_phone_rs_international(phone: Vec<String>, country: &str) -> Vec<String> {
    let region = phonenumber::country::Id::from_str(country).ok();
    phone
        .into_iter()
        .map(
            |input| match phonenumber::parse(region, strip_hyphens(&input)) {
                Ok(number) => number.format().mode(Mode::International).to_string(),
                Err(_e) => String::new(),
            },
        )
        .collect()
}

and here is a function from timeless which tries to parse datetime from a string

fn parse_guess_rs(times: Vec<String>) -> Vec<String> {
    times
        .iter()
        .map(|input| match input.parse::<DateTimeUtc>() {
            Ok(value) => value
                .0
                .format("%Y-%m-%d %H:%M:%S")
                .to_string(),
            Err(_e) => "not found".to_string(),
        })
        .collect()
}

The input and output in both cases are just character vectors. rextendr can deal with these return type (vectors) without issues. But what if we want to move beyond simple vectors?

(Trying to) Return matrices in Rust functions

With my limited experience, I thought that something like Vec<Vec<String>> could be the right structure to return something “2 dimensional”. While it actually does represent a 2D array-like structure, it does not enforce a strict rectangular shape. So each inner vector can have different lengths. So somewhat comparable to a List in R where all entries have to have the same type. So lets try to assemble a matrix-like list thing. Before showing the Rust code, here is the equivalent in R.

create_matrix <- function(n, m) {
  lapply(seq_len(n), function(r) (1:m) + (r - 1) * m)
}
create_matrix(n = 3, m = 4)

[[1]]
[1] 1 2 3 4

[[2]]
[1] 5 6 7 8

[[3]]
[1]  9 10 11 12

Now this is what it would look like in Rust.

rextendr::rust_function(
  "fn create_matrix(n: usize, m: usize) -> Vec<Vec<usize>> {
    (0..n).map(|i| {
        (0..m).map(|j| i * m + j + 1).collect()
    }).collect()
  }"
)

Error in `invoke_cargo()`:
! Rust code could not be compiled successfully. Aborting.
✖ error[E0277]: the trait bound `extendr_api::Robj: From<Vec<Vec<usize>>>` is not satisfied
 --> src/lib.rs:2:1
  |
2 | #[extendr]
  | ^^^^^^^^^^ the trait `From<Vec<Vec<usize>>>` is not implemented for `extendr_api::Robj`
  |
  = help: the following other types implement trait `From<T>`:
            `extendr_api::Robj` implements `From<&Altrep>`
            `extendr_api::Robj` implements `From<&Primitive>`
            `extendr_api::Robj` implements `From<&Vec<T>>`
            `extendr_api::Robj` implements `From<&[T; N]>`
            `extendr_api::Robj` implements `From<&[T]>`
            `extendr_api::Robj` implements `From<&extendr_api::Complexes>`
            `extendr_api::Robj` implements `From<&extendr_api::Doubles>`
            `extendr_api::Robj` implements `From<&extendr_api::Environment>`
          and 63 others
  = note: this error originates in the attribute macro `extendr` (in Nightly builds, run with -Z macro-backtrace for more info)

Ok so this doesnt work. There is a lot going on in that error message but the important part is

the trait `From<Vec<Vec<usize>>>` is not implemented for `extendr_api::Robj`

What it tries to tell us is that we can’t have <Vec<Vec<usize>> as a return value because it is not supported by the API that connects R and Rust.

Let us try it with the nalgebra crate which allows us to actually build matrices.

code <- r"(
  use nalgebra::DMatrix;

  #[extendr]
  fn create_matrix(n: usize, m: usize) -> DMatrix<usize> {
      DMatrix::from_iterator(n, m, (1..=n * m))
  }
)"


rextendr::rust_source(
  code = code,
  dependencies = list(`nalgebra` = "0.33")
)

Error in `invoke_cargo()`:
! Rust code could not be compiled successfully. Aborting.
✖ error[E0277]: the trait bound `Matrix<usize, Dyn, Dyn, VecStorage<usize, Dyn, Dyn>>: ToVectorValue` is not satisfied
 --> src/lib.rs:5:3
  |
5 |   #[extendr]
  |   ^^^^^^^^^^ the trait `ToVectorValue` is not implemented for `Matrix<usize, Dyn, Dyn, VecStorage<usize, Dyn, Dyn>>`
  |
  = help: the following other types implement trait `ToVectorValue`:
            &&str
            &(f64, f64)
            &Rbool
            &Rcplx
            &Rfloat
            &Rint
            &String
            &bool
          and 45 others
  = note: required for `extendr_api::Robj` to implement `From<Matrix<usize, Dyn, Dyn, VecStorage<usize, Dyn, Dyn>>>`
  = note: this error originates in the attribute macro `extendr` (in Nightly builds, run with -Z macro-backtrace for more info)

This looks pretty much like the same error. Something is not implemented that we need to transfer the Matrix to R. Maybe it is time to RTFM to understand what is going on.

The R/Rust interface

Our situation is actually well described in the (work in progress) user guide of the crate extendr-api in the section about macros. In order for an item to be returned to R from a function written in Rust, the return value must be able to be turned into an R object. This makes a lot of sense. Obviously, if R gets something that it doesn’t understand, it cannot deal with it. But there is a way to MAKE R understand, even if it does not understand the original result.

The ToVectorValue trait is what is used to convert Rust items into R objects. We have seen this pop up in the last error message we got from the nalgebra crate. The trait is implemented on a number of standard Rust types such as i32, f64, usize, String and many more. So if any of these are returned, R knows what to do.

You might now ask yourself: “What the hell is a trait?”. It has something to do with types (something we do not care much about in R). Say you want to write a simple function to sum up two values:

fn add(x: i32, y: i32) -> i32 {
    x + y
}

This is fine, but as soon as you give this function something else than a i32, it errors. So if we want add to work for other types, we would have to create a function for every single number type there is (u32, f64 etc.). That would be quite cumbersome. How can we abstract this? The answer is traits.

use std::ops::Add;

fn add<T: Add>(x: T, y: T) -> T {
    x + y
}

The trait Add, which looks like this

trait Add<Rhs = Self> {
    type Output;

    fn add(self, rhs: Rhs) -> Self::Output;
}

implements addition for a large variety of types (see here). But even if the type you need is not supported, you can implement it yourself with an impl block.

Implementing addition for a new struct

Say we have defined our own structure, a point, and we want to define addition of points.

struct Point {
    x: i32,
    y: i32,
}

impl Add for Point {
    type Output = Self;

    fn add(self, other: Self) -> Self {
        Self {
            x: self.x + other.x,
            y: self.y + other.y,
        }
    }
}

In this example we define a structure that is a point with two i32 coordinates and the impl block tells Rust how to use the Add trait for this structure. But as you might realize, we have again only defined the structure for one specific type, i32. We can extend it in a very similar way as above.

struct Point<T> {
    x: T,
    y: T,
}

impl<T: Add<Output = T>> Add for Point<T> {
    type Output = Self;

    fn add(self, other: Self) -> Self::Output {
        Self {
            x: self.x + other.x,
            y: self.y + other.y,
        }
    }
}

In pure Rust, you can now do

let p1 = Point { x: 1, y: 0 };
let p2 = Point { x: 2, y: 3 };
let p3 = p1 + p2;

Bring it to R

Ok so that works in Rust, but how can we get this in R now? My hope was, that I can just wrap the whole thing with rextendr and let it do its magic.

code <- r"(
use std::ops::Add;

#[derive(Debug, Copy, Clone, PartialEq)]
struct Point {
    x: i32,
    y: i32,
}

impl Add for Point {
    type Output = Self;

    fn add(self, other: Self) -> Self {
        Self {
            x: self.x + other.x,
            y: self.y + other.y,
        }
    }
}
)"

rextendr::rust_source(
  code = code
)

It actually does compile without an error, but there is nothing exported in R that we can use. After some testing, what we have to do is expose Point struct via an impl block and also expose the add function for points.

code <- r"(
use std::ops::Add;

#[derive(Debug, Copy, Clone, PartialEq)]
struct Point {
    x: i32,
    y: i32,
}

impl Add for Point {
    type Output = Self;

    fn add(self, other: Self) -> Self {
        Self {
            x: self.x + other.x,
            y: self.y + other.y,
        }
    }
}

#[extendr]
impl Point {
    fn new(x: i32, y: i32) -> Self {
        Self { x, y }
    }

    fn to_vec(&self) -> Vec<i32> {
        vec![self.x, self.y]
    }
}

#[extendr]
fn add_points(p1: Point, p2: Point) -> Point {
    p1 + p2
}
)"

rextendr::rust_source(
  code = code
)

Error in `invoke_cargo()`:
! Rust code could not be compiled successfully. Aborting.
✖ error[E0277]: the trait bound `Point: extendr_api::TryFrom<extendr_api::Robj>` is not satisfied
  --> src/lib.rs:33:1
   |
33 | #[extendr]
   | ^^^^^^^^^^ the trait `From<extendr_api::Robj>` is not implemented for `Point`
   |
   = note: required for `extendr_api::Robj` to implement `Into<Point>`
   = note: required for `Point` to implement `extendr_api::TryFrom<extendr_api::Robj>`
   = note: required for `extendr_api::Robj` to implement `extendr_api::TryInto<Point>`
   = note: this error originates in the attribute macro `extendr` (in Nightly builds, run with -Z macro-backtrace for more info)

Unfortunately, this does not work yet. The error says

the trait `From<extendr_api::Robj>` is not implemented for `Point`

Up to now, we have done everything so that R understands when it gets an object of type Point. But given that the function add_points() also has Point as input, we now need to make Rust understand when it gets a Point from R. This is done with a TryFrom block.

#
code <- r"(
use std::ops::Add;
use extendr_api::*;
#[derive(Debug, Copy, Clone, PartialEq)]
struct Point {
    x: i32,
    y: i32,
}

impl Add for Point {
    type Output = Self;

    fn add(self, other: Self) -> Self {
        Self {
            x: self.x + other.x,
            y: self.y + other.y,
        }
    }
}

impl TryFrom<Robj> for Point {
    type Error = Error;

    fn try_from(robj: Robj) -> Result<Self> {
        let vec: Vec<i32> = robj.as_integer_vector()
            .ok_or_else(|| Error::Other("Expected an integer vector of length 2".into()))?;
        if vec.len() != 2 {
            return Err(Error::Other("Point requires exactly two integers".into()));
        }
        Ok(Point { x: vec[0], y: vec[1] })
    }
}

#[extendr]
impl Point {
    fn new(x: i32, y: i32) -> Self {
        Self { x, y }
    }

    fn to_vec(&self) -> Vec<i32> {
        vec![self.x, self.y]
    }
}

#[extendr]
fn add_points(p1: Point, p2: Point) -> Point {
    p1 + p2
}
)"

rextendr::rust_source(
  code = code
)

Seems to be fine so lets try and use it.

p1 <- Point$new(3L, 4L)
p2 <- Point$new(1L, 2L)
p1$to_vec()

[1] 3 4

p3 <- add_points(p1, p2)

Error in add_points(p1, p2): Expected an integer vector of length 2

Intuitively, I would have thought that this should work. But ultimately, I think it is clear why it doesnt. The TryFrom expects an integer vector of length two comming from R, not a Point object. So what will work is

p3 <- add_points(c(3L, 4L), c(1L, 2L))
p3$to_vec()

[1] 4 6

Great, we managed with much trial-and-error to get a proper implementation.

For completeness, here is another version of the code that is a bit shorter and lets us do the addition with actual point type objects.

code <- r"(
use extendr_api::prelude::*;
use std::ops::Add;

#[derive(Debug, Clone, Copy, PartialEq)]
pub struct Point {
    x: i32,
    y: i32,
}

#[extendr]
impl Point {
    fn new(x: i32, y: i32) -> Self {
        Point { x, y }
    }

    fn add(&self, other: &Point) -> Point {
        *self + *other
    }

    fn to_vec(&self) -> Vec<i32> {
        vec![self.x, self.y]
    }
}

impl Add for Point {
    type Output = Point;

    fn add(self, other: Point) -> Point {
        Point {
            x: self.x + other.x,
            y: self.y + other.y,
        }
    }
}
)"

rextendr::rust_source(
  code = code
)

We do have the same addition trait, but now we define the actual function within the Point block. That way, we can use it as follows

p1 <- Point$new(3L, 4L)
p2 <- Point$new(1L, 2L)
p1$add(p2)$to_vec()

[1] 4 6

Conclusion

We might not have implemented anything useful here, but I hope the rudimentary example tought as a few things.

• Batteries included: If all you do is moving vectors with standard type between R and Rust, you should be fine in most cases

• Interface between R and Rust: We need impl blocks and TryFrom to make R and Rust understand each other better for types that are not so standard.

• Read the compiler messages: I skipped some intermediary steps in my exploration, because there was much more trial-and-error than I showed. Many things can be fixed simply by reading the error messages provided by the compiler. In many cases, it let’s you know what the fix is.

Addendum

Shortly after publishing this post, I found that extendr-api actually supports matrices in some way (link). I try to explore this in a later post.