This started out as a “hey, I wonder…” sort of thing, but as usual, they tend to end up as interesting voyages into the deepest depths of code, so I thought I’d write it up and share. Shoutout to [@coolbutuseless](https://twitter.com/coolbutuseless) for proving that a little curiosity can go a long way and inspiring me to keep digging into interesting topics.
This is what you get if you “glue” “strings”. Photo: https://craftwhack.com/cool-craft-string-easter-eggs/
This post came across my feed last week, referring to the roperators package on CRAN. In that post, the author introduces an infix operator from that package which ‘adds’ (concatenates/pastes) strings
"using infix (%) operators " %+% "R can do simple string addition"## [1] "using infix (%) operators R can do simple string addition"This might be familiar if you use python
"python " + "adds " + "strings"or javascript
"javascript " + "also adds " + "strings"
## javascript also adds stringsor perhaps even go
package main
import "fmt"
func main() {
fmt.Println("go " + "even adds " + "strings")
}or Julia
"julia can " * "add strings"but this is not something natively available in R
"this doesn't" + "work"## Error in "this doesn't" + "work": non-numeric argument to binary operator
Could we make it work, though? That got me wondering. My first guess was to just create a new + function which does allow for this. The normal addition operator is
`+`## function (e1, e2) .Primitive("+")so a first attempt might be
`+` <- function(e1, e2) {
if (is.character(e1) | is.character(e2)) {
paste0(e1, e2)
} else {
base::`+`(e1, e2)
}
}This checks to see if the left or right side of the operator is a character-classed object, and if either is, it pastes the two together. Otherwise it just uses the ‘regular’ addition operator between the two arguments. This works for simple cases, e.g.
"a" + "b"## [1] "ab""a" + 2## [1] "a2"2 + 2## [1] 42 + "a"## [1] "2a"But we hit an important snag if we try to add to character-represented numbers
"200" + "200"## [1] "200200"That’s probably going to be an issue if we read in unformatted data (e.g. from a CSV) as characters and try to treat it like numbers. Normally this would throw the above error about not being numeric, but now we get a silent weird number-character. That’s no good.
An extension to this checks whether or not we have the number-as-a-character situation and falls back to the correct interpretation in that case
`+` <- function(e1, e2) {
## unary
if (missing(e2)) return(e1)
if (!is.na(suppressWarnings(as.numeric(e1))) & !is.na(suppressWarnings(as.numeric(e2)))) {
## both arguments numeric-like but characters
return(base::`+`(as.numeric(e1), as.numeric(e2)))
} else if ((is.character(e1) & is.na(suppressWarnings(as.numeric(e1)))) |
(is.character(e2) & is.na(suppressWarnings(as.numeric(e2))))) {
## at least one true character
return(paste0(e1, e2))
} else {
## both numeric
return(base::`+`(e1, e2))
}
}
"a" + "b"## [1] "ab""a" + 2## [1] "a2"2 + 2## [1] 42 + "a"## [1] "2a""2" + "2"## [1] 42 + "edgy" + 4 + "me"## [1] "2edgy4me"So, that’s one option for string addition in R. Is it the right one? The idea of actually dispatching on a character class is inviting. Can we just add a +.character method (since there doesn’t seem to already be one)? Normally when we have S3 dispatch we need a generic function, which calls UseMethod("class"), but we don’t have that in this case. + is an internal generic, which is probably the first sign that we’re going to have trouble. If we try to define the method
`+` <- base::`+`
`+.character` <- function(e1, e2) {
paste0(e1, e2)
}
"a" + "b"## Error in "a" + "b": non-numeric argument to binary operatorIt seems to fail. What went wrong? Is dispatch not working?
We want to dispatch on “character” – is that what we have?
class("a")## [1] "character"What if we explicitly create an object with that class?
structure("a", class = "character") + 2## [1] "a2"2 + structure("a", class = "character")## [1] "2a"What if we try to dispatch on some new class?
`+.foo` <- function(e1, e2) {
paste0(e1, e2)
}
structure("a", class = "foo") + 2## [1] "a2"but no dice for just a regular atomic character object. Time to revisit the help pages.
In R, addition is limited to particular classes of objects, defined by the Ops group (there are also Math, Summary, and Complex groups). The methods for the Ops group members describe which classes can be involved in operations involving any of the Ops group members:
"+", "-", "*", "/", "^", "%%", "%/%"
"&", "|", "!"
"==", "!=", "<", "<=", ">=", ">"These methods are:
eval(.S3methods("Ops"), envir = baseenv())## [1] Ops.data.frame Ops.Date Ops.difftime
## [4] Ops.factor Ops.numeric_version Ops.ordered
## [7] Ops.POSIXt Ops.quosure* Ops.raster*
## [10] Ops.roman* Ops.ts* Ops.unit*
## see '?methods' for accessing help and source codeWhat’s missing from this list, in order for us to be able to just use “string” + “string” is a character method. What’s perhaps even more surprising is that there is a roman method! Whaaaat?
as.roman("1") + as.roman("5")## [1] VIas.roman("2000") + as.roman("18")## [1] MMXVIII
Since the operations need to be defined for all the members of the Ops group, we would also need to define what to do with, say, * between strings. When one side is a string and the other is a number, a reasonable approach might be that which was taken in the original post (using a new infix %s*%)
"a" %s*% 3## a
## "aaa"There is, of course, a function to do this already
strrep("a", 3)## [1] "aaa"but I could see creating "a" * 3 as a shortcut to this. Note: this exists in python
"a" * 3## 'aaa'I don’t know what one would expect "a" * "b" to produce.
The problem with where this is heading is that we aren’t allowed to create the method for an atomic class, as Joris Meys and Brodie Gaslam point out on Twitter
Yes, you're right. Below is what I remembered, which suggested that if it were not sealed, it could be defined, but that isn't true b/c
— BrodieG (@BrodieGaslam) October 4, 2018do_arithonly dispatches on objects (as you point out), although in theory it could dispatch on atomics, but probably doesn't for speed. pic.twitter.com/UXk6Tdm3lW
setMethod("+", c("character", "character"), function(e1, e2) paste0(e1, e2))## Error in setMethod("+", c("character", "character"), function(e1, e2) paste0(e1, : the method for function '+' and signature e1="character", e2="character" is sealed and cannot be re-definedso no luck there. Brodie also links to a Stack Overflow discussion on this very topic where it is pointed out by Martin Mächler that this has been discussed on r-develq – that makes for some interesting historical weigh-ins on why this isn’t a thing in R. Incidentally, the small-world effect comes into play regarding that Stack Overflow post as one of the three answers happens to be a former work colleague of mine.
So, in the end, it seems the best we can do is the rather long-winded overwrite of + which checks if the arguments really are characters. I don’t mind this, and would probably use it if it was in base R or a package. The biggest issue that people seem to have with this is that it ‘looks like’ addition, but it’s not commutative. If that word is new to you, it just means that x + y should give the same answer as y + x. For numbers, the regular + satisfies this:
2 + 3## [1] 53 + 2## [1] 5but when we try to do this with strings… not so much
"a" + "b"## [1] "ab""b" + "a"## [1] "ba"This doesn’t particularly bother me, because I’m okay with this not actually being ‘mathematical addition’. The fun turn this then took was the suggestion from Joris Meys that Julia’s non-associative operators is a strength of the language. There, the way that you group values matters
a + b + c is parsed as +(a, b, c) not +(+(a, b), c).
I’ll eventually get around to learning more Julia, but this is already hurting my brain.
That distinction may be of interest, however, to Miles McBain, whose concern was more about repeated applications of + being a bottleneck
I hate + for string concatenation. “a” + “b” + “c” is paste(“a”, paste(“b”,“c”)). So you end up copying the data in “b” and “c” twice due to the data being immutable. That can really add up fast with more +'s if you are careless. Like I was in my first programming job.
— Miles McBain (@MilesMcBain) October 4, 2018
In that case, parsing as +("a", "b", "c") is exactly what would be desired.
So, what’s the conclusion of all of this? I’ve learned (and re-learned) a heap more about how the Ops group works, I’ve played a lot with dispatch, and I’ve thought deeply about edge-cases for adding strings. I’ve also been exposed to a bit more Julia. All in all, a worthwhile dive into something potentially silly, but a lot of fun. If you have some thoughts on the matter, leave a comment here or reply on Twitter – I’d love to hear about another angle to this story.
devtools::session_info()
## ─ Session info ──────────────────────────────────────────────────────────
## setting value
## version R version 3.5.2 (2018-12-20)
## os Pop!_OS 19.04
## system x86_64, linux-gnu
## ui X11
## language en_AU:en
## collate en_AU.UTF-8
## ctype en_AU.UTF-8
## tz Australia/Adelaide
## date 2019-08-13
##
## ─ Packages ──────────────────────────────────────────────────────────────
## package * version date lib source
## assertthat 0.2.1 2019-03-21 [1] CRAN (R 3.5.2)
## backports 1.1.4 2019-04-10 [1] CRAN (R 3.5.2)
## blogdown 0.14.1 2019-08-11 [1] Github (rstudio/blogdown@be4e91c)
## bookdown 0.12 2019-07-11 [1] CRAN (R 3.5.2)
## callr 3.3.1 2019-07-18 [1] CRAN (R 3.5.2)
## cli 1.1.0 2019-03-19 [1] CRAN (R 3.5.2)
## crayon 1.3.4 2017-09-16 [1] CRAN (R 3.5.1)
## desc 1.2.0 2018-05-01 [1] CRAN (R 3.5.1)
## devtools 2.1.0 2019-07-06 [1] CRAN (R 3.5.2)
## digest 0.6.20 2019-07-04 [1] CRAN (R 3.5.2)
## dplyr * 0.8.3 2019-07-04 [1] CRAN (R 3.5.2)
## evaluate 0.14 2019-05-28 [1] CRAN (R 3.5.2)
## forcats * 0.4.0 2019-02-17 [1] CRAN (R 3.5.1)
## fs 1.3.1 2019-05-06 [1] CRAN (R 3.5.2)
## glue 1.3.1 2019-03-12 [1] CRAN (R 3.5.2)
## htmltools 0.3.6 2017-04-28 [1] CRAN (R 3.5.1)
## jsonlite 1.6 2018-12-07 [1] CRAN (R 3.5.1)
## knitr 1.24 2019-08-08 [1] CRAN (R 3.5.2)
## lattice 0.20-38 2018-11-04 [1] CRAN (R 3.5.1)
## magrittr 1.5 2014-11-22 [1] CRAN (R 3.5.1)
## Matrix 1.2-17 2019-03-22 [1] CRAN (R 3.5.2)
## memoise 1.1.0 2017-04-21 [1] CRAN (R 3.5.1)
## pillar 1.4.2 2019-06-29 [1] CRAN (R 3.5.2)
## pkgbuild 1.0.4 2019-08-05 [1] CRAN (R 3.5.2)
## pkgconfig 2.0.2 2018-08-16 [1] CRAN (R 3.5.1)
## pkgload 1.0.2 2018-10-29 [1] CRAN (R 3.5.1)
## prettyunits 1.0.2 2015-07-13 [1] CRAN (R 3.5.1)
## processx 3.4.1 2019-07-18 [1] CRAN (R 3.5.2)
## ps 1.3.0 2018-12-21 [1] CRAN (R 3.5.1)
## purrr 0.3.2 2019-03-15 [1] CRAN (R 3.5.2)
## R6 2.4.0 2019-02-14 [1] CRAN (R 3.5.1)
## Rcpp 1.0.2 2019-07-25 [1] CRAN (R 3.5.2)
## remotes 2.1.0 2019-06-24 [1] CRAN (R 3.5.2)
## reticulate 1.13 2019-07-24 [1] CRAN (R 3.5.2)
## rlang 0.4.0 2019-06-25 [1] CRAN (R 3.5.2)
## rmarkdown 1.14 2019-07-12 [1] CRAN (R 3.5.2)
## roperators * 1.1.0 2018-09-28 [1] CRAN (R 3.5.1)
## rprojroot 1.3-2 2018-01-03 [1] CRAN (R 3.5.1)
## sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 3.5.1)
## stringi 1.4.3 2019-03-12 [1] CRAN (R 3.5.2)
## stringr 1.4.0 2019-02-10 [1] CRAN (R 3.5.1)
## testthat 2.2.1 2019-07-25 [1] CRAN (R 3.5.2)
## tibble 2.1.3 2019-06-06 [1] CRAN (R 3.5.2)
## tidyselect 0.2.5 2018-10-11 [1] CRAN (R 3.5.1)
## usethis 1.5.1 2019-07-04 [1] CRAN (R 3.5.2)
## withr 2.1.2 2018-03-15 [1] CRAN (R 3.5.1)
## xfun 0.8 2019-06-25 [1] CRAN (R 3.5.2)
## yaml 2.2.0 2018-07-25 [1] CRAN (R 3.5.1)
##
## [1] /home/jono/R/x86_64-pc-linux-gnu-library/3.5
## [2] /usr/local/lib/R/site-library
## [3] /usr/lib/R/site-library
## [4] /usr/lib/R/library