If you're interested in the visualisation of networks or graphs, you might've heard of the great package "visNetwork". I think it's a really great package and I love playing around with it. The scenarios of graph-based analyses are many and diverse: whenever you can describe your data in terms of "outgoing" and "receiving" entities, a graph-based analysis and/or visualisation is possible. During my work as a linguist, I already used graphs for different purposes like linking-structures within dictionaries, visualising co-occurence patterns of words and so on.

Today, I want to show you something completely different: transfers of male football players in the German "1. Bundesliga", the first division of male football in Germany. We can also describe this data in terms of outgoing and receiving entities (the nodes in the network): the clubs who are selling the players and the clubs who are buying the players. The edges (connections) within the network are the players themselves. And there are further attributes associated with the edges, e.g. the price of the player.

I'll spare you the boring details of getting the data (please write a comment if you would like more details on that). I start with the raw data structure created by the scraping process. It's a dataframe called transfer.df that looks like this:


"abloese" (or "Ablöse") means "transfer fee" and currently holds a string with certain codes and the currency: "ablösefrei" means that no transfer fee had to be payed (remember the famous Bosman ruling?). "-" means that the information on a transfer fee doesn't make sense (e.g., when a player finishes his career). "?" means that no information about the transfer fee is available. "Mio." and "Tsd." just encodes "million" or "thousand", we have to deal with that later.

But we have to take care of something else first. In the dataframe, a player always appears twice if he changed teams within the 1. Bundesliga: the first time for his old club (as outgoing) and the second time for his new club (as incoming). I dealt with it this way (also, I loaded the required packages):

library(visNetwork)
library(igraph)
library(stringr)


transfer.df2 <- data.frame()
all.players <- unique(transfer.df$player)
for (pi in all.players) {
  vork <- grep(pi, transfer.df$player, fixed = T)
  if (length(vork) == 1) {
    transfer.df2 <- rbind(transfer.df2, transfer.df[vork,])
  } else {
    transfer.df2 <- rbind(transfer.df2, transfer.df[vork[1],])
  }
}

This basically means that, whenever a player appears more than once in transfer.df, the first appearance is kept and the second appearance is deleted. The resulting network wouldn't be any different if we would keep the second appearance. So, now we are using transfer.df2 as our data structure.

Now, we have to deal with the "abloese" (transfer fee) column:

transfer.df2$abloese.num <- sapply(transfer.df2$abloese, USE.NAMES = F, FUN = function (x) {
  if (x %in% c("-", "?")) NA else {
    if (x == "ablösefrei") 0 else {
      mio <- grepl("Mio.", x, fixed = T)
      tsd <- grepl("Tsd.", x, fixed = T)
      x2 <- gsub(",", ".", x, fixed = T)
      x3 <- gsub("Mio. €", "", x2, fixed = T)
      x4 <- as.numeric(str_trim(gsub("Tsd. €", "", x3, fixed = T)))
      if (mio) x4*1000000 else {
        if (tsd) x4*1000 else { "FEHLER" }
      }
    }
  }

})

Basically, this is what we are doing:

  • If abloese is "-" or "?", we are using NA
  • If abloese is "ablösefrei" we are putting in 0
  • Then we see whether "Mio." appears in the string.
  • Then we see whether "Tsd." appears in the string.
  • Then we are deleting these substrings and the EUR sign and
  • trim the string and convert it to a numeric value.
  • If "Mio." appeared in the string, we are multiplying the result with one million and if "Tsd." appeared in the string, we are multiplying the result with one thousand (both will never appear in the string, it doesn't make sense).
Alright, now we have the column abloese.num and can move on to group the different transfer fees because we want to assign different colours to the edges in the network dependent on the transfer sum. The thresholds are arbitrary.

transfer.df2$abl.group <- cut(transfer.df2$abloese.num, c(0, 200*1000, 1000*1000, 2000*1000, 5000*1000, 10000*1000, 60000*1000), include.lowest = T)

transfer.df2$abl.col <- ifelse(transfer.df2$abloese.num == 0, "green",
                               ifelse(transfer.df2$abl.group == "[0,2e+05]", "#ffffcc",
                                      ifelse(transfer.df2$abl.group == "(2e+05,1e+06]", "#fed976",
                                             ifelse(transfer.df2$abl.group == "(1e+06,2e+06]", "#feb24c",
                                                    ifelse(transfer.df2$abl.group == "(2e+06,5e+06]", "#fc4e2a",
                                                           ifelse(transfer.df2$abl.group == "(5e+06,1e+07]", "#e31a1c",
                                                                  ifelse(transfer.df2$abl.group == "(1e+07,6e+07]", "#800026", "grey")))))))
transfer.df2$abl.col <- ifelse(is.na(transfer.df2$abl.group), "grey", transfer.df2$abl.col) 

Now, I am converting the dataframe to an igraph object and this object to visNetwork object. I'm sure the igraph step could be skipped, but this works like a charm and doesn't take much time.

graph <- graph.data.frame(transfer.df2)

vn <- toVisNetworkData(graph)

I am assigning color codes to the nodes:

vn$nodes$color <- ifelse(vn$nodes$id %in% clubs, "tomato",
                         ifelse(vn$nodes$id == "Vereinslos", "green",
                                ifelse(vn$nodes$id == "Karriereende", "blue", "grey")))

All clubs in the 1. Bundesliga get "tomato" (clubs is an object I defined earlier) all clubs that are not in the 1. Bundesliga (e.g., Hamburger SV) get "grey". There are two other special "clubs": "Karriereende" for "end of career" and "Vereinslos" for "no club", both get "green".

Three things left to be done:

vn$edges$title <- paste(vn$edges$player, vn$edges$abloese, sep = " - ")
vn$edges$color <- vn$edges$abl.col
vn$edges$width <- 4
  • Assign a title to the edges that consists of the player name and the transfer fee. This appears upon hovering the edge.
  • Assign the grouped transfer fee color we defined earlier.
  • Increase the width of the edges to make the color more visible.
No, for creating the HTML file for the graph:

visNetwork(nodes = vn$nodes, edges = vn$edges, height = "1000px", width = "100%") %>%
  visOptions(highlightNearest = TRUE) %>%
  #visIgraphLayout(layout = "layout_with_dh") %>%
  visEdges(arrows = "to", arrowStrikethrough = F) %>% visSave(file = "~/Desktop/transfers.html", selfcontained = T)

Please visit my personal webspace for the final result. The "redder" an edge in the network is, the more expensive the transfer was. You can also click on the nodes to only highlight all adjacent nodes (selling and buying clubs), drag nodes around (graph physics!) and hover over edges to see the specific player being transfered. Of course, zooming is enabled. visNetwork does all that. I love that package!

12

View comments

Hi all, this is just an announcement.

I am moving Rcrastinate to a blogdown-based solution and am therefore leaving blogger.com.

Alright, seems like this is developing into a blog where I am increasingly investigating my own music listening habits.

Recently, I've come across the analyzelastfm package by Sebastian Wolf. I used it to download my complete listening history from Last.FM for the last ten years.

3

Giddy up, giddy it up

Wanna move into a fool's gold room

With my pulse on the animal jewels

Of the rules that you choose to use to get loose

With the luminous moves

Bored of these limits, let me get, let me get it like

Wow!

When it comes to surreal lyrics and videos, I'm always thinking of Be

Click here for the interactive visualization

If you're interested in the visualisation of networks or graphs, you might've heard of the great package "visNetwork". I think it's a really great package and I love playing around with it.

12

Here is some updated R code from my previous post. It doesn't throw any warnings when importing tracks with and without heart rate information. Also, it is easier to distinguish types of tracks now (e.g., when you want to plot runs and rides separately).

3

So, Strava's heatmap made quite a stir the last few weeks. I decided to give it a try myself. I wanted to create some kind of "personal heatmap" of my runs, using Strava's API.

I've been using the ggplot2 package a lot recently. When creating a legend or tick marks on the axes, ggplot2 uses the levels of a character or factor vector. Most of the time, I am working with coded variables that use some abbreviation of the "true" meaning (e.g.

It's been a while since I had the opportunity to post something on music. Let's get back to that.

I got my hands on some song lyrics by a range of artists. (I have an R script to download all lyrics for a given artist from a lyrics website.

4

Lately, I got the chance to play around with Shiny and Leaflet a lot - and it is really fun! So I decided to catch up on an old post of mine and build a Shiny application where you can upload your own GPX files and plot them directly in the browser.

9

[EDIT: The function now also inludes the possibility to plot the IQR around the median. I shifted the median slightly downwards to prevent the SD and the IQR from overlapping.]

I wrote a function to visualise results of Likert scale items. Please find the function below the post.

9

Today I want to write about a solution to a quite specific problem. Suppose, you want to label cells in your 'vcd' package mosaic plots in a custom way. For example, we might want to use cell labels which indicate "too much" or "too few" cases (given your expected values).

3

png("goodbye.png", height = 625, width = 500)

par(col = "purple")

plot(1, 1, xlim = c(0,800), ylim = c(0,1600), type = "n", bty = "n", xaxt = "n", yaxt = "n", xlab = "", ylab = "")

symbols(x = 400, y = 1200, circles = 400, add = T, lwd = 40)

lines(x = c(400, 400), y = c(900, 100), lwd = 40, lend

R is great, and you can do a LOT OF stuff with it.

However, sometimes you want to do really basic stuff with huge or a lot of files. At work, I have to do that a lot because I am mostly dealing with language data that often needs some pre-processing.

I work with R on both Mac OS and Windows. On Windows, you get the option to copy the path of a file or folder by holding Shift while right-clicking on the file or folder. As useful as this feature is, it copies paths to your clipboard in Windows format, e.g.

4

I've got a NetAtmo weather station. One can download the measurements from its web interface as a CSV file.

5

Want to change the font used in your R plots? I got a quite simple solution that works on Mac OS.

You need the function 'quartzFonts'. With this function, you can define additional font families to use in your R base graphic plots. The default font families are 'sans', 'serif' and 'mono'.

5

Many GPS devices and apps have the capability to track your current position via GPS. If you go walking, running, cycling, flying or driving, you can take a look at your exact route and your average speed.

5

This is something I did a while ago using the Berlin Affective Word List (BAWL).

The BAWL contains ratings for 2902 German words (2107 nouns, 504 verbs, 291 adjectives). Ratings were collected for emotional valence (bad vs.

Alright, let's test some parallelization functionalities in R.

The machine:

MacBook Air (mid-2013) with 8 GB of RAM and the i7 CPU (Intel i7 Haswell 4650U). This CPU is hyper-threaded, meaning (at least that's my understanding of it) that it has two physical cores but can run up to four threads.

4

I recently encountered some functionality in R which most of you might already know. Nevertheless, I want to share it here, because it might come in handy for those of you who do not know this yet.

Suppose you want to read in a large number of very large text tables in R.

The biggest German railway company, the 'Deutsche Bahn', is subject of frequent emotional discussions about being late all the time. A big German newspaper, the Süddeutsche Zeitung built the so-called 'train monitor' (Zugmonitor).

I used knitr to hack together a very short tutorial about XML in R.

It's in German. And it's not very long. But, hey, it's free :)

I hope it can be of help to someone who wants to get started with XML processing in R.

Please feel free to post or send any comments about the thing.

2

Which function rbinds dataframes together fastest?

First competitor: classic rbind in a for loop over a list of dataframes

Second competitor: do.call("rbind", <list of dataframes>)

Third competitor: rbind.fill(<list of dataframes>) from the plyr package

The job:

- rbinding a list of dataframes

2

I already introduced some stuff I did with the last.fm API. But did you ever wonder if your taste of music changes over the year? Sunny music in the sunny months and dark music in darker months? Well, I did. And I want to check it out with the RLastFM package and some additional functions.

3

I want to share a function I wrote for my dissertation. The function is useful for putting up to two R tables into one TeX table.

You have to load the package 'languageR' to have the dataset 'dative' available.

last.fm is an internet radio and music suggestion service. Registered users can also use last.fm to 'scrobble' tracks they've been listening to. last.fm then keeps track of a user's statistics in terms of top artists, albums and tracks.

Let's get back to the age-value relationship from my last post. I did some more plotting to see on which position this inversed U-shaped relationship is strongest.

It's been some time since my last post on football. And we're talking about european soccer here.

So I finally managed to write some functions which allow me to extract player stats from www.transfermarkt.de. The site tracks lots of stats in the world of soccer.

As long as I can't find the time to post my newest adventuRes, why don't you check out the great collection of other R-blogs on the web:

www.r-bloggers.com 

Have fun!

Just as a quick reply to a friend of mine who suggested testing the swearing capabilities of The Dude:

Click to enlarge.

As you can see, "The Big Lebowski" (2.79 % swear words) takes the Tarantino threshold (0.98 %) easily, but it's no match against "Reservoir Dogs" (3.28 %).

Fortunately, there is a page called www.opensubtitles.org, where you can get subtitle (.SRT) files for virtually every movie. Now let's see what we can do with these. SRT files are in plain text format (human readable) and can thus be read quite easily with R.

Just a fast note: I came across the R-package "knitr" which enables you to generate PDF files by mixing LaTeX and R code in one document. The result looks very nice and is great to create documentations, manuals and so on.

Blog Archive
BlogRoll
BlogRoll
  • Thanks, on its way to CRAN The generic seal of approval from the CRAN team – countless hours spent tabbing between R CMD check and R CMD build logs, ‘Writing R Extensions’ and Stac...

    6 hours ago
Loading
Dynamic Views theme. Powered by Blogger. Report Abuse.