Visualizing Data 📈

MATH/COSC 3570 Introduction to Data Science

Dr. Cheng-Han Yu
Department of Mathematical and Statistical Sciences
Marquette University

Categorical vs. Numerical Variables

• A categorical (qualitative) variable provides non-numerical information which can be placed in one (and only one) category from two or more categories.

• Gender (Male 👨, Female 👩, Other 🏳️‍🌈)
• Class (Freshman, Sophomore, Junior, Senior, Graduate)
• Country (USA 🇺🇸, Canada 🇨🇦, UK 🇬🇧, Germany 🇩🇪, Japan 🇯🇵, Korea 🇰🇷)

• A numerical (quantitative) variable is recorded in a numerical value representing counts or measurements.

• GPA
• The number of relationships you’ve had
• Height

Data: Lending Club

• Lending Club is a platform that allows individuals to lend to other individuals.

• Not all loans are created equal – ease of getting a loan depends on ability to pay back the loan.

• Data includes loans made, these are not loan applications.

• OK. The data set we will be working with comes from the lending club.
• Thousands of loans made through the Lending Club, which is a platform that allows individuals to lend to other individuals.
• If you are finance or econ major, or you know about loans, you know that Not all loans are created equal – ease of getting a loan depends on (apparent) ability to pay back the loan.
• Data include loans made, these are not loan applications.
• It’s important to keep this in mind when we are looking at the relationships between the variables that we are going to see, since the pattern may be different for those relationships for loans that never got approved.

Take a Peek at Data

library(openintro) ## for loading the data set
dplyr::glimpse(loans_full_schema)

Rows: 10,000
Columns: 55
$ emp_title                        <chr> "global config engineer ", "warehouse…
$ emp_length                       <dbl> 3, 10, 3, 1, 10, NA, 10, 10, 10, 3, 1…
$ state                            <fct> NJ, HI, WI, PA, CA, KY, MI, AZ, NV, I…
$ homeownership                    <fct> MORTGAGE, RENT, RENT, RENT, RENT, OWN…
$ annual_income                    <dbl> 90000, 40000, 40000, 30000, 35000, 34…
$ verified_income                  <fct> Verified, Not Verified, Source Verifi…
$ debt_to_income                   <dbl> 18.01, 5.04, 21.15, 10.16, 57.96, 6.4…
$ annual_income_joint              <dbl> NA, NA, NA, NA, 57000, NA, 155000, NA…
$ verification_income_joint        <fct> , , , , Verified, , Not Verified, , ,…
$ debt_to_income_joint             <dbl> NA, NA, NA, NA, 37.7, NA, 13.1, NA, N…
$ delinq_2y                        <int> 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 0…
$ months_since_last_delinq         <int> 38, NA, 28, NA, NA, 3, NA, 19, 18, NA…
$ earliest_credit_line             <dbl> 2001, 1996, 2006, 2007, 2008, 1990, 2…
$ inquiries_last_12m               <int> 6, 1, 4, 0, 7, 6, 1, 1, 3, 0, 4, 4, 8…
$ total_credit_lines               <int> 28, 30, 31, 4, 22, 32, 12, 30, 35, 9,…
$ open_credit_lines                <int> 10, 14, 10, 4, 16, 12, 10, 15, 21, 6,…
$ total_credit_limit               <int> 70795, 28800, 24193, 25400, 69839, 42…
$ total_credit_utilized            <int> 38767, 4321, 16000, 4997, 52722, 3898…
$ num_collections_last_12m         <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0…
$ num_historical_failed_to_pay     <int> 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0…
$ months_since_90d_late            <int> 38, NA, 28, NA, NA, 60, NA, 71, 18, N…
$ current_accounts_delinq          <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0…
$ total_collection_amount_ever     <int> 1250, 0, 432, 0, 0, 0, 0, 0, 0, 0, 0,…
$ current_installment_accounts     <int> 2, 0, 1, 1, 1, 0, 2, 2, 6, 1, 2, 1, 2…
$ accounts_opened_24m              <int> 5, 11, 13, 1, 6, 2, 1, 4, 10, 5, 6, 7…
$ months_since_last_credit_inquiry <int> 5, 8, 7, 15, 4, 5, 9, 7, 4, 17, 3, 4,…
$ num_satisfactory_accounts        <int> 10, 14, 10, 4, 16, 12, 10, 15, 21, 6,…
$ num_accounts_120d_past_due       <int> 0, 0, 0, 0, 0, 0, 0, NA, 0, 0, 0, 0, …
$ num_accounts_30d_past_due        <int> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0…
$ num_active_debit_accounts        <int> 2, 3, 3, 2, 10, 1, 3, 5, 11, 3, 2, 2,…
$ total_debit_limit                <int> 11100, 16500, 4300, 19400, 32700, 272…
$ num_total_cc_accounts            <int> 14, 24, 14, 3, 20, 27, 8, 16, 19, 7, …
$ num_open_cc_accounts             <int> 8, 14, 8, 3, 15, 12, 7, 12, 14, 5, 8,…
$ num_cc_carrying_balance          <int> 6, 4, 6, 2, 13, 5, 6, 10, 14, 3, 5, 3…
$ num_mort_accounts                <int> 1, 0, 0, 0, 0, 3, 2, 7, 2, 0, 2, 3, 3…
$ account_never_delinq_percent     <dbl> 92.9, 100.0, 93.5, 100.0, 100.0, 78.1…
$ tax_liens                        <int> 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0…
$ public_record_bankrupt           <int> 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0…
$ loan_purpose                     <fct> moving, debt_consolidation, other, de…
$ application_type                 <fct> individual, individual, individual, i…
$ loan_amount                      <int> 28000, 5000, 2000, 21600, 23000, 5000…
$ term                             <dbl> 60, 36, 36, 36, 36, 36, 60, 60, 36, 3…
$ interest_rate                    <dbl> 14.07, 12.61, 17.09, 6.72, 14.07, 6.7…
$ installment                      <dbl> 652.5, 167.5, 71.4, 664.2, 786.9, 153…
$ grade                            <ord> C, C, D, A, C, A, C, B, C, A, C, B, C…
$ sub_grade                        <fct> C3, C1, D1, A3, C3, A3, C2, B5, C2, A…
$ issue_month                      <fct> Mar-2018, Feb-2018, Feb-2018, Jan-201…
$ loan_status                      <fct> Current, Current, Current, Current, C…
$ initial_listing_status           <fct> whole, whole, fractional, whole, whol…
$ disbursement_method              <fct> Cash, Cash, Cash, Cash, Cash, Cash, C…
$ balance                          <dbl> 27016, 4651, 1825, 18853, 21430, 4257…
$ paid_total                       <dbl> 1999, 499, 282, 3313, 2325, 873, 2731…
$ paid_principal                   <dbl> 984, 349, 175, 2747, 1570, 743, 1440,…
$ paid_interest                    <dbl> 1015.2, 150.5, 106.4, 566.1, 754.8, 1…
$ paid_late_fees                   <dbl> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0…

• The data set is stored in the openintro package.
• In addition to head, structure function, we can also use glimpse() function to see the general picture of the data.
• We can see that we have 10000 rows, so there are 10000 observations, or loans made in the data set.
• And there are 55 columns.
• And the name of the data set is loans_full_schema.

Selected Variables

loans <- loans_full_schema |> 
    dplyr::select(loan_amount, 
                  interest_rate, 
                  grade, 
                  homeownership, 
                  debt_to_income)
glimpse(loans)

Rows: 10,000
Columns: 5
$ loan_amount    <int> 28000, 5000, 2000, 21600, 23000, 5000, 24000, 20000, 20…
$ interest_rate  <dbl> 14.07, 12.61, 17.09, 6.72, 14.07, 6.72, 13.59, 11.99, 1…
$ grade          <ord> C, C, D, A, C, A, C, B, C, A, C, B, C, B, D, D, D, F, E…
$ homeownership  <fct> MORTGAGE, RENT, RENT, RENT, RENT, OWN, MORTGAGE, MORTGA…
$ debt_to_income <dbl> 18.01, 5.04, 21.15, 10.16, 57.96, 6.46, 23.66, 16.19, 3…

• There are 55 variables in the data, but our visualization purpose, we are only going to focus on 8 of these columns.
• loan_amount, …, debt_to_income ratio
• I piped the data into the select function, which allows me to select variables I want by name, and called the selected data loans.
• So the data has all 10000 observations and 8 selected variables.

Variable Description

variable	description
loan_amount	Amount of the loan received, in US dollars
interest_rate	Interest rate on the loan, in an annual percentage
grade	Loan grade, which takes a values A through G and represents the quality of the loan
homeownership	Indicates whether the person owns, owns but has a mortgage, or rents
debt_to_income	Debt-to-income ratio

Variable Types

variable	type
loan_amount	numerical, continuous
interest_rate	numerical, continuous
grade	categorical, ordinal
homeownership	categorical, nominal
debt_to_income	numerical, continuous

Visualizing Categorical Data

Bar Chart

• A bar chart shows the frequency table of a categorical variable.

## geom_bar(stat = "count")
loans |> ggplot(aes(x = homeownership)) +
    geom_bar()  #<<

• One way representing categorical data is using bar plot.
• Here we don’t use geom_point anymore. Instead, we use the geom_bar function.
• Here we would like to see the frequency distribution of variable homeownership, so I map x to the homeownership variable.
• When the geom_bar() is called. ggplot2 actually counts the number of each category for us. In other words, ggplots automatically creates the frequency table of homeownership for us, telling us how many observations belong to mortgage, how many belong to own and how many belong to rent.
• So the question is, Where does count come from? How does ggplot2 do the calculation?
• Note that on the y-axis, it displays count, but count is NOT a variable in loans!
• Where does count come from?

Stacked Bar Chart

(freq_tbl <- 
    as.data.frame(
        table(loans$homeownership)
        ))

      Var1 Freq
1             0
2      ANY    0
3 MORTGAGE 4789
4      OWN 1353
5     RENT 3858

## remove count 0 item
freq_tbl <- freq_tbl[-c(1, 2), ]

## column names
names(freq_tbl) <- c("type","count")
freq_tbl

      type count
3 MORTGAGE  4789
4      OWN  1353
5     RENT  3858

bar <- freq_tbl |> 
    ggplot(aes(x = "", 
               y = count, 
               fill = type)) + 
    geom_bar(stat = "identity")
bar

• Check the proportion or percentage of each category, we may want to create a stacked bar chart.
• To create such plot, we need to prepare a frequency table.
• x = “” because we don’t map any variable in the frequency table data.
• fill = type because we are gonna fill the bar with colors by the type of homeownership.
• stat = “identity” because here we don’t need to calculate the counts of each category, “identity” means using the values provided in the data.
• geom_bar() uses stat_count() by default: it counts the number of cases at each x position. geom_col() uses stat_identity(): it leaves the data as is.

bar + theme_minimal() + labs(x = ““) ggplot(freq_tbl, aes(x =”“, y = count, color = type)) + geom_bar(stat =”identity”)

Pie Chart

pie <- bar + 
    coord_polar(theta = "y")
pie

pie + theme_void()

theta = variable to map angle to (x or y) Offset of starting point from 12 o’clock in radians. pie + geom_text(aes(label = count), position = position_stack(vjust = 0.5))

Segmented Bar Plot: Stacked

loans |> ggplot(aes(x = homeownership, 
                    fill = grade)) + #<<
    geom_bar()

• We can use Segmented Bar Plot, which is helpful when you want to represent two variables.
• So here we fill these bars based on the grade of the loan, which we know ranges from A to G.
• So each one of theses segments tells us how many of that grade of loan was observed within a particular homeownership.

Segmented Bar Plot: Compare Proportions

• position = "fill" makes each set of stacked bars the same height.

loans |> ggplot(aes(x = homeownership, fill = grade)) +
    geom_bar(position = "fill") #<<

• We may want to add another argument in the geom_bar layer, which is position = “fill”.
• position = "fill" works like stacking, but makes each set of stacked bars the same height.
• And now the y-axis is not count anymore. Now the y limit is from 0 to 1, and each grade segment represents the relative frequency of a particular homeownership.
• Which bar plot is a more useful representation for visualizing the relationship between homeownership and grade?

Segmented Bar Plot: Compare Individual Values

• position = "dodge" places overlapping objects directly beside one another.

loans |> ggplot(aes(x = homeownership, fill = grade)) +
  geom_bar(position = "dodge") #<<

• If you wanna compare Individual Values, here the individual category of the grade of loan in each ownership type, position = "dodge" may be a goo option.
• The option places the grade of loans directly beside one another.
• This way, it’s easier to see the counts of grade of loan within each ownership.

Customizing Bar Plots

loans |> 
  ggplot(
    aes(x = homeownership,
        fill = homeownership)
  ) + 
  geom_bar(
    color = "blue", 
    width = 0.2, 
    alpha = 0.5
  ) + 
  labs(
    x = "Homeownership", 
    title = "Homeownership Counts"
  ) +
  geom_text(
    aes(label = after_stat(count)), 
        stat = 'count', 
        hjust = 3, 
        color = "red", 
        size = 5
  ) + 
  theme_minimal() + 
  coord_flip() ## y = homeownership

• ggplot2::geom_bar()

• ggplot2::geom_text()

• stat is short for statistical transformation.
• stat = 'count' uses the stat_count() method to get the count of each homeownership.
• One reason why ggplot2 is powerful is you can customize your plot with great flexibility.
• When you use aesthetic option fill, each bar will be colored according to the variable we assign.
• Because here x and fill map to the same variable, we have each bar has its own one color.
• We can specify some settings in geom_bar. color is the color of the edge of the bar, and we can also control the width and transparency of bars.
• We know labs already.
• If you wanna add text other then labels to your plot, you can use geom_text() function.
• If you wanna add the counts on each bar, you map label to the variable ..count..
• When you see the the variable or data has ..variable_name.., it means that it is an interval variable created by ggplot2.
• In order to use this ..count.. internal variable, we have to set the stat transformation = “count.
• And we can adjust the text horizontally through [hjust], and we can also specify its color and size as we usually do.
• Finally I use theme_minimal() and I flip the coordinate so that the bars become horizontal. It is the same as y = homeownership

Visualizing Numerical Data

Histogram

loans |> ggplot(aes(x = loan_amount)) +
    geom_histogram() #<<

`stat_bin()` using `bins = 30`. Pick better value with `binwidth`.

• One of the most commonly used ways of visualizing numerical data is with histograms.
• In ggplot2, we use geom_histogram to create a histogram.
• When you use this function, ggplot2 send you a message saying that it uses 30 bins, or the number of classes created from the data, each representing one coloumn in the histogram.
• And you can pick better value with binwidth, or the class width.

• binwidth = 1000
• binwidth = 5000
• binwidth = 20000

ggplot(loans, aes(x = loan_amount)) +
    geom_histogram(binwidth = 1000)

ggplot(loans, aes(x = loan_amount)) +
    geom_histogram(binwidth = 5000)

ggplot(loans, aes(x = loan_amount)) +
    geom_histogram(binwidth = 20000)

• In this plot I use a binwidth of a thousand dollars, so each one of the bars represents a range of thousand dollars, and the heights of these bars tell me how many loans fall into that range.
• In the second plot, the binwidth is 500 thousand dollars. So here we can have better sense of the shape of the distribution, comparing to the previous one.
• The last plot use the binwidth 20000 dollars. And now we lose anything interesting about the shape of the distribution.
• So picking a binwidth or the number of bins is an art and science. But try 10 to 20 bins may be a good start.

Customizing Histograms

loans |> 
  ggplot(
    aes(x = loan_amount)
  ) +
  geom_histogram(
    binwidth = 5000,
    fill = "#003366",  
    colour = "#FFCC00",  
    alpha = 0.8,  
    linetype = "dashed"
  ) +  
  labs(
    x = "Loan amount ($)", 
    y = "Frequency", 
    title = "Lending Club loans"
  ) + 
  theme_light()

• ggplot2::geom_histogram()

• We can also customize histograms.
• What I did here is that I change the color of these bars with fill argument
• And the color argument is for the edge of the bars.
• linetype tells ggplot that the linetype of the edge of the bars is dashed line.
• And again, we can add labels using labs function.

Fill with a Categorical Variable: Stack

loans |> 
  ggplot(
    aes(x = loan_amount,
        fill = homeownership) #<<
  ) +
  geom_histogram(
    binwidth = 5000
  ) +
  labs(
    x = "Loan amount ($)",
    y = "Frequency"
  ) +
  theme_linedraw()

• We can also fill a histogram with a categorical variable, so it allows us to explore the relationship between the numerical and categorical variables.
• This is done by adding aesthetic fill. Here basically filling the bars based on home ownership.
• So we can actually see what’s happening behind each level

Fill with a Categorical Variable: Identity (bad)

loans |> 
  ggplot(
    aes(x = loan_amount,
        fill = homeownership)
  ) +
  geom_histogram(
    binwidth = 5000,
    position = "identity" #<<
  ) +
  labs(
    x = "Loan amount ($)",
    y = "Frequency",
    title = "Lending Club loans"
  ) +
  theme_minimal()

Why such plot is bad?

• So here, if we don’t specify alpha, there is no transparency at all.
• And we can see that the histogram of loan amount for house owners is totally masked by the histogram for renters, because there are much more renters than house owners, and the frequency of each bin in the histogram for the renters is higher than the frequency for the owners.
• So use visualization carefully. Sometimes the plot does not provide more information, and sometimes it is misleading.

Fill with a Categorical Variable: Identity

loans |> 
  ggplot(
    aes(x = loan_amount,
        fill = homeownership)
  ) +
  geom_histogram(
    binwidth = 5000,
    alpha = 0.5,  #<<
    position = "identity"
  ) +
  labs(
    x = "Loan amount ($)",
    y = "Frequency",
    title = "Lending Club loans"
  ) +
  theme_minimal()

• We can also compare the histograms of loan amount of different homeownership by simply superimposing on one to another.
• We use position = “identity” to do that.
• In this case, we basically put three histograms together in one plot, sharing the same x-axis and y-axis
• If you wanna use this type of plot, remember to make it transparent, so that each histogram can be seen even they are superimposed.

Facet with a Categorical Variable

loans |> 
  ggplot(
    aes(x = loan_amount,
        fill = homeownership)
  ) +
  geom_histogram(
    binwidth = 5000
  ) +
  labs(
    x = "Loan amount ($)",
    y = "Frequency",
    title = "Lending Club loans"
  ) +
  facet_wrap(
    ~ homeownership, #<<
    nrow = 3  #<<
  )       

• If you don’t want to put all three histograms in one single plot, remember you can use faceting trick.
• first again I fill in with colors based on homeownership
• Then instead of changing alpha level, and put them on top of each other, I can facet the plot.
• Since I am using a single categorical variable to facet, I am going to use facet_wrap() function by homeownership variable and I want the data to be represented in 3 rows, so that the three histograms share the same x-axis and are compared easily. - Basically, there are not many house owners in the data set. And basically all three distributions are right-skewed.

13-Visualization

In lab.qmd ## Lab 13 section,

• Import the data penguins.csv.

• Generate the following

# library(tidyverse)
penguins <- read_csv(__________________)
________ |> ggplot(_______________________) +  ## mapping layer  
    ___________________ +  ## geometry layer
    _____________________________  ## label layer

________ |> ggplot(______________________________) +  ## mapping layer  
    _______________ +  ## geometry layer
    _______________ +  ## label layer
    ______________________________  +   ## facet layer
    ______________________________      ## theme layer (set legend.position = "none")

Density Plot

• geom_density() uses kernel density estimation to smooth the histogram or our data. (MATH 4750 Statistical Computing)

ggplot(loans, aes(x = loan_amount)) +
    geom_density()  #<<

• Let’s continue. Another way we can visualize numerical data is using a density plot.
• A density plot is basically a smoothed version of a histogram.
• Remember that a continuous random variable has a continuous probability distribution that is a density curve as a plot. Right?
• So a normal r.v. has a normal density curve that this.
• But given a data set of continuous variable, we can only draw a histogram that is sort of an approximation to its density curve. The histogram will only be very much like its true density plot when the data size is huge or approaching to infinity.
• A density plot here smooths the histogram, telling us that given the data set we have, what the true density curve might look like.
• In ggplot, we have geometry geom_density() to create a density plot.
• And ggplot uses the so-called kernel density estimation to smooth the histogram. The kernel is not a the linux kernel in computer science. The kernel here is a probability distribution.
• If you are interested, it will be taught in 4750 Computational Statistics which is a new course starting this fall.
• You can see in the graph that the density curve smooths the histogram, but it;s still not very smooth, right. It’s a little bit jagged.
• The ggplot choose a level of smoothness for us. And we can adjust the smoothness by ourselves.

Density Plots and Adjusting Bandwidth

• adjust = 0.5
• adjust = 1
• adjust = 2

ggplot(loans, aes(x = loan_amount)) +
    geom_density(adjust = 0.5)

ggplot(loans, aes(x = loan_amount)) +
    geom_density(adjust = 1) # default bandwidth

ggplot(loans, aes(x = loan_amount)) +
    geom_density(adjust = 2)

• We can adjust the smoothness of a density plot by the argument adjust in geom_density() function.
• basically, the larger the value is, the smoother the curve will be.
• and the default is adjust = 1.
• Keep in mind that if you use a very smooth density plot, some local peaks or features may be washed out or smoothed away. If these local things contain some information, you may lose the information by just looking at the smooth density plot.
• So a better idea is to try different level of smoothness, and check different plots, such as histogram, bar plots, and scatter plots as well.

Customizing Density Plots

loans |> 
  ggplot(
    aes(x = loan_amount)
  ) +
  geom_density(
    adjust = 2,
    fill = "#FFCC00", 
    color = "#003366", 
    alpha = 0.5, 
    linetype = "dashed"
  ) + 
  labs(
    x = "Loan amount ($)", 
    y = "Density", 
    title = "Lending Club loans"
  ) 

• ggplot2::geom_density()

• We can also customize density plots as others.
• In addition to adjust, we can also use fill, color, alpha, line type as we have used in histogram.

Density Curve on a Histogram

loans |> 
  ggplot(
    aes(x = loan_amount)
  ) + 
  ## scale down to 
  ## match the density
  geom_histogram(
    binwidth = 5000, 
    aes(y = ..density..) #<<
  ) + 
  geom_density(
    alpha = 0.1, 
    fill = "#FF6666"
  ) +
  labs(
    x = "Loan amount ($)",
    y = "Density",
    title = "Lending Club loans"
  )

• Sometimes we like to put the density on top of the histogram to have a better understanding how smooth the density curve is, comparing to the histogram.
• Well it’s very easy to do that.
• We just need to add the geom_density layer to the existent geom_histogram layer.
• And don’t forget to use density scale in geom_histogram so that the height of histogram and density plot can share the same y-axis each other.

Adding a Categorical Variable to Density Plots

loans |> 
  ggplot(
    aes(x = loan_amount,
        fill = homeownership)
  ) +
  geom_density(
    adjust = 2,
    alpha = 0.4
  ) +
  labs(
    x = "Loan amount ($)",
    y = "Density",
    title = "Amounts of loans",
    fill = "Homeownership"
  )

• We can also add a categorical variable to a density plot. This is similar to what we did with the histograms, where you can see them overlaid on top of each other.
• So once again, we can fill in the density curves based on the value of homeownership.
• and I am changing alpha, so that we can actually see what’s happening behind the curve. If I didn’t change alpha, we are not able to see the peaks of mortagage and own because they are masked by the density curve of home renters. OK.

Box Plot

loans |> ggplot(aes(x = interest_rate)) +
  geom_boxplot() +
  labs(x = "interest rate (%)")

• Remember in base R, we can use boxplot function to create a boxplot.
• In ggplot, we add the geometry layer, geom_boxplot().
• One of things that boxplots are good at is uncovering our potential outliers, and here, the distribution of interest rates is right-skewed, and has several outliers.
• Most of the interest rates are between 10 to 15%, but some can be as high as 30% or more.

Customizing Box Plots

loans |> 
  ggplot(aes(x = interest_rate)) +
  geom_boxplot(
    # custom outliers
    outlier.colour = "red",
    outlier.shape = 8,
    outlier.size = 3,
    # Notch?
    notch = TRUE,
    notchwidth = 0.1,
    # custom boxes
    fill = "#FFCC00",
    colour = "#003366",
    alpha = 0.2
  ) +
  labs(
    x = "Interest rate (%)",
    title = "Interest rates of loans"
  ) +
  theme(
    axis.ticks.y = element_blank(),
    axis.text.y = element_blank()
  )

• ggplot2::geom_boxplot()

• How do we customize our boxplot?
• For a boxplot, we can change outlier’s color, shape and size, using outlier.colour, outlier.shape and outlier.size.
• I remembered I show you point shape and its corresponding number when I introduced base R plotting. You can use the same point shape numbering system here.
• You can also decide if the boxplot has a notch, although I don’t see the importance of this option.
• And again you can customize the box using fill, color, and alpha argument.
• And one interesting part is that, when we show a boxplot like this, the y-axis does not mean anything right? the width or height of the box does not meaning anything.
• So we can actually remove the y axis tick marks and labels.
• And how? Because the axes labels and ticks are part of plotting theme, we should go to theme and set ticks.y and text.y = element_blank().
• This is very ggplot2 syntax. It is a little bit harder to read and make sense of, so it does take time to get used to it.

Adding a Categorical Variable to Box Plots

loans |> 
  ggplot(
    aes(x = interest_rate,
        y = grade) #<<
  ) + 
  geom_boxplot() +
  labs(
    x = "Interest rate (%)",
    y = "Grade",
    title = "Interest Rates of Loans",
    subtitle = "by grade of loan"
  )

• We can also add categorical variable to a boxplot, so these are called side-by-side boxplots. They are very commonly used.
• This is the interest rate distribution by the grade of loan, where A is of the highest quality and G is the lowest quality.
• And it looks like the higher the quality of the loan, the lower the interest rate it has.
• We can also see that there is an outlier in grade D data, where the interest rate is super low, even lower than most of the interest rate of the grade A loans. You as a reasearcher or data analyst have to figure out if it is typo, or it is the true interest rate. If it is a correct number, why such low interest rate happened?
• And to create such side-by-side boxplot, we just add y mapping, setting it to grade.

Adding Two Categorical Variables to Box Plots

loans |> 
  ggplot(
    aes(x = interest_rate,
        y = grade,
        fill = homeownership) #<<
  ) +
  geom_boxplot() +
  labs(
    x = "Interest rate (%)",
    y = "Grade",
    fill = "Homeowership",
    title = "Interest Rates of Loans",
    subtitle = "by grade and ownership"
  ) +
  theme(
    legend.position = "bottom"
  )

• If we want to add two categorical variables to box plots, that’s easy.
• We can fill the boxplots with colors based on another variable, here the homeownership.
• ANd now each grade of loans is furthered divided into three categories of the homeownership variable.
• And we can see that basically, the type of homeownership does not affect interest rate much given the same grade of loans.

Adding Points on Box Plots

ggplot(loans, aes(x = homeownership, y = interest_rate)) +
    geom_boxplot() +
    geom_point(alpha = 0.1, shape = 1) #<< geom_jitter(width = 0.2, alpha = 0.1, shape = 1)

• If we don’t just want the boxplot, and we also want to show data points along with the boxplot, we can add the another layer geom_point(), as shown here.
• But this plot is a little misleading. Anybody see why?
• The boxplot tell us 50% of the data points should be inside the box, right?
• But according to the boxplot, it is not the case. Why?
• Actually, there are lots points overlapped at the same interest rate. So it looks like there is only one data point, but actually there are many data points right there. They just overlapped each other.
• So how do we actually show all data points?

Relationships between numerical variables

Scatterplot

ggplot(loans, aes(x = debt_to_income, y = interest_rate)) +
    geom_point(shape = 23, fill = "blue", size = 0.8)  #<<

• We already talked about scatter plot using geom_point().
• Here we have a scatter plot of interest rate and the debt to income ratio.
• One potential problem here is that like adding points to boxplot, we have so many data points overlapped together, and their relationship cannot be truly revealed. and so it’s really difficult to make sense of this scatter plot built with geom_point.

Hex Plot

ggplot(loans, aes(x = debt_to_income, y = interest_rate)) +
    geom_hex()  #<< (hexbin pkg)

• So one solution is using Hex Plot instead by using geom_hex() function.
• The idea is that we first bin these data with the hex shape, grouping data points together for each hex bin.
• And the color of these hexes is what tells us how many data points or how dense my data are.
• So the lighter the color, the more data points we have there.
• This is particularly useful when we have lots of data points that have similar or the same values.

Hex Plot Zoom-in

loans |> 
    filter(debt_to_income < 100) |> 
    ggplot(aes(x = debt_to_income, y = interest_rate)) +
    geom_hex() +
    viridis::scale_fill_viridis()

• Here just show you that we can further zoom in the plot, focusing on the the data that has debt_to_income ratio less that 100%.
• And now we can more clearly see what’s going on.
• Basically, more data points happen around debt_to_income ratio 12 to 15% and interest rate 10%.
• And the two variables are a little bit positively correlated that higher debt_to_income ratio corresponds to higher interest rate. But again the correlation is very weak here.

Other Plots

Line Chart for Time Series ggplot2::geom_line()

economics_long |> ggplot(aes(x = date, y = value01, 
                             colour = variable, linetype = variable)) +
    geom_line(linewidth = 1) + 
    theme_bw() +
    theme(legend.position = "bottom")

• If you have time series data, line plot is your good friend, which shows the trend of some variable.
• For example here, the unemployment number.

QQ-plots

• Quantile-Quantile plots are used to check if data are normally distributed (or follow any distribution).

ggplot(mpg, aes(sample = hwy)) + geom_qq() + geom_qq_line()

• To create a qqplot, we add the geometry geom_qq(), and geom_qq_line() if you want to add a straight line that helps you determine whether the data are normally distributed.

Violin Plots

• Violin plots are similar to box plots, but show the smooth density of the data.

f <- ggplot(loans, aes(x = loan_amount, y = grade))

f + geom_boxplot()

f + geom_violin() 

• In ggplot, we can use geom_violin() to create one.
• See if we just check the boxplot, we will miss the information that the loan amount of the grade G actually has a bimodal distribution, right?

Add-on 📦: ggridges for Ridge Plots

library(ggridges)
ggplot(loans, aes(x = loan_amount, y = grade, fill = grade, color = grade)) +
    geom_density_ridges(alpha = 0.9)

• To show better smooth density plots, we can create the so-called ridge plots provided by the add-on package called ggridges.
• everything is basically the same. You just need to install and load the package, and add the geometry called geom_density_ridges().

Add-on 📦: ggrepel

• ggrepel provides geoms for ggplot2 to repel overlapping text labels.

library(ggrepel)
p <- mtcars |> filter(wt > 2.75 & wt < 3.45) |> rownames_to_column("car") |> 
    ggplot(aes(wt, mpg, label = car)) +
    geom_point(color = "red")

p + geom_text() + 
    labs(title = "geom_text()")

p + geom_text_repel() + 
    labs(title = "geom_text_repel()")

• Another useful ggplot2 extension is ggrepel that provides geoms for ggplot2 to repel overlapping text labels.
• This makes your labels much more clear.
• For example, here I want to label each data point by its observation’s name, I can add geom_text(). But the ggplot2 setting basically put the text right on the top of the point, which sometimes the labels are mixed together and makes it hard to read.
• And we can use the function geom_text_repel(), and then it will automatically find a good spot for those texts that are close to their data point, but not overlap each other.

Wordcloud from ggwordcloud Package

• Plot
• Code

library(ggwordcloud)
library(showtext)
where <- font_files()[which(str_detect(font_files()$family, "Arial Unicode MS")), ]
thankyou_words_small |> ggplot(aes(label = word, size = speakers, color = name)) + 
    geom_text_wordcloud(area_corr = TRUE, rm_outside = TRUE, 
                        family = where[1, ]$family) +
    scale_size_area(max_size = 24) + 
    theme_minimal() +
    theme(plot.margin = margin(t = 0,  # Top margin
                               r = 0,  # Right margin
                               b = 0,  # Bottom margin
                               l = 0)) # Left margin

https://towardsdatascience.com/create-a-word-cloud-with-r-bde3e7422e8a http://www.sthda.com/english/wiki/text-mining-and-word-cloud-fundamentals-in-r-5-simple-steps-you-should-know http://www.sthda.com/english/wiki/word-cloud-generator-in-r-one-killer-function-to-do-everything-you-need https://semba-blog.netlify.app/10/29/2019/creating-wordcloud-in-r/ https://stackoverflow.com/questions/74415534/why-do-characters-from-foreign-alphabets-not-show-in-my-wordcloud-on-r

Radar Chart from fmsb and ggradar Package

• Plot
• Code

library(fmsb)
radar_data <- readr::read_csv(
    file = "./data/radar_data.csv")
# Color vector
colors_border <- c(rgb(0.2,0.5,0.5,0.9), 
                   rgb(0.8,0.2,0.5,0.9), 
                   rgb(0.7,0.5,0.1,0.9))
colors_in <- c(rgb(0.2,0.5,0.5,0.4), 
               rgb(0.8,0.2,0.5,0.4), 
               rgb(0.7,0.5,0.1,0.4))
radarchart(radar_data, axistype = 1, 
           #custom polygon
           pcol = colors_border, 
           pfcol = colors_in, 
           plwd = 4, plty = 1,
           #custom the grid
           cglcol = "grey", cglty = 1, 
           axislabcol = "grey", 
           caxislabels = seq(0, 20, 5), 
           cglwd = 0.8,
           #custom labels
           vlcex = 1.2)

# legend("topright", legend = rownames(radar_data[-c(1, 2), ]), bty = "n", pch = 20 , 
#        col = colors_in, text.col = "grey", cex = 1.2, pt.cex = 3)

library(ggradar)

ggradar_data <- radar_data |>
    as_tibble(rownames = "group") |>
    mutate_at(vars(-group), rescale) |>
    tail(3)

ggradar(ggradar_data,
        base.size = 5,
        grid.label.size = 6,
        axis.label.size = 5,
        group.point.size = 3,
        fill.alpha = 0.2,
        grid.line.width = 0.4,
        plot.legend = FALSE,
        fill = TRUE)

lazy-load database ‘/Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/library/callr/R/callr.rdb’ is corrupt Restart R

Network from igraph Package

• Plot
• Code

library(igraph)
network_data <- read_rds(file = "./data/network_data.rds")

# build the graph object
network <- graph_from_adjacency_matrix(network_data)
 
# plot it
plot(network)

More R Graphics Resources

• More add-on 📦: ggplot2 extensions
• The R Graph Gallery
• R Graphics Cookbook
• R CHARTS

• As I mentioned before, there are tons of ggplot2 extensions.
• If you are interested, check those links, and learn more about them. OK.