Data Science Overview 📖

MATH/COSC 3570 Introduction to Data Science

Dr. Cheng-Han Yu
Department of Mathematical and Statistical Sciences
Marquette University

What is Data Science or a Data Scientist?

A Little History of Data Science

• In 1985, Dr. Jeff Wu used the term Data Science for the first time as an alternative name for Statistics.
• In 1997, Dr. Wu calls for statistics to be renamed data science and statisticians to be renamed data scientists.
• In 2001, Dr. William S. Cleveland advocated an expansion of statistics beyond theory into six technical areas, and the altered expanded field will be called data science.

• The first question is What is DS? Actually this is a really hard question to answer, at least at this moment.

• 1985. Originally, DS was proposed to replace statistics. DS is just another name or maybe better name for statistics.
• 1997. But still at that time, nobody cares. Statisticians are weird guys in math departments who deal with proofs and theorems and sometimes real data, and they are stubborn, not willing to change. But if you wiki the definition of statistics, statistics is a science of data, and you tell me what is the difference of science of data and data science?
• 2001. Using this definition, DS becomes a broader term than statistics. Anything related to data can be a part of data sicence, while statistics now becomes a narrower subject focusing on inference and data analysis part.

• Here shows the google trend of DS. When I was in college or beginning of PHD studies, very few people used DS as a term. And all of sudden, DS become more and more popular, and everybody is talking about it.

Source: https://www.reddit.com/r/meme/comments/floq3q/reality_behind_data_science/

Source: https://br.ifunny.co/picture/we-will-work-together-statistics-computer-science-please-teach-now-h4hdtthT9?s=cl

😕 Still what on earth is data science?

Battle of the Data Science Venn Diagrams

• 2010 Drew Conway

• 2012 Brendan Tierney

Battle of the Data Science Venn Diagrams

• 2013 Ulrich Matter

• 2013 Joel Grus

• Starting 2010, people tried to use venn diagrams to define DS, what should be include, what should be not. It’s a subset of something, or superset of something.

• We have to do DS on some subject/field.

• (left) ML use CS and STAT but doesn’t care about domain knowledge.

• If you apply CS to substantive expertise without solid math and stats, you are doing bullshit basically.

• (right) DS is a subset of KDD - Knowledge Discovery in Data

Battle of the Data Science Venn Diagrams

• 2013 Harlan Harris

• 2014 Steven Geringer

Battle of the Data Science Venn Diagrams

• 2014 Michael Malak

• 2015 Stephan Kolassa

Battle of the Data Science Venn Diagrams

• 2016 Gregory Piatetsky-Shapiro

• 2017 Longbing Cao

Shall We Continue?

• totally-not-a-Venn-diagram by Calvin Andrus

• You probably get the idea. There are so many ways to define data science.

Nobody Knows What Data Science (Scientist) is

• 💰 💰 Dr. Grant (Statistics Education Research Journal, 2017): Data science is the field of people who decide to print “Data Scientist” on their business cards to get a salary bump! 💵 💵

What Wiki Defines

Data Science in This Course

• Data science is an discipline that allows us to turn raw data into understanding, insight, and knowledge.

• We’re going to learn to do this in a tidy way – more on that later!

• This is a introductory data science course with an emphasis on important tools in R/Python that help us do data science.

A Data Science Project

Data Science Workflow

• Import: Take data stored somewhere and load it into your workspace.
• Tidy: Storing data in a consistent rectangular form, i.e., a data matrix.
• Transform: Narrowing in on observations of interest, creating new variables, calculating statistics.

• To do DS, we need data, so the first step is to import data (——).

• But usually the raw data are very messy, so we need tidy the data by storing data in a consistent rectangular form (data frame in R), which is easy for modeling and analysis.

• Depending on our research goal, we may just want to focus on some variables , or observations in the data, or we may want to create some variables for example, the data use feet, but we want to use meter instead. This data transformation further cleans the data, so that we can extract any information we need from data more easily and efficiently. In other words, we improve the quality of data.

Data Matrix

• Each row corresponds to a unique case or observational unit.
• Each column represents a characteristic or variable.
• This structure allows new cases to be added as rows or new variables as new columns.

ggplot2::mpg |> print(n = 10)

# A tibble: 234 × 11
   manufacturer model      displ  year   cyl trans drv     cty   hwy fl    class
   <chr>        <chr>      <dbl> <int> <int> <chr> <chr> <int> <int> <chr> <chr>
 1 audi         a4           1.8  1999     4 auto… f        18    29 p     comp…
 2 audi         a4           1.8  1999     4 manu… f        21    29 p     comp…
 3 audi         a4           2    2008     4 manu… f        20    31 p     comp…
 4 audi         a4           2    2008     4 auto… f        21    30 p     comp…
 5 audi         a4           2.8  1999     6 auto… f        16    26 p     comp…
 6 audi         a4           2.8  1999     6 manu… f        18    26 p     comp…
 7 audi         a4           3.1  2008     6 auto… f        18    27 p     comp…
 8 audi         a4 quattro   1.8  1999     4 manu… 4        18    26 p     comp…
 9 audi         a4 quattro   1.8  1999     4 auto… 4        16    25 p     comp…
10 audi         a4 quattro   2    2008     4 manu… 4        20    28 p     comp…
# ℹ 224 more rows

• The process of importing, tidying and transforming data is called data wrangling or data munging.

• Visualisation: A good visualisation shows you things that you did not expect or raise new questions about the data.

• Once the data are ready, it’s time to explore our data.

• My recommendation is that always plot your data before building any model or algorithm or doing analysis because good visualization let you understand more about your data, and may provide some hints about how to build your statistical model, machine learning algorithms or how to do analysis.

• And good visualization shows you things that you did not expect or raise new questions about the data. (outliers, missing data)

• So after visualizing your data, you may have better idea of how to transforming your data so that you can be better prepared for modeling and answer your research questions. (nonlinear relationship <-> linear relationship)

mpg |> ggplot(aes(x = displ, y = hwy)) +
    geom_point(aes(color = class)) + 
    geom_smooth() + 
    theme_bw()

• Model: Models are complementary tools to visualisation. Once you have made your questions sufficiently precise, you can use a model to answer them.

• The next step is to use or build a rigorous statistical or machine learning model to answer our research questions. For example, forecasting stock prices, classifying images.

• And transform-visualize-model becomes a loop of understanding or learning from data. We keep transforming, visualizing and modeling our data until our questions are properly answered by our data and model.

library(tidymodels)
linear_reg() |>  
    set_engine("lm") |> 
    fit(hwy ~ displ, data = mpg)

parsnip model object


Call:
stats::lm(formula = hwy ~ displ, data = data)

Coefficients:
(Intercept)        displ  
      35.70        -3.53  

• Communication: It doesn’t matter how well your models and visualization have led you to understand the data unless you can also communicate your results to others.

• Finally we have to share our analysis results with others.

• At this stage, visualization plays an important role as well.

• Figures/tables speak louder than words.

• We can have different types of deployment. In addition to papers or reports, we can create a website, a software package, a mobile app, dashboard and so on.

• Programming: Surrounding all these tools is programming.

• Of course, every aspect of DS is done by programming. DS cannot be done by pieces of paper and pens. It has to be done using computers.

R for Data Science

Source: https://teachdatascience.com/tidyverse/

• If you are ready, let’s dive in and learn all these tools. Well not all, but some of these tools.

• By the way, you see these hex stickers? It kind of becomes a trend that when we create an R package, or software, we also create a hex logo associated with it, so it works as product or brand logo. It may be easier for you to promote your package if you have a cute or beautiful hex logo.

Python for Data Science

Source: https://www.e2enetworks.com/blog/9-python-libraries-for-data-science-and-artificial-intelligence