12  Billboard Hot100

Welcome to Case Studies

You’ve now completed the foundation chapters covering R basics, data wrangling, analysis, and visualization. The following case studies put all these skills together in realistic journalism scenarios.

Each case study increases in complexity, building on skills from previous chapters. Work through them in order for the best learning experience, or jump to a case study that interests you and reference earlier chapters as needed.

12.1 Load Packages

library(tidyverse)

── Attaching core tidyverse packages ──────────────────────── tidyverse 2.0.0 ──
✔ dplyr     1.1.4     ✔ readr     2.1.5
✔ forcats   1.0.1     ✔ stringr   1.6.0
✔ ggplot2   4.0.0     ✔ tibble    3.3.0
✔ lubridate 1.9.4     ✔ tidyr     1.3.1
✔ purrr     1.2.0     
── Conflicts ────────────────────────────────────────── tidyverse_conflicts() ──
✖ dplyr::filter() masks stats::filter()
✖ dplyr::lag()    masks stats::lag()
ℹ Use the conflicted package (<http://conflicted.r-lib.org/>) to force all conflicts to become errors

12.2 Import Data

This dataset is downloaded from Kaggle, which contains the Billboard Hot 100 chart data from 1958 to 2024.

df <- read_csv("data/hot100.csv")

Rows: 345887 Columns: 8
── Column specification ────────────────────────────────────────────────────────
Delimiter: ","
chr  (4): Song, Artist, Weeks in Charts, Image URL
dbl  (3): Rank, Last Week, Peak Position
date (1): Date

ℹ Use `spec()` to retrieve the full column specification for this data.
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.

12.3 Data Inspection

str(): Structure of the Data

str(df)

spc_tbl_ [345,887 × 8] (S3: spec_tbl_df/tbl_df/tbl/data.frame)
 $ Date           : Date[1:345887], format: "1958-08-06" "1958-08-06" ...
 $ Song           : chr [1:345887] "Poor Little Fool" "Nel Blu Dipinto Di Blu (Volare)" "Patricia" "Splish Splash" ...
 $ Artist         : chr [1:345887] "Ricky Nelson" "Domenico Modugno" "Perez Prado And His Orchestra" "Bobby Darin" ...
 $ Rank           : num [1:345887] 1 2 3 4 5 6 7 8 9 9 ...
 $ Last Week      : num [1:345887] 1 54 2 3 5 8 4 6 12 9 ...
 $ Peak Position  : num [1:345887] 1 2 2 3 5 6 4 6 9 9 ...
 $ Weeks in Charts: chr [1:345887] "2" "2" "2" "2" ...
 $ Image URL      : chr [1:345887] "#" "https://charts-static.billboard.com/img/1958/08/domenico-modugno-x2d-180x180.jpg" "#" "https://charts-static.billboard.com/img/1958/08/bobby-darin-hm6-180x180.jpg" ...
 - attr(*, "spec")=
  .. cols(
  ..   Date = col_date(format = ""),
  ..   Song = col_character(),
  ..   Artist = col_character(),
  ..   Rank = col_double(),
  ..   `Last Week` = col_double(),
  ..   `Peak Position` = col_double(),
  ..   `Weeks in Charts` = col_character(),
  ..   `Image URL` = col_character()
  .. )
 - attr(*, "problems")=<externalptr> 

Here is the description for each column:

• Date: the date of the chart
• Song: the name of the song
• Artist: the name of the artist
• Rank: the position of the song on the chart
• Last Week: the position of the song on the previous week’s chart
• Peak Position: the highest position the song reached on the chart
• Weeks in Chart: the number of weeks the song has been on the chart
• Image URL: the URL of the song’s cover image

summary(): Summary of the Data

summary(df)

      Date                Song              Artist               Rank      
 Min.   :1958-08-06   Length:345887      Length:345887      Min.   :  1.0  
 1st Qu.:1975-03-05   Class :character   Class :character   1st Qu.: 26.0  
 Median :1991-10-09   Mode  :character   Mode  :character   Median : 51.0  
 Mean   :1991-10-05                                         Mean   : 50.5  
 3rd Qu.:2008-05-07                                         3rd Qu.: 76.0  
 Max.   :2024-12-04                                         Max.   :100.0  
   Last Week      Peak Position    Weeks in Charts     Image URL        
 Min.   :  1.00   Min.   :  1.00   Length:345887      Length:345887     
 1st Qu.: 25.00   1st Qu.:  7.00   Class :character   Class :character  
 Median : 51.00   Median : 28.00   Mode  :character   Mode  :character  
 Mean   : 50.52   Mean   : 32.95                                        
 3rd Qu.: 76.00   3rd Qu.: 55.00                                        
 Max.   :100.00   Max.   :100.00                                        

check missing values

# check missing values
df |>
  summarise_all(~sum(is.na(.)))

12.4 Data Cleaning

Note

Do we need to clean the data? any issue with the column names? data types?

Rename Columns

# rename all columns to be all lowercase, and ideally single words, no spaces, or snake_case

df1 <- df |>
  rename(date = Date,
         song = Song,
         artist = Artist,
         rank = Rank,
         last_week = `Last Week`,
         peak_position = `Peak Position`,
         weeks_in_charts = `Weeks in Charts`,
         image_url = `Image URL`)

# check column names and data types
str(df1)

spc_tbl_ [345,887 × 8] (S3: spec_tbl_df/tbl_df/tbl/data.frame)
 $ date           : Date[1:345887], format: "1958-08-06" "1958-08-06" ...
 $ song           : chr [1:345887] "Poor Little Fool" "Nel Blu Dipinto Di Blu (Volare)" "Patricia" "Splish Splash" ...
 $ artist         : chr [1:345887] "Ricky Nelson" "Domenico Modugno" "Perez Prado And His Orchestra" "Bobby Darin" ...
 $ rank           : num [1:345887] 1 2 3 4 5 6 7 8 9 9 ...
 $ last_week      : num [1:345887] 1 54 2 3 5 8 4 6 12 9 ...
 $ peak_position  : num [1:345887] 1 2 2 3 5 6 4 6 9 9 ...
 $ weeks_in_charts: chr [1:345887] "2" "2" "2" "2" ...
 $ image_url      : chr [1:345887] "#" "https://charts-static.billboard.com/img/1958/08/domenico-modugno-x2d-180x180.jpg" "#" "https://charts-static.billboard.com/img/1958/08/bobby-darin-hm6-180x180.jpg" ...
 - attr(*, "spec")=
  .. cols(
  ..   Date = col_date(format = ""),
  ..   Song = col_character(),
  ..   Artist = col_character(),
  ..   Rank = col_double(),
  ..   `Last Week` = col_double(),
  ..   `Peak Position` = col_double(),
  ..   `Weeks in Charts` = col_character(),
  ..   `Image URL` = col_character()
  .. )
 - attr(*, "problems")=<externalptr> 

Convert Data Types

# weeks_in_chart should be numeric
df2 <- df1 |>
  mutate(weeks_in_charts = as.numeric(weeks_in_charts))

Warning: There was 1 warning in `mutate()`.
ℹ In argument: `weeks_in_charts = as.numeric(weeks_in_charts)`.
Caused by warning:
! NAs introduced by coercion

# check data types
str(df2)

tibble [345,887 × 8] (S3: tbl_df/tbl/data.frame)
 $ date           : Date[1:345887], format: "1958-08-06" "1958-08-06" ...
 $ song           : chr [1:345887] "Poor Little Fool" "Nel Blu Dipinto Di Blu (Volare)" "Patricia" "Splish Splash" ...
 $ artist         : chr [1:345887] "Ricky Nelson" "Domenico Modugno" "Perez Prado And His Orchestra" "Bobby Darin" ...
 $ rank           : num [1:345887] 1 2 3 4 5 6 7 8 9 9 ...
 $ last_week      : num [1:345887] 1 54 2 3 5 8 4 6 12 9 ...
 $ peak_position  : num [1:345887] 1 2 2 3 5 6 4 6 9 9 ...
 $ weeks_in_charts: num [1:345887] 2 2 2 2 2 2 2 2 2 2 ...
 $ image_url      : chr [1:345887] "#" "https://charts-static.billboard.com/img/1958/08/domenico-modugno-x2d-180x180.jpg" "#" "https://charts-static.billboard.com/img/1958/08/bobby-darin-hm6-180x180.jpg" ...

Can you answer these questions?

• How many rows/columns in the dataset?
• What are the column names?
• What are the data types of each column?

12.5 Data Analysis

# rename df2 as hot100

hot100 = df2
# save(hot100, file = "data/hot100.RData")

Single Variable Descriptive Statistics

# oldest and newest date
summary(hot100$date)

        Min.      1st Qu.       Median         Mean      3rd Qu.         Max. 
"1958-08-06" "1975-03-05" "1991-10-09" "1991-10-05" "2008-05-07" "2024-12-04" 

# which artist has most appearances in the hot-100 charts?
table(hot100$artist) |>
  sort(decreasing = TRUE) |>
  head(10)

 Taylor Swift         Drake    Elton John       Madonna Kenny Chesney 
         1574           932           888           856           782 
   Tim McGraw Morgan Wallen   Keith Urban Stevie Wonder   Rod Stewart 
          749           731           674           659           657 

Multiple Variables

Which artist appeared most

hot100 |>
  filter(rank == 1, date > "2015-01-01") |>
  distinct(artist, song) |>
  group_by(artist) |>
  summarise(n = n()) |>
  arrange(-n) |>
  head(10) |>
  ggplot(aes(x = artist, y = n)) +
  geom_col() +
  coord_flip()

Which song (song & artist) has been on the charts the most?

hot100 |>
  group_by(song, artist) |>
  summarize(appearance = n()) |>
  arrange(-appearance) |>
  head(10)

`summarise()` has grouped output by 'song'. You can override using the
`.groups` argument.

Which song (song & artist) was No. 1 for the most number of weeks?

hot100 |>
  filter(rank == 1) |>
  group_by(song, artist) |>
  summarize(appearance = n()) |>
  arrange(-appearance) |>
  head(10)

`summarise()` has grouped output by 'song'. You can override using the
`.groups` argument.

Which artist has the most No. 1 songs in last 10 years?

hot100 |>
  filter(rank == 1, year(date) >= 2015) |>
  distinct(artist, song) |>
  count(artist, sort = TRUE) |>
  head(10)

Note

count(artist, sort = TRUE) is equivalent to group_by(artist) |> summarize(n = n()) |> arrange(desc(n))

12.6 Data Visualization

Bar Chart

Bar chart is used to show the frequency of each category. Bar chart is a good choice when you want to show the relationship between a numerical variable and a categorical variable (e.g., song, artist).

# bar chart to show the top 10 artists with most appearances in the hot-100 charts
hot100 |> 
  filter(rank == 1, year(date) >= 2000) |> 
  group_by(artist) |> 
  summarize(appearance = n()) |> 
  arrange(-appearance) |> 
  head(10) |> 
  ggplot(aes(x = reorder(artist, appearance), y = appearance, fill = appearance)) + 
  geom_col(show.legend = FALSE) + 
  geom_text(aes(label = appearance), hjust = -0.2, size = 3) + 
  scale_fill_gradient(low = "lightblue", high = "darkblue") + 
  labs(
    title = "Top 10 Artists with Most Billboard No.1 Hits in the 21st Century",
    subtitle = str_wrap("Adele, Taylor Swift, and Mariah Carey are the top 3 artists with the most Billboard No.1 Hits in the 21st century."),
    x = "Artist",
    y = "Number of No. 1 Songs",
    caption = "Source: Billboard | Author: Bin Chen"
  ) +
  coord_flip() + 
  theme_minimal(base_size = 10) +
  theme(
    plot.title = element_text(face = "bold", size = 12),
    plot.subtitle = element_text(size = 10, color = "gray30"),
    plot.caption = element_text(size = 10, color = "gray50", hjust = 0)
  )

Line Chart

# Get top 10 artists with the most No. 1 songs (each song counted only once)
top10_artists <- hot100 |>
  filter(rank == 1) |>
  distinct(artist, song) |>  # Ensure each song is counted only once per artist
  group_by(artist) |>
  summarize(appearance = n()) |>  # Count unique No. 1 songs per artist
  arrange(desc(appearance)) |>  # Sort by the number of unique songs
  slice_head(n = 10)  # Select the top 10 artists

# Print the result
top10_artists

# line chart to show the number of No. 1 songs for the top 10 artists
hot100 |>
  filter(rank == 1, artist %in% top10_artists$artist) |>
  group_by(artist, year = year(date)) |>
  summarize(appearance = n()) |>
  ggplot(aes(x = year, y = appearance, color = artist)) +
  geom_line() +
  geom_point() +
  labs(
    title = "Number of No. 1 Songs by Top 10 Artists",
    x = "Year",
    y = "Number of No. 1 Songs",
    color = "Artist",
    caption = "Source: Billboard | Author: Bin Chen"
  ) +
  theme_minimal(base_size = 10) +
  theme(
    plot.title = element_text(face = "bold", size = 12),
    plot.subtitle = element_text(size = 10, color = "gray30"),
    plot.caption = element_text(size = 10, color = "gray50", hjust = 0)
  )

`summarise()` has grouped output by 'artist'. You can override using the
`.groups` argument.

12.7 Key Functions Recap

Function	Package	Purpose	Example Use
read_csv()	readr	Read CSV files into R	read_csv("data/hot100.csv")
str()	Base R	Display data structure and types	str(df)
summary()	Base R	Generate summary statistics	summary(hot100$date)
summarise_all()	dplyr	Apply summary function to all columns	summarise_all(~sum(is.na(.)))
is.na()	Base R	Test for missing values	is.na(df)
rename()	dplyr	Rename columns in a data frame	rename(date = Date, song = Song)
mutate()	dplyr	Create or modify columns	mutate(weeks_in_charts = as.numeric(weeks_in_charts))
as.numeric()	Base R	Convert values to numeric type	as.numeric(weeks_in_charts)
filter()	dplyr	Filter rows based on conditions	filter(rank == 1, year(date) >= 2015)
distinct()	dplyr	Return distinct rows	distinct(artist, song)
group_by()	dplyr	Group data by one or more variables	group_by(artist)
summarise()	dplyr	Compute summary statistics for groups	summarise(n = n())
arrange()	dplyr	Sort rows by one or more columns	arrange(-appearance)
count()	dplyr	Count occurrences of unique values	count(artist, sort = TRUE)
slice_head()	dplyr	Select first n rows from each group	slice_head(n = 10)
head()	Base R	Get first n rows of data	head(10)
table()	Base R	Build contingency table of counts	table(hot100$artist)
sort()	Base R	Sort vector elements	sort(decreasing = TRUE)
year()	lubridate	Extract year component from date	year(date)
ggplot()	ggplot2	Create a new ggplot object	ggplot(aes(x = artist, y = appearance))
aes()	ggplot2	Specify aesthetic mappings (x, y, color, fill)	aes(x = reorder(artist, appearance), y = appearance, fill = appearance)
geom_col()	ggplot2	Create a bar chart	geom_col(show.legend = FALSE)
geom_line()	ggplot2	Create a line plot	geom_line()
geom_point()	ggplot2	Add points to a plot	geom_point()
geom_text()	ggplot2	Add text labels to a plot	geom_text(aes(label = appearance), hjust = -0.2, size = 3)
coord_flip()	ggplot2	Flip x and y axes	coord_flip()
scale_fill_gradient()	ggplot2	Create gradient color scale for fill	scale_fill_gradient(low = "lightblue", high = "darkblue")
labs()	ggplot2	Add titles, labels, and captions	labs(title = "...", x = "Artist", y = "Number of No. 1 Songs")
str_wrap()	stringr	Wrap long strings for display	str_wrap("Adele, Taylor Swift, and Mariah Carey are the top 3 artists...")
theme_minimal()	ggplot2	Apply minimal theme	theme_minimal(base_size = 10)
element_text()	ggplot2	Customize text elements in themes	element_text(face = "bold", size = 12)
theme()	ggplot2	Customize plot appearance	theme(plot.title = element_text(face = "bold", size = 12))
reorder()	Base R	Reorder levels of a variable	reorder(artist, appearance)