data visualization episode 1st draft

Author: jt14den

config.yaml

@@ -71,6 +71,7 @@ episodes:
 - writing-functions.md
 - tidy.md
 - wrap.md
+- data-visualisation.md
 
 # Information for Learners
 learners: 

episodes/data-visualisation.md

@@ -0,0 +1,258 @@
+---
+title: 'Data Visualisation'
+teaching: 10
+exercises: 2
+---
+
+:::::::::::::::::::::::::::::::::::::: questions 
+
+- How can I use Python tools like Pandas and Plotly to visualize library circulation data?
+
+::::::::::::::::::::::::::::::::::::::::::::::::
+
+::::::::::::::::::::::::::::::::::::: objectives
+
+- Generate plots using Python to interpret and present data on library circulation.
+- Apply data manipulation techniques with pandas to prepare and transform library circulation data into a suitable format for visualization.
+- Analyze and interpret time-series data by identifying key trends and outliers in library circulation data.
+
+::::::::::::::::::::::::::::::::::::::::::::::::
+
+For this module, we will use a different version of our circulation data that is in a tidy data format, where each variable forms a column, each observation forms a row, and each type of observation unit forms a row. If your workshop included the Tidy Data episode, you should be set and have an object called `df_long` in your Jupyter environment. If not, we’ll read that dataset in now, as it was provided for this lesson.
+
+
+``` python
+#import if it is already not
+import pandas as pd
+```
+
+If `df_long` isn’t loaded already, we can read it in using `read_csv`. Note, if it isn’t present in your local `data/` folder.
+
+``` python
+df_long = pd.read_csv('data/circ_long.tsv', sep="\t")
+```
+
+We are using a couple of new parameters in our `read_csv` function above. Since the file we want to read in is a tab-separated values (TSV) file, we tell our `read_csv` function that using the `sep="\t"` parameter. `\t` is encoding for tab character in Python. Other separators or delimeters could include `|` or `;` depending on your data file.
+
+Let’s look at the data:
+
+``` python
+df_long.head()
+```
+
+``` output
+            branch                  address  ... circulation        date
+0     Albany Park     5150 N. Kimball Ave.  ...        8427  2011-01-01
+1         Altgeld    13281 S. Corliss Ave.  ...        1258  2011-01-01
+2  Archer Heights      5055 S. Archer Ave.  ...        8104  2011-01-01
+3          Austin        5615 W. Race Ave.  ...        1755  2011-01-01
+4   Austin-Irving  6100 W. Irving Park Rd.  ...       12593  2011-01-01
+```
+
+
+In order to plot this data over time we need to do two things to prepare it first. First, we need to tell Python that the data column is a datetime object using the Pandas `to_dateime` function. Second, we assign the date column as our index for the data. These two steps will set up our data for plotting.
+
+``` python
+df_long['date']= pd.to_datetime(df_long['date'])
+```
+
+The above converts our date column to a data time object. Let’s confirm it worked.
+
+``` python
+df_long.info()
+```
+
+
+``` output
+<class 'pandas.core.frame.DataFrame'>
+RangeIndex: 11556 entries, 0 to 11555
+Data columns (total 9 columns):
+#   Column       Non-Null Count  Dtype         
+ ---  ------       --------------  -----         
+ 0   branch       11556 non-null  object        
+ 1   address      7716 non-null   object        
+ 2   city         7716 non-null   object        
+ 3   zip code     7716 non-null   float64       
+ 4   ytd          11556 non-null  int64         
+ 5   year         11556 non-null  int64         
+ 6   month        11556 non-null  object        
+ 7   circulation  11556 non-null  int64         
+ 8   date         11556 non-null  datetime64[ns]
+dtypes: datetime64[ns](1), float64(1), int64(3), object(4)
+memory usage: 812.7+ KB
+```
+
+That worked! Now, we can make this datetime object our dataframe index.
+
+
+``` python
+df_long.set_index('date', inplace=True)
+```
+
+If we look at the data again, we will see our index will be set to date.
+
+``` python
+df_long.head()
+```
+
+``` output
+                     branch                  address  ...    month  circulation
+ date                                                 ...                      
+ 2011-01-01     Albany Park     5150 N. Kimball Ave.  ...  january         8427
+ 2011-01-01         Altgeld    13281 S. Corliss Ave.  ...  january         1258
+ 2011-01-01  Archer Heights      5055 S. Archer Ave.  ...  january         8104
+ 2011-01-01          Austin        5615 W. Race Ave.  ...  january         1755
+ 2011-01-01   Austin-Irving  6100 W. Irving Park Rd.  ...  january        12593
+
+```
+
+Ok! We are now ready to plot our data. Since this data is monthly data, we can plot the circulation data over time.
+
+At first, let’s focus on a specific branch. We can select the rows for the Albany Park branch:
+
+``` python
+albany = df_long[df_long['branch'] == 'Albany Park']
+```
+
+``` python
+albany.head()
+```
+
+``` output
+                   branch               address  ...    month  circulation
+  date                                           ...                      
+  2011-01-01  Albany Park  5150 N. Kimball Ave.  ...  january         8427
+  2016-01-01  Albany Park                   NaN  ...  january        10905
+  2017-01-01  Albany Park                   NaN  ...  january        11031
+  2022-01-01  Albany Park   3401 W. Foster Ave.  ...  january         5561
+  2018-01-01  Albany Park                   NaN  ...  january         9381
+```
+
+Now we can use the `plot()` function built into pandas. Let’s try it:
+
+``` python
+albany.plot()
+```
+
+![](fig/albany-plot-1.png) 
+
+That’s great! By default `plot` use a line plot and will plot all numeric variables of the data frame. This isn’t exactly what we want, so let’s tell `plot` what variable to use by selecting `circulation_count`.
+
+
+``` python
+albany['circulation'].plot()
+```
+
+![](fig/albany-circ-3.png)
+
+::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: challenge
+
+## Analyze the Circulation Trends
+
+Examine the line graph depicting library circulation data. You will notice two significant periods where the circulation drops to zero: first in March 2020 and then a two-year zero circulation period starting in 2012. Evaluate the graph and identify any trends, unusual patterns, or notable changes in the data.
+
+::::::::::::::::::::::::::::::::::::::::::::::::::: solution
+
+The significant drop in circulation in March 2020 is likely due to the COVID-19 pandemic, which caused widespread temporary closures of public spaces, including libraries. 
+
+The drop from 2012 through part of 2014 corresponds to the reconstruction period of the Albany Park Branch. The original building at 5150 N. Kimball Avenue was demolished in 2012, and a new, modern building was constructed at the same site. The new Albany Park Branch opened on September 13, 2014, at 3104 W. Foster Avenue in the North Park neighborhood of Chicago. More details about this renovation can be found on the Chicago Public Library webpage: [Chicago Public Library - Albany Park](https://www.chipublib.org/news/stories-we-tell-albany-park-exhibit/).
+::::::::::::::::::::::::::::::::::::::::::::::::
+::::::::::::::::::::::::::::::::::::::::::::::::
+
+What if we want to alter the axis labels and the title of the graph. In order to do that, we need to first import `matplotlib`, an extensive plotting package in Python that lets us alter all aspects of a graph.
+
+``` python
+# the plotting package pandas is using under the hood to `plot()`  
+import matplotlib.pyplot as plt
+
+albany['circulation'].plot(title='Circulation Count Over Time', figsize=(10, 5), color='blue')
+# Adding labels and showing the plot
+plt.xlabel('Date')
+plt.ylabel('Circulation Count')
+plt.show()
+```
+![](fig/albany-circ-labeling-5.png)
+
+What if we want to use a different plot type for this graphic? To do so, we can change the `kind` parameters in our `plot` function.
+
+``` python
+albany['circulation'].plot(kind='area', title='Circulation Count Area Plot at Albany Park', alpha=0.5)
+plt.xlabel('Date')
+plt.ylabel('Circulation Count')
+plt.show()
+```
+![](fig/albany-circ-area-7.png)
+
+We can also look at our circulation data as a histogram.
+
+``` python
+albany['circulation'].plot(kind='hist', bins=20, title='Distribution of Circulation Counts at Albany Park')
+plt.xlabel('Circulation Count')
+plt.show()
+```
+
+![](fig/albany-circ-hist-9.png)
+
+## Interactive Line Plot for Circulation Over Time
+
+Let’s switch back to the full dataframe in `df_long` and use another
+plotting package in Python called Plotly. First let’s install and then use
+the package.
+
+```{python}
+# uncomment below to install plotly if the import fails. 
+# !pip install plotly
+import plotly.express as px
+```
+Now we can visualize how circulation counts have changed over time for selected
+branches. This can be especially useful for identifying trends,
+seasonality, or anomalies. We will first create a subset of our data and
+only look at branches starting with the letter A. Feel free to select
+different branches. After subsetting, we will sort our new dataframe by
+date and then plot our data by date and ciculation count.
+
+``` python
+# Creating a line plot for a few selected branches to avoid clutter
+selected_branches = df_long[df_long['branch'].isin(['Altgeld',
+ 'Archer Heights',
+ 'Austin',
+ 'Austin-Irving',
+ 'Avalon'])]
+selected_branches = selected_branches.sort_values(by='date')
+```
+
+``` python
+fig = px.line(selected_branches, x=selected_branches.index, y='circulation', color='branch', title='Circulation Over Time for Selected Branches')
+fig.show()
+```
+TODO: include either a static, gif, or html output of plotly 
+* https://plotly.com/python/interactive-html-export/ 
+* https://pypi.org/project/plotly-gif/
+
+Plotly provides some nice interactive features out of the box. Hover
+over the data and interact witht he plot controls.
+
+#### Barplot to Compare Circulation Distributions Among Branches
+
+Let’s use a barplot to compare the distribution of circulation counts
+among branches. We first need to group our data by branch and sum up the
+circulation counts. Then we can use the bar plot to show the
+distribution of total circulation over branches.
+
+``` python
+# Aggregate circulation by branch
+total_circulation_by_branch = df_long.groupby('branch')['circulation'].sum().reset_index()
+
+# Create a bar plot
+fig = px.bar(total_circulation_by_branch, x='branch', y='circulation', title='Total Circulation by Branch')
+fig.show()
+```
+TODO: include either a static, gif, or html output of plotly 
+* https://plotly.com/python/interactive-html-export/ 
+* https://pypi.org/project/plotly-gif/
+
+::: keypoints
+-   Explored the use of pandas for basic data manipulation, ensuring correct indexing with DatetimeIndex to enable time-series operations like resampling.
+-   Used pandas’ built-in plot() for initial visualizations and faced issues with overplotting, leading to adjustments like data filtering and resampling to simplify plots.
+-   Introduced Plotly for advanced interactive visualizations, enhancing user engagement through dynamic plots such as line graphs, area charts, and bar plots with capabilities like dropdown selections.
+:::