Home

Awesome

Python for Data Analysis

Course in data science. Learn to analyze data of all types using the Python programming language. No programming experience is necessary.

Quick links: 📁 lessons ⏬ Lesson Schedule

Software covered:

Course topics include:

Instructor

Online Content

Textbooks

O'Reilly Media titles are free to UCSD affiliates with Safari Books Online.

Additional Materials

Command Line Resources

Python Resources

IPython Resources from Cyrille Rossant

Data Analysis Resources

Course Philosophy

  1. You learn Python by doing, just like anything else. With a few exceptions, you're not going to break your computer by trying new commands. So just try it and see what happens. Print output of commands. Print values of variables. Kick the thing until it works.
  2. Resist the urge to get frustrated and blame the computer when your code doesn't run. Computers are deterministic machines; it's almost always your fault. But that's OK! Your computer will give you error messages that describe what went wrong. Read them and try to understand them.
  3. When you don't know how to do something, google it. You'll be amazed by the solutions you'll find to do thing x if you google "python thing x".
  4. Learn keyboard shortcuts, as many as you can. Tab-complete in the shell and IPython/Jupyter!
  5. Remember Zed's sage wisdom:
    • Practice every day.
    • Don't over-do it. Slow and steady wins the race.
    • It's alright to be totally lost at first.
    • When you get stuck, get more information.
    • Try to solve it yourself first.

Assignments and Grading

Weekly Assignments

Weekly take-home assignments will follow the course schedule, reinforcing skills with exercises to analyze and visualize scientific data. Assignments will given out on Wednesdays and will be due the following Wednesday, using TritonEd. Assignments are worth 8 points each and will be graded on effort, completeness, and accuracy.

Final Project

You will choose a dataset of your own or provided in one of the texts and write a Python program (or set of Python programs or mixture of .ipynb and .py/.sh scripts) to carry out a revealing data analysis or create a software tool. Have a look at Shaw Ex43-52 and McKinney Ch10-12 for more ideas. The final project is worth 20 points and will be graded on effort, creativity, and fulfillment of the requirements below.

Requirements:

Grading

There are 100 points total possible for the course:

Participation is based on completing the pre-course survey, showing up to class (when you are able), and completing the course evaluation (this is on the honor system as I won't know who completes it). There are no midterm or final exams.

Schedule Overview

The course consists of 20 lessons. As a class, it is taught as two lessons per week for 10 weeks, but the material can be covered at any pace.

Lessons 1-3 will be an introduction to the command line. By the end of this tutorial, everyone will be familiar with basic Unix commands.

Lessons 4-9 will be an introduction to programming using Python. The main text will be Shaw's Learn Python 3 the Hard Way. For those with experience in a programming language other than Python, Lutz's Learning Python will provide a more thorough introduction to programming Python. We will learn to use IPython and IPython Notebooks (also called Jupyter Notebooks), a much richer Python experience than the Unix command line or Python interpreter.

Lessons 10-18 will focus on Python packages for data analysis. We will work through McKinney's Python for Data Analysis, which is all about analyzing data, doing statistics, and making pretty plots. You may find that Python can emulate or exceed much of the functionality of R and MATLAB.

Lessons 19-20 conclude the course with two skills useful in developing code: writing your own classes and modules, and sharing your code on GitHub.

Lesson Schedule

Lessons are available as .md or .ipynb files by clicking on the lesson numbers below. Readings should be completed while typing out the code (this is integral to the Shaw readings) and doing any Study Drills (Shaw) and Chapter Quizzes (Lutz).

LessonTitleReadingsTopicsAssignment
1Overview--Introductions and overview of coursePre-course survey; Acquire texts
2Command Line Part IShaw: Introduction,<br>Ex0, Appendix ACommand line crash course; Text editorsAssignment 1: Basic Shell Commands
3Command Line Part IIYale: The 10 Most Important Linux CommandsAdvanced commands in the bash shell--
4Conda, IPython, and Jupyter NotebooksGeohackweek: Introduction to CondaConda tutorial including Conda environments, Python packages, and PIP; Python and IPython in the command line; Jupyter notebook tutorial; Python crash courseAssignment 2: Bash, Conda, IPython, and Jupyter
5Python Basics, Strings, PrintingShaw: Ex1-10; Lutz: Ch1-7Python scripts, error messages, printing strings and variables, strings and string operations, numbers and mathematical expressions, getting help with commands and Ipython--
6Taking Input, Reading and Writing Files, FunctionsShaw: Ex11-26; Lutz: Ch9,14-17Taking input, reading files, writing files, functionsAssignment 3: Python Fundamentals I
7Logic, Loops, Lists, Dictionaries, and TuplesShaw: Ex27-39; Lutz: Ch8-13Logic and loops, lists and list comprehension, tuples, dictionaries, other types--
8Python and IPython ReviewMcKinney: Ch1, Ch2, Ch3Review of Python commands, IPython reviewAssignment 4: Python Fundamentals II
9Regular ExpressionsKuchling: Regular Expression HOWTORegular expression syntax, Command-line tools: grep, sed, awk, perl -e, Python examples: built-in and re module--
10Numpy, Pandas and Matplotlib CrashcoursePratik: Introduction to Numpy and PandasNumpy, Pandas, and Matplotlib overviewAssignment 5: Regular Expressions
11Pandas Part IMcKinney: Ch4, Ch5Introduction to NumPy and Pandas: ndarray, Series, DataFrame, index, columns, dtypes, info, describe, read_csv, head, tail, loc, iloc, ix, to_datetime--
12Pandas Part IIMcKinney: Ch6, Ch7, Ch8Data Analysis with Pandas: concat, append, merge, join, set_option, stack, unstack, transpose, dot-notation, values, apply, lambda, sort_index, sort_values, to_csv, read_csv, isnullAssignment 6: Pandas Fundamentals
13Plotting with MatplotlibMcKinney: Ch9; Johansson: Matplotlib 2D and 3D plotting in PythonMatplotlib tutorial from J.R. Johansson--
14Plotting with SeabornSeaborn TutorialSeaborn tutorial from Michael WaskomAssignment 7: Plotting
15Pandas Time SeriesMcKinney: Ch11Time series data in Pandas--
16Pandas Group OperationsMcKinney: Ch10groupby, melt, pivot, inplace=True, reindexAssignment 8: Time Series and Group Operations
17Statistics PackagesHandbook of Biological StatisticsStatistics capabilities of Pandas, Numpy, Scipy, and Scikit-bio--
18Interactive Visualization with BokehBokeh User GuideQuickstart guide to making interactive HTML and notebook plots with BokehAssignment 9: Statistics and Interactive Visualization
19Modules and ClassesShaw: Ex40-52Packaging your code so you and others can use it again--
20Git and GitHubGitHub GuidesSharing your code in a public GitHub repositoryFinal Project