Find us on GitHub

A Data Carpentry Workshop

California State University, Monterey Bay

Mar 22-23, 2017

9:00 am - 5:00 pm

Instructors: Daniel Standage, Chris Hamm

Helpers: Eric Crandall, Jon Detka

General Information

Data Carpentry workshops are for any researcher who has data they want to analyze, and no prior computational experience is required. This hands-on workshop teaches basic concepts, skills and tools for working more effectively with data.

We will cover Introduction to the Shell and Cloud Computing, The Shell and Project Setup, Know Your Data and Variant Calling and Workflow Automation. Participants should bring their laptops and plan to participate actively. By the end of the workshop learners should be able to more effectively manage and analyze data and be able to apply the tools and approaches directly to their ongoing research.

Who: The workshop primarily includes students who will participate in an NIH BD2K funded "Big Biomedical Data" program at UCSC in summer 2017, but may also include additional interested students and faculty.

Where: Joel and Dena Gambord Business and Information Technology Building Rm. 118
3052 Divarty St., Seaside, CA 93955. Get directions with OpenStreetMap or Google Maps.

Requirements: Participants must bring a laptop with a Mac, Linux, or Windows operating sytem (not a tablet, Chromebook, etc.) that they have administrative privileges on. They should have a few specific software packages installed (listed below). They are also required to abide by Data Carpentry's Code of Conduct.

Contact: Please mail for more information.

Day 1

Morning Introduction to the Shell and Cloud Computing
Afternoon The Shell and Project Setup

Day 2

Morning Know Your Data
Afternoon Variant Calling and Workflow Automation

We will use this Etherpad for chatting, taking notes, and sharing URLs and bits of code.


Please be sure to complete these surveys before and after the workshop.

Pre-workshop Survey

Post-workshop Survey

Preliminary Schedule

Wednesday, March 22

MorningIntroduction to the Shell and Cloud Computing
09:00 Workshop overview
09:30 Using cloud computing for genomics
10:00 Introduction to Unix: navigating, paths, shortcuts
10:45 Coffee/tea break
11:00 Introduction to Unix: examining files, modifying, writing
12:00 Lunch (provided)
Afternoon The Shell and Project Setup
13:00 Introduction to Unix: searching and redirection
13:45 Shell scripts
14:30 Break
14:45 Shell scripts (cont)
15:45 Genomics planning
16:45 Wrap-up

Thursday, March 23

Morning Know Your Data
09:00 Introduction to the workshop dataset
09:15 Examining data on the NCBI SRA database
09:45 QC of NGS data
10:45 Coffee/tea break
11:00 QC of NGS data (cont)
12:00 Lunch (provided)
Afternoon Variant Calling and Workflow Automation
13:00 Variant calling workflow (QC, alignment, variant calling)
14:00 Automating the workflow
14:30 Break
14:45 Automating the workflow (cont)
16:45 Wrap-up


The Unix Shell

  • Files and directories
  • History and tab completion
  • Pipes and redirection
  • Looping over files
  • Creating and running shell scripts
  • Finding things
  • Reference...


To participate in a Data Carpentry workshop, you will need working copies of the described software. Please make sure to install everything (or at least to download the installers) before the start of your workshop. Participants should bring and use their own laptops to insure the proper setup of tools for an efficient workflow once you leave the workshop.

Mac users

Windows users

We maintain a list of common issues that occur during installation as a reference for instructors that may be useful on the Configuration Problems and Solutions wiki page.