Find us on GitHub

A Data Carpentry Workshop

Rutgers University

Dec 05-06, 2017

9:00 am - 5:00 pm

Instructors: Jennifer Shelton, Chris Hamm

Helpers: Janet Alder, Doreen Badheka

General Information

Data Carpentry workshops are for any researcher who has data they want to analyze, and no prior computational experience is required. This hands-on workshop teaches basic concepts, skills and tools for working more effectively with data. The focuses of this workshop will be working with genomics data, and data management & analysis for genomics research. They will cover metadata organization in spreadsheets, data organization, connecting to and using cloud computing, the command line for sequence quality control and bioinformatics workflows.

We will cover Introduction to the Shell and Cloud Computing, The Shell and Project Setup, Know Your Data and Variant Calling and Workflow Automation. Participants should bring their laptops and plan to participate actively. By the end of the workshop learners should be able to more effectively manage and analyze data and be able to apply the tools and approaches directly to their ongoing research.

Who: The workshop primarily includes students and postdocs who are in the Rutgers iJOBS program funded by the NIH BEST initiative but may also include additional interested students and faculty.

Where: New Jersey Medical School, 185 South Orange Ave, Medical Science Building Room B516, Newark, NJ. Get directions with OpenStreetMap or Google Maps.

Requirements: Participants must bring a laptop with a Mac, Linux, or Windows operating sytem (not a tablet, Chromebook, etc.) that they have administrative privileges on. They should have a few specific software packages installed (listed below). They are also required to abide by Data Carpentry's Code of Conduct.

Contact: Please mail for more information.

Day 1

Morning Introduction to the Shell and Cloud Computing
Afternoon The Shell and Project Setup

Day 2

Morning Know Your Data
Afternoon Variant Calling and Workflow Automation

We will use this Etherpad for chatting, taking notes, and sharing URLs and bits of code.


Please be sure to complete these surveys before and after the workshop.

Pre-workshop Survey

Post-workshop Survey

Preliminary Schedule

Tuesday, Dec 5

MorningIntroduction to the Shell and Cloud Computing
09:00 Workshop overview
09:30 Using cloud computing for genomics
10:00 Introduction to Unix: navigating, paths, shortcuts
10:30 Coffee/tea break
10:45 Introduction to Unix: examining files, modifying, writing
12:00 Lunch (provided)
Afternoon The Shell and Project Setup
13:00 Introduction to Unix: searching and redirection
13:45 Shell scripts
15:00 Break
15:15 Shell scripts (cont)
15:45 Genomics planning
16:45 Wrap-up

Wednesday, Dec 6

Morning Know Your Data
09:00 Introduction to the workshop dataset
09:15 Examining data on the NCBI SRA database
09:45 QC of NGS data
10:30 Coffee/tea break
10:45 QC of NGS data (cont)
12:00 Lunch (provided)
Afternoon Variant Calling and Workflow Automation
13:00 Variant calling workflow (QC, alignment, variant calling)
14:00 Automating the workflow
15:00 Break
15:15 Automating the workflow (cont)
16:45 Wrap-up


The Unix Shell

  • Files and directories
  • History and tab completion
  • Pipes and redirection
  • Looping over files
  • Creating and running shell scripts
  • Finding things
  • Reference...


To participate in a Data Carpentry workshop, you will need working copies of the described software. Please make sure to install everything (or at least to download the installers) before the start of your workshop. Participants should bring and use their own laptops to insure the proper setup of tools for an efficient workflow once you leave the workshop.

Mac users

Windows users

We maintain a list of common issues that occur during installation as a reference for instructors that may be useful on the Configuration Problems and Solutions wiki page.