Aaron Schroeder

Data Science Framework

Data Discovery

Data Profiling

Data Preparation

Data Linkage

Ontology Alignment
Entity Resolution

Data Exploration

Characterization
Visualization

Data Analysis

Inference
Textual Analysis
Network Analysis
Agent Modeling
Machine Learning

Data Fitness

Fitness for Use

Data Science Platform

Server Configuration

Encrypted Storage
LXC Container Configuration
Authentication Configuration
Permissions Configuration
Software Configuration

Apache Spark
RStudio Server
Shiny Server
Jupyter Server
PostgreSQL

Workstation Configuration

Mac
Linux
Windows

Code Examples & Tools

Code Examples

R
Python
Bash
SQL

Tools

d3
iGraph
GATE
MANN
SAFR-Link
Protege

Data Management

Data Management Plan

Project Management

Project File Layout

Ethics Checklist Template for Data Science Projects

Project Initiation
Recognize and affirm that all project plans will incorporate regular checks, discussion, and documentation to ensure adherence to the ethical principles of research.

Problem Identification (Relevant Theories and Working Hypotheses)
Establish the ethical basis for undertaking the project as well as the project requirements of both the protection of research participant and the equitable allocation of all potential project benefits and risks.
What are the expected benefits of the project to the “public good,” and do they outweigh potential risks to participant welfare? Are there implicit assumptions and biases in the framing of the project regarding the studied communities and how will they be addressed? What type of Institutional Review Board (IRB) approval process is needed? Has the team reviewed the IRB protocol?

Data Discovery, Inventory, Screening, & Acquisition
Consider potential biases that may be introduced through the choice of datasets and variables.
Do the data include disproportionate coverage of the different communities of study? Do data have adequate geographic coverage? Have checks and balances been established to identify and address implicit biases in the data?

Data Ingestion and Governance
Put in place data platforms and processes to ensure data transfer, storage, and database development adheres to data governance agreements and best practices for data quality assurance.
Have team members reviewed standard operating procedures (SOPs) and data management plans? Do data have adequate geographic coverage? Do additional procedures need to be defined for this project?

Statistical Modeling & Analysis
Establish transparency in methods, results and limitations.
Have project methods and outputs been made as transparent as possible? Are the potential limitations of the research clearly presented? Should the research be used as the basis for policy action, have the predicted benefits and social costs to all potentially affected communities been considered?

Fitness-for-Use Assessment
Critically asses the overall utility of the results in achieving the predicted benefits of the study, to be transparent about potential limitations of the study, and to ensure that unintended biases haven’t been introduced as a result of data choice and model refinement.
What are the limitations of the results? Are the results useful given the purpose of the study? Do the statistical results support the potential benefits of the study previously stated? Do the statistical results support the mitigation of the potential risks of the study previously stated? Have any data been deemed unusable that require revisiting the question of potential biases being introduced through the choice of datasets and variables (from the data discovery to fitness-for-use phase)?

Fitness-for-Use Assessment

Critically asses the overall utility of the results in achieving the predicted benefits of the study, to be transparent about potential limitations of the study, and to ensure that unintended biases haven’t been introduced as a result of data choice and model refinement.

What are the limitations of the results? Are the results useful given the purpose of the study?
Do the statistical results support the potential benefits of the study previously stated?
Do the statistical results support the mitigation of the potential risks of the study previously stated?
Have any data been deemed unusable that require revisiting the question of potential biases being introduced through the choice of datasets and variables (from the data discovery to fitness-for-use phase)?

Communication and Dissemination
Summarize questions and actions taken to reinforce the process of ethical consideration on all continuing and future projects.. Establish protocols for replication and expansion of the research findings, and information dissemination.
Did key ethical questions arise during the research and, if so, how were they addressed? How could they be addressed differently on future projects? Are research protocols, methods and data available to other researchers? If so, in what way, and, if not, what factors are limiting the ability to do so?