1 Data preparation

We need to incorporate new data (fetal brain related) to the model. It contains two sub tasks:

1. Annotate DeepSEA sequence with new labels.
   • See pipeline memo here for E081, E082, Noonan, E129 (DeepSEA label)
   • See pipeline memo here for ATAC-seq data from Noonan’s lab
   • See pipeline memo here for d27-NSC, d41-neuron data from GSE70823
   • See pipeline memo here for ATAC-seq data obtained from Yifan 5/30/18
2. Extract new sequences of interest based on new data set.
   • See pipeline memo here for E081, E082, Noonan, E129 (DeepSEA label)
   • See pipeline memo here for ATAC-seq data from Noonan’s lab
   • See pipeline memo here for d27-NSC, d41-neuron data from GSE70823
   • See pipeline memo here for ATAC-seq data obtained from Yifan 5/30/18
3. Define positive and negative sequences
   • See memo here for fantom fetal brain cage peak from Zhongshan (06/02/18)

2 Genomic Data

1. Stage1
   • Fetal brain DNase-seq from Roadmap (E081, E082)
   • H3K27ac mark from Noonan’s lab (Noonan)
   • One label from Deepsea model (E129)
2. Stage2
   • ATAC-seq data from Noonan’s lab (hNSC-50, hNSC-P15-1, hNSC-P5-1, hNSC-P5-2)
3. paperNSC
   • NSC data from this paper and the data is obtained from this link
   • d27, d41-1, d41-2 data are used
   • Is referred as paperNSC
4. ATAC053018
   • Obtained from Yifan on 5/30/18
   • Contains cell types labelled as CN, DN, GA, ips, NSC

3 Train new models

There are three strategies:

0. Baseline classifier. here
   • Train an SVM or logistic classifier using JASPAR as annotation
   • Gkm-SVM classifier is trained on 5000 positive and 5000 negative sequences one annotation at a time.
1. Use DeepSEA sequences and DeepSEA feature representation.
   • See training result Stage1 here.
   • See performance on new sequences Stage1 here
   • Evalution using allelic imbalance site Stage1 here
2. Use DeepSEA sequences but train a new model using the same architecture as DeepSEA.
   • See results and analysis Stage1 here. Leave for future
3. Include new sequences and train a new model using the same architecture. Leave for future

4 Results

1. ROC/PR curve
   • Stage1, JASPAR logistic, deepsea+extracted, no reweight
   • Stage1 JASPAR logistic, deepsea+extracted, reweight
   • Stage1 gkmsvm, deepsea, no reweight
   • Stage1 gkmsvm, deepsea, reweight
   • Stage1, train_deepsea, deepsea+extracted, no reweight
   • Stage1, train_deepsea, deepsea+extracted, reweight
   • Stage1 Stage2, all methods, deepsea sequence, reweight
   • Stage1 Stage2, all methods, extracted sequence, reweight
   • Stage1 Stage2, all methods, extracted sequence, reweight, 080917 For Xin,
   • paperNSC, all methods, deepsea sequences, reweighted GC,
   • paperNSC, all methods, extracted sequences, reweighted GC
   • ATACseq 05/30/18 from Yifan, DeepSEA sequences
   • ATACseq 05/30/18 from Yifan, extracted sequences
2. AUC scores
   • Stage1, train_deepsea vs. JASPAR, no reweight
   • Stage1, train_deepsea vs. JASPAR, reweight
   • Stage1, train_deepsea vs. gkmsvm, reweight
   • Stage1 Stage2, compare all methods, reweight
   • Stage1 Stage2, compare all methods, reweight, by group,
   • paperNSC, all methods, reweighted GC
   • ATACseq 05/30/18 from Yifan, all methods, reweighted by GC content
3. Cross validation
   • Fantom promoter

5 Other analysis

There are some other directions and issues we need to explore and tackle. The analysis of them are listed here.

1. Motif Analysis:
   • Examine which filter or a set of filters that contribute most to a particular prediction task. See details Stage1 here. ONGOING
2. Training Strategy
   • How to boost the model to specialize on a particular prediction task, say on certain tissue type? (change architecture? how to deal with overfitting?) TODO
3. Performance Evaluation
   • How to measure the performance on imbalanced data. For instance, how to interpret ROC AUC and PR AUC? See related discussion here
   • Redo DeepSEA evaluation on DNase footprint data (See Allelic Imbalance Analysis below)
4. Application
   • IBD GWAS SNPs. See here Rmarkdown script is at here
5. Data Quality:
   • GC content.
     • Stage1 DeepSEA test set,
     • Stage1 Sequences extracted from BED file
     • Stage2 Deepsea sequence set
     • Stage2 Extracted sequence set
     • paperNSC DeepSEA sequences
     • paperNSC Extracted sequences
     • ATACseq 05/30/18 from Yifan, DeepSEA sequence set
     • ATACseq 05/30/18 from Yifan, extracted sequence set
   • Similarity between data sets Stage1 Stage2 with Normal Brain data from DeepSEA
6. Allelic Imbalance Analysis:
   • DeepSEA results (Table S3). Presented by data set at here
   • Rerun analysis considering both imbalanced variants and balanced variants here
7. Detecting Regulartory Grammar:
   • Critical windows are used to detect important “word” in the sequence and potentially the combination of “word” (“phrase”) in the sequence. Here, we first explore the distribution of critical window along the sequence. E081, Noonan

6 TODO list

• 4/9/17:
  1. ROC/PR AUC interpretation link
  2. Extract sequences from new data sets link
• 4/10/17:
  3. New loss function? Weighted hinge loss? link TODO
  4. Motif analysis of current model link ONGOING
• 4/14/17:
  5. Build up pipeline using snakemake. See issue1 at link
• 4/26/17:
  6. Build baseline classifier link
• 5/29/17:
  7. GC content bias in training and testing data
     • To overcome this, we can either use unbiased sequence set or reweight the performance evaluation function based on the GC content.
     • At first, do reweighting (testing - training - motif analysis) ONGOING see testing with reweight and motif analysis with gc matched sequences
     • Construct a sequence set with matched GC content in positive and negative instances (use Homer?) TODO
• 5/31/17:
  8. Search for motif hits using top 20 motifs in motif activation analysis by GC matched sequences TODO