Last updated: 2021-11-18

Checks: 2 0

Knit directory: ebpmf_data_analysis/

This reproducible R Markdown analysis was created with workflowr (version 1.6.2). The Checks tab describes the reproducibility checks that were applied when the results were created. The Past versions tab lists the development history.

R Markdown file: up-to-date

Great! Since the R Markdown file has been committed to the Git repository, you know the exact version of the code that produced these results.

Repository version: 30b634d

Great! You are using Git for version control. Tracking code development and connecting the code version to the results is critical for reproducibility.

The results in this page were generated with repository version 30b634d. See the Past versions tab to see a history of the changes made to the R Markdown and HTML files.

Note that you need to be careful to ensure that all relevant files for the analysis have been committed to Git prior to generating the results (you can use wflow_publish or wflow_git_commit). workflowr only checks the R Markdown file, but you know if there are other scripts or data files that it depends on. Below is the status of the Git repository when the results were generated: 


Ignored files:
    Ignored:    .DS_Store
    Ignored:    .Rhistory
    Ignored:    .Rproj.user/
    Ignored:    analysis/ebpmf_bg_tutorial_cache/
    Ignored:    analysis/ebpmf_wbg_model_intro_cache/
    Ignored:    analysis/ebpmf_wbg_simulate_big_data2_cache/
    Ignored:    analysis/ebpmf_wbg_simulate_big_data3_cache/
    Ignored:    analysis/ebpmf_wbg_simulate_big_data_cache/
    Ignored:    analysis/ebpmf_wbg_simulation_cache/
    Ignored:    analysis/investigate_np_ebpmf_wbg_cache/
    Ignored:    analysis/pmf_greedy_experiment_cache/
    Ignored:    analysis/sla_data_analysis_k10_cache/
    Ignored:    data/.DS_Store
    Ignored:    output/.DS_Store
    Ignored:    output/News/.DS_Store
    Ignored:    topicView-app/.DS_Store

Untracked files:
    Untracked:  analysis/.index_draft.txt.swp
    Untracked:  analysis/covid_dataset.Rmd
    Untracked:  analysis/draft.Rmd
    Untracked:  analysis/draft2.Rmd
    Untracked:  analysis/ebpmf_wbg_simulate_correlated.Rmd
    Untracked:  analysis/ebpmf_wbg_simulation_big.Rmd
    Untracked:  analysis/ebpmf_wbg_simulation_big2_more.Rmd
    Untracked:  analysis/heatmap.Rmd
    Untracked:  analysis/index_draft.txt
    Untracked:  analysis/investigate_largeK.Rmd
    Untracked:  analysis/investigate_news_topics.Rmd
    Untracked:  analysis/poissonmix_vs_pmf.Rmd
    Untracked:  analysis/simulate_data_bg.Rmd
    Untracked:  analysis/simulate_data_bg2.Rmd
    Untracked:  analysis/sinkhorn.Rmd
    Untracked:  analysis/summary_sla_news_nips.Rmd
    Untracked:  analysis/test.R
    Untracked:  data/GSE145926_RAW/
    Untracked:  data/cell_data.csv
    Untracked:  data/sim/init_random.sim_bg_block_n1100_p2100_K50.Rds
    Untracked:  data/sim/simulated_data_small.RData
    Untracked:  data/sim/simulated_data_small2.RData
    Untracked:  data/subject/
    Untracked:  output/poissonmix_vs_pmf.RDS
    Untracked:  output/poissonmix_vs_pmf.RData
    Untracked:  output/sim/v0.4.5/exper2/init_random.sim_bg_block_n1100_p2100_K50.Rds
    Untracked:  output/sim/v0.4.5/exper2/sim_bg_block_n1100_p2100_K50_ebpmf_wbg_K50_maxiter100_init_random.Rds
    Untracked:  output/sim/v0.4.5/exper2/sim_bg_block_n1100_p2100_K50_ebpmf_wbg_K50_maxiter10_from_truth.Rds
    Untracked:  output/sim/v0.4.5/exper2/sim_bg_block_n1100_p2100_K50_ebpmf_wbg_K50_maxiter10_init_random.Rds
    Untracked:  output/sim/v0.4.5/exper2/sim_bg_block_n1100_p2100_K50_ebpmf_wbg_K50_maxiter10_init_random2.Rds
    Untracked:  output/sim/v0.4.5/exper2/sim_bg_block_n1100_p2100_K50_ebpmf_wbg_K50_maxiter10_pmf_bg_K50_maxiter10_from_truth_scaled.Rds
    Untracked:  output/sim/v0.4.5/exper2/sim_bg_block_n1100_p2100_K50_ebpmf_wbg_K50_maxiter10_pmf_bg_K50_maxiter10_from_truth_scaled0.Rds
    Untracked:  output/sim/v0.4.5/exper2/sim_bg_block_n1100_p2100_K50_ebpmf_wbg_K50_maxiter10_pmf_bg_K50_maxiter10_from_truth_scaled1.Rds
    Untracked:  output/sim/v0.4.5/exper2/sim_bg_block_n1100_p2100_K50_ebpmf_wbg_K50_maxiter1_from_truth.Rds
    Untracked:  output/sim/v0.4.5/exper2/sim_bg_block_n1100_p2100_K50_ebpmf_wbg_K50_maxiter20_init_random.Rds
    Untracked:  output/sim/v0.4.5/exper2/sim_bg_block_n1100_p2100_K50_ebpmf_wbg_K50_maxiter2_from_truth.Rds
    Untracked:  output/sim/v0.4.5/exper2/sim_bg_block_n1100_p2100_K50_ebpmf_wbg_K50_maxiter30_init_random.Rds
    Untracked:  output/sim/v0.4.5/exper2/sim_bg_block_n1100_p2100_K50_ebpmf_wbg_K50_maxiter3_from_truth.Rds
    Untracked:  output/sim/v0.4.5/exper2/sim_bg_block_n1100_p2100_K50_ebpmf_wbg_K50_maxiter40_init_random.Rds
    Untracked:  output/sim/v0.4.5/exper2/sim_bg_block_n1100_p2100_K50_ebpmf_wbg_K50_maxiter4_from_truth.Rds
    Untracked:  output/sim/v0.4.5/exper2/sim_bg_block_n1100_p2100_K50_ebpmf_wbg_K50_maxiter50_init_random.Rds
    Untracked:  output/sim/v0.4.5/exper2/sim_bg_block_n1100_p2100_K50_ebpmf_wbg_K50_maxiter5_from_truth.Rds
    Untracked:  output/sim/v0.4.5/exper2/sim_bg_block_n1100_p2100_K50_ebpmf_wbg_K50_maxiter5_pmf_bg_K50_maxiter10_from_truth_scaled.Rds
    Untracked:  output/sim/v0.4.5/exper2/sim_bg_block_n1100_p2100_K50_ebpmf_wbg_K50_maxiter5_pmf_bg_K50_maxiter10_from_truth_scaled0.Rds
    Untracked:  output/sim/v0.4.5/exper2/sim_bg_block_n1100_p2100_K50_ebpmf_wbg_K50_maxiter5_pmf_bg_K50_maxiter10_from_truth_scaled1.Rds
    Untracked:  output/sim/v0.4.5/exper2/sim_bg_block_n1100_p2100_K50_ebpmf_wbg_K50_maxiter60_init_random.Rds
    Untracked:  output/sim/v0.4.5/exper2/sim_bg_block_n1100_p2100_K50_ebpmf_wbg_K50_maxiter6_from_truth.Rds
    Untracked:  output/sim/v0.4.5/exper2/sim_bg_block_n1100_p2100_K50_ebpmf_wbg_K50_maxiter70_init_random.Rds
    Untracked:  output/sim/v0.4.5/exper2/sim_bg_block_n1100_p2100_K50_ebpmf_wbg_K50_maxiter7_from_truth.Rds
    Untracked:  output/sim/v0.4.5/exper2/sim_bg_block_n1100_p2100_K50_ebpmf_wbg_K50_maxiter80_init_random.Rds
    Untracked:  output/sim/v0.4.5/exper2/sim_bg_block_n1100_p2100_K50_ebpmf_wbg_K50_maxiter8_from_truth.Rds
    Untracked:  output/sim/v0.4.5/exper2/sim_bg_block_n1100_p2100_K50_ebpmf_wbg_K50_maxiter90_init_random.Rds
    Untracked:  output/sim/v0.4.5/exper2/sim_bg_block_n1100_p2100_K50_ebpmf_wbg_K50_maxiter9_from_truth.Rds
    Untracked:  output/sim/v0.4.5/exper2/sim_bg_block_n1100_p2100_K50_fastTopics_K50_50em_950_scd.Rds
    Untracked:  output/smallsim2_1.RData
    Untracked:  script/Rplots.pdf
    Untracked:  script/init_ebpmf_wbg_from_pmf_bg.R
    Untracked:  script/init_ebpmf_wbg_random.R
    Untracked:  script/save_volcano_plot.R
    Untracked:  topicView-app/app_utils.R
    Untracked:  topicView-app/data/
    Untracked:  topicView-app/output/
    Untracked:  topicView-app/rsconnect/

Unstaged changes:
    Modified:   analysis/cone_NMF_l2_2.Rmd
    Modified:   analysis/ebpmf_wbg_simulate_big_data2.Rmd
    Modified:   analysis/experiment_ebpmf_wbg_subject.Rmd
    Modified:   analysis/multinom_sampling.Rmd
    Deleted:    analysis/sla_data_analysis_k10.Rmd
    Deleted:    analysis/sla_data_analysis_k5.Rmd
    Deleted:    analysis/sla_data_analysis_k50.Rmd
    Deleted:    data/SLA/SCC2016/Code/APL/compCM.m
    Deleted:    data/SLA/SCC2016/Code/APL/compMuI.m
    Deleted:    data/SLA/SCC2016/Code/APL/compParamErr2.m
    Deleted:    data/SLA/SCC2016/Code/APL/cpl4c.m
    Deleted:    data/SLA/SCC2016/Code/APL/cplEstimParam.m
    Deleted:    data/SLA/SCC2016/Code/APL/cpl_basic_demo_PJ.m
    Deleted:    data/SLA/SCC2016/Code/APL/cpl_demo.m
    Deleted:    data/SLA/SCC2016/Code/APL/cpl_demo2a.m
    Deleted:    data/SLA/SCC2016/Code/APL/dcBlkMod.m
    Deleted:    data/SLA/SCC2016/Code/APL/dcBlkMod2.m
    Deleted:    data/SLA/SCC2016/Code/APL/dcBlkMod3.m
    Deleted:    data/SLA/SCC2016/Code/APL/dcbm_nmi_beta_D.m
    Deleted:    data/SLA/SCC2016/Code/APL/dcbm_nmi_lambda_D.m
    Deleted:    data/SLA/SCC2016/Code/APL/dcbm_time_vs_n_D.m
    Deleted:    data/SLA/SCC2016/Code/APL/genDCBlkMod.c
    Deleted:    data/SLA/SCC2016/Code/APL/genDCBlkMod.mexa64
    Deleted:    data/SLA/SCC2016/Code/APL/genDCBlkMod2.m
    Deleted:    data/SLA/SCC2016/Code/APL/initLabel5b.m
    Deleted:    data/SLA/SCC2016/Code/BCPL/ProfileLike.m
    Deleted:    data/SLA/SCC2016/Code/BCPL/calCri1.m
    Deleted:    data/SLA/SCC2016/Code/BCPL/calCri2.m
    Deleted:    data/SLA/SCC2016/Code/BCPL/mutiExp.m
    Deleted:    data/SLA/SCC2016/Code/MatlabCode.m
    Deleted:    data/SLA/SCC2016/Code/NewmanSM/NewmanSM.m
    Deleted:    data/SLA/SCC2016/Code/coauthorThresh2GiantAdj.txt
    Deleted:    data/SLA/SCC2016/Code/coauthorThresh2GiantCommLabelK2Matlab.txt
    Deleted:    data/SLA/SCC2016/Code/functions.R
    Deleted:    data/SLA/SCC2016/Code/main.R
    Deleted:    data/SLA/SCC2016/Data/authorList.txt
    Deleted:    data/SLA/SCC2016/Data/authorPaperBiadj.txt
    Deleted:    data/SLA/SCC2016/Data/paperCitAdj.txt
    Deleted:    data/SLA/SCC2016/Data/paperList.txt
    Deleted:    data/SLA/SCC2016/ReadMe.txt
    Modified:   data/sim/docword.sim_bg_block_n1100_p2100_K50.txt
    Deleted:    data/sim/init.sim_bg_block_n1100_p2100_K50.Rds
    Modified:   data/sim/truth.sim_bg_block_n1100_p2100_K50.Rds
    Deleted:    data/uci_BoW.sh
    Deleted:    data/uci_BoW/docword.kos.txt
    Deleted:    data/uci_BoW/readme.txt
    Deleted:    data/uci_BoW/vocab.kos.txt
    Deleted:    output/sim/v0.4.5/fit_sim_bg_block_n1100_p2100_K50_ebpmf_wbg_maxiter_5000.Rout
    Deleted:    output/sim/v0.4.5/fit_sim_bg_block_n1100_p2100_K50_ebpmf_wbg_maxiter_5000_from_truth.Rout
    Deleted:    output/sim/v0.4.5/sim_bg_block_n1100_p2100_K50_ebpmf_wbg_K50_maxiter3.Rds
    Deleted:    output/sim/v0.4.5/sim_bg_block_n1100_p2100_K50_ebpmf_wbg_K50_maxiter5000.Rds
    Deleted:    output/sim/v0.4.5/sim_bg_block_n1100_p2100_K50_ebpmf_wbg_K50_maxiter5000_from_truth.Rds
    Deleted:    output/sim/v0.4.5/sim_bg_block_n1100_p2100_K50_ebpmf_wbg_K50_maxiter50_from_truth2.Rds
    Deleted:    output/uci_BoW/v0.3.8/fit_kos_ebpmf_bg_K20_maxiter_1000.Rout
    Deleted:    output/uci_BoW/v0.3.8/fit_kos_ebpmf_bg_K20_maxiter_500.Rout
    Deleted:    output/uci_BoW/v0.3.8/fit_kos_ebpmf_bg_K20_maxiter_5000.Rout
    Deleted:    output/uci_BoW/v0.3.8/kos_ebpmf_bg_K20_maxiter1000.Rds
    Deleted:    output/uci_BoW/v0.3.8/kos_ebpmf_bg_K20_maxiter500.Rds
    Deleted:    output/uci_BoW/v0.3.8/kos_ebpmf_bg_K20_maxiter5000.Rds
    Deleted:    output/uci_BoW/v0.3.8/kos_ebpmf_bg_K2_maxiter10.Rds
    Deleted:    output/uci_BoW/v0.3.9/fit_kos_ebpmf_bg_K100_maxiter_5000.Rout
    Deleted:    output/uci_BoW/v0.3.9/fit_kos_ebpmf_bg_K20_maxiter_5000.Rout
    Deleted:    output/uci_BoW/v0.3.9/fit_kos_ebpmf_bg_K50_maxiter_5000.Rout
    Deleted:    output/uci_BoW/v0.3.9/fit_kos_ebpmf_bg_initLF_K100_maxiter_5000.Rout
    Deleted:    output/uci_BoW/v0.3.9/fit_kos_ebpmf_bg_initLF_K20_maxiter_5000.Rout
    Deleted:    output/uci_BoW/v0.3.9/fit_kos_ebpmf_bg_initLF_K300_maxiter_1000.Rout
    Deleted:    output/uci_BoW/v0.3.9/fit_kos_ebpmf_bg_initLF_K500_maxiter_1000.Rout
    Deleted:    output/uci_BoW/v0.3.9/fit_kos_ebpmf_bg_initLF_K50_maxiter_5000.Rout
    Deleted:    output/uci_BoW/v0.3.9/fit_kos_pmf_initLF_K100_maxiter_5000.Rout
    Deleted:    output/uci_BoW/v0.3.9/fit_kos_pmf_initLF_K20_maxiter_5000.Rout
    Deleted:    output/uci_BoW/v0.3.9/fit_kos_pmf_initLF_K300_maxiter_1000.Rout
    Deleted:    output/uci_BoW/v0.3.9/fit_kos_pmf_initLF_K500_maxiter_5000.Rout
    Deleted:    output/uci_BoW/v0.3.9/fit_kos_pmf_initLF_K50_maxiter_5000.Rout
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_K100_maxiter1000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_K100_maxiter1500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_K100_maxiter2000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_K100_maxiter500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_K20_maxiter1000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_K20_maxiter1500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_K20_maxiter2000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_K20_maxiter2500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_K20_maxiter3000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_K20_maxiter3500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_K20_maxiter4000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_K20_maxiter4500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_K20_maxiter500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_K20_maxiter5000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_K50_maxiter1000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_K50_maxiter1500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_K50_maxiter2000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_K50_maxiter2500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_K50_maxiter3000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_K50_maxiter3500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_K50_maxiter4000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_K50_maxiter4500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_K50_maxiter500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K100_maxiter1000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K100_maxiter1500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K100_maxiter2000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K100_maxiter2500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K100_maxiter500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K20_maxiter10.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K20_maxiter1000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K20_maxiter1500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K20_maxiter2000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K20_maxiter2500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K20_maxiter3000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K20_maxiter3500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K20_maxiter4000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K20_maxiter4500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K20_maxiter5.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K20_maxiter500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K20_maxiter5000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K300_maxiter100.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K300_maxiter1000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K300_maxiter200.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K300_maxiter300.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K300_maxiter400.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K300_maxiter500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K300_maxiter600.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K300_maxiter700.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K300_maxiter800.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K300_maxiter900.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K50_maxiter1000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K50_maxiter1500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K50_maxiter2000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K50_maxiter2500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K50_maxiter3000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K50_maxiter3500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K50_maxiter4000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K50_maxiter4500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K50_maxiter500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_ebpmf_bg_initLF50_K50_maxiter5000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_init_nmf_K100_iter50.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_init_nmf_K20_iter50.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_init_nmf_K300_iter50.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_init_nmf_K500_iter50.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_init_nmf_K50_iter50.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K100_maxiter1000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K100_maxiter1500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K100_maxiter2000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K100_maxiter2500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K100_maxiter3000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K100_maxiter3500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K100_maxiter4000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K100_maxiter4500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K100_maxiter500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K100_maxiter5000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K20_maxiter10.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K20_maxiter1000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K20_maxiter1500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K20_maxiter2000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K20_maxiter2500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K20_maxiter3000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K20_maxiter3500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K20_maxiter4000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K20_maxiter4500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K20_maxiter5.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K20_maxiter500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K20_maxiter5000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K300_maxiter100.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K300_maxiter1000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K300_maxiter200.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K300_maxiter300.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K300_maxiter400.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K300_maxiter500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K300_maxiter600.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K300_maxiter700.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K300_maxiter800.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K300_maxiter900.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K50_maxiter1000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K50_maxiter1500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K50_maxiter2000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K50_maxiter2500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K50_maxiter3000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K50_maxiter3500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K50_maxiter4000.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K50_maxiter4500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K50_maxiter500.Rds
    Deleted:    output/uci_BoW/v0.3.9/kos_pmf_initLF50_K50_maxiter5000.Rds
    Deleted:    output/uci_BoW/v0.4.2/fit_kos_ebpmf_wbg_initLF_K100_maxiter_5000.Rout
    Deleted:    output/uci_BoW/v0.4.2/fit_kos_ebpmf_wbg_initLF_K20_maxiter_5000.Rout
    Deleted:    output/uci_BoW/v0.4.2/fit_kos_ebpmf_wbg_initLF_K50_maxiter_5000.Rout
    Deleted:    output/uci_BoW/v0.4.2/fit_kos_ebpmf_wbg_initL_K100_maxiter_5000.Rout
    Deleted:    output/uci_BoW/v0.4.2/fit_kos_ebpmf_wbg_initL_K20_maxiter_5000.Rout
    Deleted:    output/uci_BoW/v0.4.2/fit_kos_ebpmf_wbg_initL_K50_maxiter_5000.Rout
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initL50_K20_maxiter10.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initL50_K20_maxiter1000.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initL50_K20_maxiter1500.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initL50_K20_maxiter2000.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initL50_K20_maxiter2500.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initL50_K20_maxiter3000.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initL50_K20_maxiter3500.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initL50_K20_maxiter4000.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initL50_K20_maxiter4500.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initL50_K20_maxiter500.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initL50_K20_maxiter5000.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initL50_K50_maxiter1000.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initL50_K50_maxiter1500.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initL50_K50_maxiter2000.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initL50_K50_maxiter2500.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initL50_K50_maxiter3000.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initL50_K50_maxiter3500.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initL50_K50_maxiter4000.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initL50_K50_maxiter4500.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initL50_K50_maxiter500.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initL50_K50_maxiter5000.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initLF50_K100_maxiter1000.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initLF50_K100_maxiter1500.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initLF50_K100_maxiter2000.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initLF50_K100_maxiter2500.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initLF50_K100_maxiter500.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initLF50_K20_maxiter10.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initLF50_K20_maxiter1000.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initLF50_K20_maxiter1500.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initLF50_K20_maxiter2000.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initLF50_K20_maxiter2500.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initLF50_K20_maxiter3000.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initLF50_K20_maxiter3500.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initLF50_K20_maxiter4000.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initLF50_K20_maxiter4500.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initLF50_K20_maxiter500.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initLF50_K20_maxiter5000.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initLF50_K3_maxiter10.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initLF50_K50_maxiter1000.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initLF50_K50_maxiter1500.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initLF50_K50_maxiter2000.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initLF50_K50_maxiter2500.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initLF50_K50_maxiter3000.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initLF50_K50_maxiter3500.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initLF50_K50_maxiter4000.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initLF50_K50_maxiter4500.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initLF50_K50_maxiter500.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_ebpmf_wbg_initLF50_K50_maxiter5000.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_init_nmf_K100_iter50.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_init_nmf_K20_iter50.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_init_nmf_K300_iter50.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_init_nmf_K3_iter50.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_init_nmf_K500_iter50.Rds
    Deleted:    output/uci_BoW/v0.4.2/kos_init_nmf_K50_iter50.Rds
    Deleted:    output/uci_BoW/v0.4.4/fit_kos_np_ebpmf_wbg_initLF_K100_maxiter_5000.Rout
    Deleted:    output/uci_BoW/v0.4.4/fit_kos_np_ebpmf_wbg_initLF_K20_maxiter_5000.Rout
    Deleted:    output/uci_BoW/v0.4.4/fit_kos_np_ebpmf_wbg_initLF_K50_maxiter_5000.Rout
    Deleted:    output/uci_BoW/v0.4.4/kos_np_ebpmf_wbg_initLF50_K100_maxiter500.Rds
    Deleted:    output/uci_BoW/v0.4.4/kos_np_ebpmf_wbg_initLF50_K20_maxiter500.Rds
    Deleted:    output/uci_BoW/v0.4.4/kos_np_ebpmf_wbg_initLF50_K50_maxiter500.Rds
    Modified:   script/fit_kos_NMF_F.R
    Modified:   topicView-app/app.R

Note that any generated files, e.g. HTML, png, CSS, etc., are not included in this status report because it is ok for generated content to have uncommitted changes.

These are the previous versions of the repository in which changes were made to the R Markdown (analysis/index.Rmd) and HTML (docs/index.html) files. If you’ve configured a remote Git repository (see ?wflow_git_remote), click on the hyperlinks in the table below to view the files as they were in that past version.

File Version Author Date Message
Rmd 30b634d Zihao Wang 2021-11-18 clean up index
html c7777da Zihao Wang 2021-10-15 Build site.
Rmd cc70a30 Zihao Wang 2021-10-15 update index
html 2bfa2a4 Zihao Wang 2021-10-15 Build site.
Rmd 948280d Zihao Wang 2021-10-15 update index
html c875946 zihao12 2021-08-29 Build site.
Rmd 3019c5e zihao12 2021-08-29 experiment_ebpmf_wbg_subject
html 49ee772 zihao12 2021-07-27 Build site.
Rmd 284f2b7 zihao12 2021-07-27 index
html 5647e60 zihao12 2021-07-27 Build site.
Rmd a6f3462 zihao12 2021-07-27 index
html f7ae95c zihao12 2021-07-27 Build site.
Rmd fe13972 zihao12 2021-07-27 index
html 7340479 zihao12 2021-07-24 Build site.
Rmd 645fd47 zihao12 2021-07-24 index
html 9fbd025 zihao12 2021-07-23 Build site.
Rmd 8bf34ac zihao12 2021-07-23 index
html 22c0824 zihao12 2021-07-23 Build site.
Rmd eafbd3f zihao12 2021-07-23 index
html 510c956 Zihao 2021-07-05 Build site.
Rmd 328705b Zihao 2021-07-05 index
html e24ebf5 zihao12 2021-07-01 Build site.
Rmd b92829c zihao12 2021-07-01 index
html 6c9d8b1 zihao12 2021-07-01 Build site.
Rmd f6104be zihao12 2021-07-01 index
html eafad60 Zihao 2021-06-30 Build site.
Rmd 5400458 Zihao 2021-06-30 index
html 6a75f05 zihao12 2021-06-24 Build site.
Rmd 00174e1 zihao12 2021-06-24 index
html 4bff483 zihao12 2021-06-22 Build site.
Rmd ed8e509 zihao12 2021-06-22 index
html 451291a zihao12 2021-06-22 Build site.
Rmd 69b784c zihao12 2021-06-22 index
html 93cc221 zihao12 2021-06-16 Build site.
Rmd 7d57e70 zihao12 2021-06-16 index
html 1db3c5c zihao12 2021-06-16 Build site.
Rmd 86f370b zihao12 2021-06-16 index
html b8991e5 zihao12 2021-06-15 Build site.
Rmd 4d7a1ea zihao12 2021-06-15 index
html 5c3c833 zihao12 2021-06-14 Build site.
Rmd 0e0732d zihao12 2021-06-14 index.Rmd update
html b7da766 zihao12 2020-11-05 Build site.
Rmd 1aae3e9 zihao12 2020-11-05 update index
html b4a4d19 zihao12 2020-10-05 Build site.
Rmd d8ba5a1 zihao12 2020-10-05 update index
html bbfcc6c zihao12 2020-10-05 Build site.
Rmd 7714094 zihao12 2020-10-05 update index
html acf2eec zihao12 2020-09-28 Build site.
Rmd a6f5d72 zihao12 2020-09-28 add link to sla_data_analysis_k10
html ad46861 zihao12 2020-09-26 Build site.
Rmd d01dda7 zihao12 2020-09-26 demo for processing SLA data
html d481b1d zihao12 2020-06-05 Build site.
Rmd 448f658 zihao12 2020-06-05 updatye index
html a719d74 zihao12 2020-06-05 Build site.
Rmd e075657 zihao12 2020-06-05 index
html 041d240 zihao12 2020-05-19 Build site.
Rmd dc62cf9 zihao12 2020-05-19 update index
html 8aeda46 zihao12 2020-05-19 Build site.
Rmd 2aec1ad zihao12 2020-05-19 update index
html 7505013 zihao12 2020-05-16 Build site.
Rmd 1bc4a3b zihao12 2020-05-16 add links to some data analysis
html 9037b15 zihao12 2020-05-16 Build site.
Rmd 68802ce zihao12 2020-05-16 add links to some data analysis
html a4c37a9 zihao12 2020-05-11 Build site.
Rmd f4ae184 zihao12 2020-05-11 Start workflowr project.

Model:

Poisson Matrix Factorization (PMF)

  • Suppose we observe count matrix \(X \in R_{+}^{n \times p}\). In topic modeling context, \(X_{ij}\) means the number occurrences of word \(i\) in document \(j\); in gene expression data, it means the number of occurrences in gene \(i\) in cell \(j\).
  • PMF assumes its underlying mean matrix can be decomposed into two low dimensional, non-negative matrices \(L F^T\) where \(L \in R^{n \times K}, F \in R^{p \times K}\). Each column of \(F\) is called a factor/topic, representing the importance of a word/gene in a topic/biological process. Each row of \(L\) is the membership vector for each sample (document/cell) \(i\), representing the importances of topics/bioligical processes for the sample.
  • The model can be written as: \[\begin{align} & X_{ij} \sim \text{Pois}(\lambda_{ij})\\ & \lambda_{ij} = \sum_{k} l_{ik} f_{jk}\\ & l_{ik}, f_{jk} \geq 0 \end{align}\]

  • To estimate \(L, F\) we maximize the likelihood of the model above, through various types of coordinate descent methods.

  • Sometimes the data is very sparse so MLE tends to overfit. Therefore we need to find ways to regularize \(L, F\).

Adaptive penalization with Empirical Bayes approach (EB)

  • If we simply add a penalization function on \(F\), we either have only one penalization parameter for all \(k\) factors, or need to tune \(k\) penalization parameters simultaneously. The same is true for regularizing \(L\).
  • With EB approach, we can flexibly incorporate penalization in a data-adaptive fashion: \[\begin{align} & X_{ij} \sim \text{Pois}(\lambda_{ij})\\ & \lambda_{ij} = \sum_{k} l_{ik} f_{jk}\\ & l_{ik} \sim g_{L, k}, f_{jk} \sim g_{F, k}\\ & g_{L, k} \in \mathcal{G}_L, g_{F, k} \in \mathcal{G}_F \end{align}\]
  • We estimate \(\hat{g}\)’s with marginal MLE (so the prior/penalization is adaptive to data); then compute posterior for \(L, F\). (The actual derivations uses data augmentation and mean-field variational inference)
  • This is a rather general framework (implemented in ebpmf function) for imposing penalizations. Next let’s discuss how to impose “sparsity” assumption.

“Sparsity” assumption with EB + “background model”

  • In topic modeling or gene expression data analysis, a “sparse” model is more interpretable and probably generalizes better than MLE with limited data. However, the usual defintion of sparsity (parameters have few nonzero elements) is not suitable for the applications here (unreasonable to think a topic does not allow a certain word), and easily cause numerical issue (a nonzero count has 0 likelihood if the mean is \(0\)).
  • A more natural definition of sparsity: each word/gene has a background frequency (call the \(p\)-vector \(f_0\)); each factor \(f_k\) encodes the deviation from the background frequency. Then “sparse” factor means very few words/genes deviate from \(f_0\):
\[\begin{align} & X_{ij} \sim \text{Pois}(\lambda_{ij})\\ & \lambda_{ij} = f_{j0} \sum_{k} w_k l_{ik} f_{jk}\\ & l_{ik} \sim \text{Ga}(a_k, b_k) \\ & f_{jk} \sim g_{F, k}\\ & g_{F, k} \in \mathcal{G}_F \end{align}\]
  • Here each prior \(g_{F, k}\) has most weights around \(1\), encouraging elements of \(f_k\)’s to be close to 1 (“sparse”).

choice of prior family

We use mixture of gamma priors (each gamma prior has mean \(1\) and variance \(\phi_l\)):

\[\begin{align} \mathcal{G}_F = \{g: g(.) = \sum_{l} \pi_l \mathrm{Ga}(.; 1/\phi_l, 1/\phi_l)\} \end{align}\]

Here \(\{\phi_l\}_l\) are pre-specified grids and the free parameters are weights \(\{\pi_l\}_l\). If the estimated weights are concentrated around small \(\phi_l\)’s then it means very few words/genes deviate from the background;if the estimated weights allow very large \(\phi_l\), then more words/genes can deviate from the backrground. (More discussion). This model is implemented as ebpmf_wbg2 function.

Data Analysis

Future work