Skip to contents

call_scn() returns a scphasedcn object; write_scphasedcn(res, out) writes it as flat tables (the command line does both). read_scphasedcn(out) reads a written run back.

Files

file content
cells.tsv one row per called cell: cell type, depth stratum, null cell flag, n_calls, n_events, and the multiplet flag (normal mode always; tumour mode when multiplets are given: the cells above its multiplet_prob); tumour mode adds clone, posterior, doublet_like, ref_pool
calls.tsv per cell x region calls (normal mode: the called rows; tumour mode: every clone row and every per-cell inherited / confirmed row)
pooled_calls.tsv normal mode: called rows of the cell pools (named event regions only with pool_events, off by default: experimental, not calibrated per event)
events.tsv a cell’s overlapping called regions merged into events (normal mode: with null_cell, p, q, null_borrowed; q is descriptive, not a false discovery rate, see False-event load)
thresholds.tsv the null thresholds per score, region class and depth stratum (and cross-validation fold); named event regions have no null rows of their own and use the thresholds of their sd_class (column borrowed_from, present when a run has events; the pooled ones only with pool_events)
regions.tsv the regions scored (arms, chromosomes, 5-50 Mb tiles, events; tumour mode also clone pieces)
null_fit.tsv the depth null (Student-t per class x stratum) and the allelic overdispersion rho
false_calls.tsv, false_events.tsv normal mode: the realised false-call rate on the null cells
expected_false.tsv normal mode: expected false calls and events per class x state (event_fdr(); lower bounds)
null_selection_check.tsv normal mode with call_scn(null_check = TRUE) / --null-check: the selection-free check of the per-cell null (null_selection_check())
clones.tsv, assignments.tsv, clone_segments.tsv, clone_profiles.tsv.gz, reference_pool.txt tumour mode: clones, per-cell assignment, clone copy-number segments, clone window profiles, the normal reference cells
summary.tsv headline numbers (scphasedcn_summary())
params.yaml every parameter of the run; reads back with caller_params("params.yaml")
NOTES.md the run notes (inputs, steps, run time, package version)
result.rds the object without the large scores / features tables
scores.tsv.gz, features.tsv.gz optional (write_scores, write_features; large)

Calls

column meaning
cell the cell (normal mode pools: the pool id; tumour mode clone rows: the clone)
region, class, chrom, start, end the region (0-based, half-open) and its class: arm, chrom, 5Mb … 50Mb, event
state loss (1:0), gain (2:1), cnloh (2:0, copy-neutral LOH), gain_as, amp, del, neutral, or a clone state label
score, thr, called the region score, its null threshold and score > thr
level cell, pool or clone
source cell (the cell’s own evidence), inherited (from its clone or pool) or confirmed (a clone call the cell’s own evidence confirms)
reportable the call survives the nesting rules (e.g. a chromosome gain is not reported inside a stronger nested arm loss)
resolved normal mode, per cell: why resolve_nested() changed the row (NA = unchanged): no_depth (a chromosome / event gain without depth support), nested <region> <state> (a gain explained by a stronger nested loss / CN-LOH), replaces <region> gain (the nested call made reportable in its place)
gate_z, gate_thr normal mode with gain_gate: "z": on the gated chromosome / event gains, the depth z and its threshold (the larger of the midpoint between neutral and the gain shift and the gain_gate_q quantile of the depth null); a gain has depth support when gate_z > gate_thr. With the default gain_gate: "dl" the gate is the sign of the gain depth log-likelihood ratio (z above the midpoint) and these columns are absent
z, f, n_hap normal mode, cell rows: depth z, major-haplotype fraction and molecule votes of the cell in the region (not in tumour mode)
stratum normal mode, cell rows: the cell’s depth stratum (clean fragments; <20k, 20-50k, >=50k; not in tumour mode)
p, q, null_borrowed normal mode, tested per-cell rows: the empirical p-value against the null rows, the row-level Benjamini-Hochberg q-value (descriptive: not a false discovery rate, not a filter) and whether the row’s chromosome (chrX) or class has no null rows of its own (see below); NA on other rows
confident tumour mode: the cell’s clone assignment is confident

False-event load (normal mode)

Normal mode calls a cell-region when its score exceeds the 1 / 1,000 quantile of the null rows of its class and depth stratum (null_q). The budget is per cell-region, and a cell has about a hundred reportable rows (chromosomes, arms, 50 Mb tiles), so among thousands of cells a run carries a predictable number of false per-cell events. event_fdr() (run by call_scn()) makes that number visible; it changes no call, threshold or score.

Expected false calls and events (expected_false.tsv, one row per class x state x chromosome group):

column meaning
class, state the reportable per-cell class x state (e.g. chrom gain, arm loss)
chrom_group A (autosomes), X, Y
null_borrowed the rate comes from the autosomal null rows (chrX rows are never null rows), or from the sd_class of a class without null rows
n_rows per-cell rows of the class (every cell x region; each one a null hypothesis)
n_tested of these, the rows whose best state is this state and that would be reportable if called (calling policy, gain depth gate)
n_called, n_events reportable called rows of this state; events whose top region is of this class and state
n_null, n_null_called, n_null_events, null_per_1000 null rows of the class, how many of them are reportable calls of this state (fold-cross-validated thresholds, policy and gain gate applied as for every cell), how many null-cell events have such a top region, and the call rate per 1,000 null rows
expected_false_calls, expected_false_events the null rate per depth stratum x n_rows of that stratum, summed over the strata; _lo / _hi: 95 % Poisson interval of the null count
false_frac_calls, false_frac_events expected false / called (capped at 1): the implied false proportion (a lower bound, see below)

These are lower bounds. Every row counts as a null hypothesis, but the null rate is measured on the selected null cells, which were chosen for having no chromosome or arm call at a stricter threshold (select_null_cells()). Their rows are cleaner than those of an unselected normal cell; when the parametric selection null is too thin (shared allelic bias, noisier cells), the selection truncates the null tail and the realised false rate of the other cells is higher. chrX rows (and their events) take the autosomal rate, which ignores X-inactivation driven allelic imbalance in females. summary.tsv carries the headline: events | cell_events_in_null_cells (events found in the null cells themselves; their chrX is not screened), events | expected_false_cell_events (and per chromosome group), events | expected_false_cell_event_fraction, calls | expected_false_reportable_cell_calls.

p- and q-values. For every tested per-cell row (classes reportable per cell, plus 20 Mb rows promoted by the nesting rule) p compares the score of a reportable call (the score when the row would be reportable if called, else minus infinity) with the same statistic on the null rows of the row’s threshold group, in the cross-validation folds its threshold was trained on: p = (1 + #{null >= score}) / (n + 1) (a conformal p-value). A named event region borrows the thresholds of its sd_class, and its rows are compared with the same null rows (those of the sd_class group and folds), scored as event rows would be (null_borrowed = TRUE). With n null rows p cannot go below 1 / (n + 1); a strong event would then get a q-value that only reflects how many other events are as strong (with m tests no single event can reach a q below about m / (n + 1), which is above 1 on real data). Beyond the (51st) largest null score of the group, p therefore follows an exponential tail fitted to the top 50 null scores times the reportable share of that tail (event_fdr(tail = "none") gives the plain conformal p-value; summary.tsv events | empirical_p_floor reports the floor). q is the Benjamini-Hochberg q-value over all tested rows, called or not. In events.tsv, p / q / null_borrowed are those of the event’s top region and null_cell says whether the event is in a selected null cell. q is a descriptive column: it is not a false discovery rate and must not be used as a filter. Two independent limits make it too small, and no event-level q-value is computed:

  • Relative to the selected null. p, and the exponential tail beyond the largest null scores, are computed against the selected null cells. The selection removes the cells with the heaviest tails (e.g. cells with more allelic overdispersion than modelled), so for cells like those p is too small and the tail model extrapolates a truncated distribution; the tail is conservative only for a correctly specified null with light tails. In a sample where part of the cells are noisier than the null cells, the default tail can give false events a small q that event_fdr(tail = "none") does not (on a simulated null in which 10 % of the cells had a higher allelic overdispersion, 28 events reached q < 0.05, all false; none with tail = "none"); compare the two when the null is in doubt.
  • Rows, not events. Benjamini-Hochberg runs over rows, and one real event usually rejects several nested rows (chromosome, arms, tiles), while rows of one cell are positively dependent. A cut on q in events.tsv is therefore not an event-level false discovery rate: the share of false events below the cut can be well above it (in a simulation with 300 planted chromosome losses in 3,000 cells, 12 % of the events with q < 0.05 were false). The same holds for summary.tsv events | cell_events_q_below_0.05, a descriptive count of events at a row-level q.

Selection-free check (null_selection_check.tsv, call_scn(null_check = TRUE) / --null-check; it costs about one more scoring pass). The autosomes are split into odd and even chromosomes; null cells are selected on one half only, thresholds are trained as usual on that half’s null rows, and exceedances are counted on the other half of the same cells, rows that neither the selection nor the thresholds saw; then the halves swap. Per class x stratum (and all): n_rows, n_exceed, per_1000 with a 95 % Clopper-Pearson interval (lo_per_1000, hi_per_1000; rows of one cell overlap, so the interval is somewhat narrow), nominal_per_1000 (1000 x (1 - null_q)), insample_per_1000 (the cross-validated rate on the selection half of the same passes) and cv_per_1000 (the run’s own in-sample rate, false_calls.tsv). Cells are selected on one half only, so a cell with a real event on the measured half stays in and that event’s rows count as exceedances: per_1000 is the null rate plus the rate of real events on the measured half, an upper bound of the null rate. A rate clearly above the nominal and in-sample rates has two possible causes: a selected null that is cleaner than the cells it is applied to (the expected false counts above are then too low), or real events. Read it next to the share of cells with a called event (cells.tsv n_events, summary.tsv): in a sample without real events the excess measures the selection; in a sample with them it is an upper bound. The halves carry different chromosomes, so a class whose regions differ in their null spread shows up here as well.

Clone segments (tumour mode)

clone_segments.tsv: per clone and segment chrom, start, end, n_win (1 Mb windows), the pooled oriented votes nA, nB, the total copy number cn, the allele-specific state (1:1, 1:0, 2:1, 2:0, 3:1, …), log2 (depth relative to the clone baseline), f (major-haplotype fraction), llr, confident, label (neutral, loss, cnloh, gain, gain_as, amp, del) and ploidy (1 on a male’s chrX / chrY). The clone N0 is the fixed normal clone.

assignments.tsv: per cell the likelihood-best clone, its posterior, the margin to the second clone, whether the cell looks like a doublet of two clones (doublet_like, pair, p_mix), and confident (posterior >= 0.9, not doublet-like).

The object

library(scphasedcn)
library(data.table)
sim = simulate_scn(n_cells = 60, sex = "female", cell_types = c(a = 0.5, b = 0.5), events = NULL,
                   multiplet_rate = 0, depth = c(meanlog = log(12000), sdlog = 0.3), phase_block_mean = 1e9,
                   switch_rate = 2e-6, seed = 3)
d = file.path(tempdir(), "out_extract")
invisible(write_sim_dataset(sim, d))
cen = sim$scn$bins[, .(chrom, cstart = 1, len = max(end)), by = chrom][, .(chrom, cstart, cen_left = 0.4 * len,
                                                                             cen_right = 0.4 * len, cend = len)]
res = call_scn(d, "normal", sex = "female", centromeres = cen, null_cells = "select", verbose = FALSE)
names(res)
#>  [1] "mode"           "cells"          "regions"        "features"      
#>  [5] "scores"         "thresholds"     "calls"          "events"        
#>  [9] "pools"          "clones"         "assignments"    "segments"      
#> [13] "profiles"       "null_fit"       "params"         "notes"         
#> [17] "false_calls"    "false_events"   "expected_false" "fdr_info"      
#> [21] "embedding"      "feature_info"   "run"
res$thresholds[level == "cell" & fold == 0][1:5, .(score, class, stratum, q, thr, n_null)]
#>        score  class stratum     q        thr n_null
#>       <char> <char>  <char> <num>      <num>  <int>
#> 1:   s_depth   10Mb    <20k 0.999  4.0226146    780
#> 2: s_allelic   10Mb    <20k 0.999  3.1379661    780
#> 3:    s_comb   10Mb    <20k 0.999 -0.8174322    780
#> 4:   s_depth   20Mb    <20k 0.999  3.6923150    480
#> 5: s_allelic   20Mb    <20k 0.999  2.8782689    480
out = file.path(tempdir(), "out_caller")
invisible(write_scphasedcn(res, out))
res2 = read_scphasedcn(out)
identical(res2$calls, res$calls)
#> [1] FALSE

The components of the object:

component normal mode tumour mode
cells cell, cell_type, stratum, fold, null_cell, multiplet the cells with clone, posterior, …
regions the scored regions the scored regions plus clone pieces
scores per cell x region scores (with the null flag) NULL
thresholds per-cell (cross-validated) and pool thresholds (level) per depth stratum and clone hypothesis
calls, events see above see above
pools pool membership (kind = type or knn) NULL
clones, assignments, segments, profiles NULL see above
null_fit depth t parameters, rho, the null cells and how they were selected depth t parameters, rho, reference pool, clone discovery details
false_calls, false_events, expected_false, fdr_info, null_check null false-call rates, expected false calls / events, the q-value settings (tests, tail, calibration groups), the selection check (with null_check = TRUE) NULL
params all parameters, including the normal: caller arguments all parameters, including the tumour: options
features, feature_info, embedding, notes the per-cell region features and the run notes the same

Allelic tile mask (normal mode, opt-in)

With mask_allelic = TRUE (overdispersion rule) and / or mask_f0 = TRUE (shared-bias rule) in the parameters, call_scn() selects null cells once, assesses every autosomal 5 Mb tile on their votes (allelic_noisy_tiles()), and leaves the votes of the masked tiles out of the allele features before calling (depth is untouched). The two rules can be used alone or together (the union is masked); the run notes give the number of tiles each rule masked. null_fit$allelic_mask holds the per-tile table:

column meaning
tile, chrom, start, end the 5 Mb tile
n_rows, votes null rows (cells) of the tile and their molecule votes
rho, z_rho method-of-moments beta-binomial overdispersion of the null votes, and its robust z of log rho over the assessed tiles
assessed the tile has at least mask_min_votes votes per null row
masked the tile’s votes were left out (the active rules)
f0, d0, z_f0 vote-weighted haplotype fraction of the null rows sum(nA) / sum(n_hap), its deviation abs(f0 - 0.5), and the robust z (median / MAD) of d0 over the assessed tiles
masked_rho overdispersion rule: assessed and z_rho > mask_rho_mad
masked_f0 bias rule: assessed, z_f0 > mask_f0_mad and d0 > mask_f0_min

masked_rho and masked_f0 are reported whether or not their rule is on; the run notes list the masked tiles. The bias rule masks tiles with a shared allelic imbalance among the null cells, such as a stretch off 1:1 in every cell (germline copy-number variant or segmental duplication, imprinted or allele-specific accessible region, mapping bias, false heterozygous SNPs). A real clonal event that is frequent enough for its carriers to remain among the null cells looks the same and can be masked: in a simulation with a focal event in 35 % of cells, 60 of 96 carriers stayed in the null set and all four event tiles were masked. The mask is therefore a diagnostic to be read with the list of masked tiles, not a filter to apply blindly to samples that may carry clonal events. allelic_bias_tiles(features) computes the same table over all cells of a features object, without a caller run, to look at shared bias before calling. Under mirrored orientation (phasing = "mirrored") the haplotype fraction is the per-block major fraction, which measures coverage rather than shared bias: call_scn() then runs without the bias rule and says so in the run notes.