Review performance and accuracy evidence¶
Status: public qualification snapshot, 2026-08-29; integration-regression recovery, final hot-path optimization, and six-aligner supplement including BSBolt, 2026-08-31
This page supports a narrow claim: the qualified alignment implementation
(measured under the historical bsbit-align executable name and now exposed
canonically as bsbit align) is fast and accurate on the documented workloads while
keeping peak process-tree memory near 8 GiB on the primary GRCh38 test. It does
not qualify every bsbit command, assay, read length, architecture, or host.
How to read the product description
Ultrafast refers to the measured throughput below. High accuracy refers to the explicit simulated-truth definition below. Memory-efficient refers to the measured peak process-tree memory on the documented workload; it does not claim that bsbit always uses less RAM than every peer. In the contemporaneous sensitive comparison, bsbit used more peak RSS than BitMapperBS.
Qualified workload¶
The primary corpus, sim_wgbs_10m, contains 5,000,000 directional PE150 WGBS
fragments generated by BSReadSim 0.4.0 against the GRCh38 primary assembly.
Generation used seed 42, uniform Phred 40, 0.5% sequencing error, 99.8%
bisulfite conversion, and a 250--500 bp insert range.
| Artifact | Identity |
|---|---|
| BSReadSim run | e2a3d144-ea87-4755-9043-4fe50a44d623 |
| Normalized simulation config | SHA-256 cf153d97da521746b7f1e86fa228106e2d1d0dca858da8fb8d347d78274409fa |
| GRCh38 FASTA gzip | SHA-256 b760d18dbb651dd14dfc290083371b3ef3bff122d43a9cefb13ca4ecf38f05ca |
| R1 FASTQ gzip | SHA-256 22c4ba66773e1de2a9c6e503b1938809572f34cf11c32706c03d96d66d9c461f |
| R2 FASTQ gzip | SHA-256 0967106baf7f722c50f658840a0ec0b9b738bc47718ed4bf089544cd27adc9dd |
Sensitive-promotion bsbit-align |
SHA-256 3cb61d063d763ac1bf7bd505605566267abd074bb95744a6f246839bfdebcd6f |
Default-v2 qualification bsbit-align |
SHA-256 d8a40b158fb40248dbaeab25386bba0cc18f93ad05b8f2265ed58b335e226608 |
Final FM/SE-batching bsbit |
SHA-256 4ddfcbe0b108db6e1e2388ba499054553391c289d9d98c59a1e4a6ede3795288 |
The binary digests identify the two qualification artifacts. A source
checkout should additionally record its own git rev-parse HEAD, compiler,
feature set, binary digest, and dirty state rather than assuming byte identity.
Truth and reporting definitions¶
A pair is correct when both primary mates match the generating contig and orientation and both strand-aware 5-prime origins are within five bases of truth. Precision divides correct pairs by proper mapped pairs. Recall always divides by all 5,000,000 input pairs, and F1 is their harmonic mean. Pair MAPQ is the smaller MAPQ of the two primary mates.
Both public modes use a read-complete output contract: this corpus produces
10,000,000 primary records, including retained MAPQ-0 representatives and
explicit unmapped records. Proper pairs includes mapped MAPQ-0 pairs; it is
neither the input count nor the BAM-record count.
The comprehensive report also publishes a primary score-bearing-unique view:
proper pairs with pair-min MAPQ at least 1 for PE, and mapped primaries with
numeric MAPQ at least 1 for SE. It reports exact and within-5-bp precision and
recall, and integrates step PR-AUC only over Q60..Q1. Recall and every PR point
still divide by all 5,000,000 truth units. Tool-native classification is a
separate table and must satisfy
unique + ambiguous + unmapped_or_other = 5,000,000; discarded and omitted
inputs are counted in unmapped_or_other.
2026-08-29 qualified bsbit scorecard¶
| Mode | Proper pairs | Within-5 correct | Precision | Recall | F1 | Timing provenance |
|---|---|---|---|---|---|---|
| Default | 4,942,069 | 4,837,150 | 97.877020% | 96.743000% | 97.306710% | same-source paired n=5 median |
--sensitive |
4,991,738 | 4,867,358 | 97.508283% | 97.347160% | 97.427655% | randomized n=5 median |
The sensitive Q40 cumulative set contains 4,626,202 correct pairs and 267
errors. Its one-sided 95% upper error bound is 6.3868e-5, below the literal
1e-4 Phred-40 gate. These tiers are qualified within this aligner; equal raw
MAPQ values from different aligners need not express equal probabilities.
External accuracy context¶
These are preserved truth evaluations on the same primary corpus, but the external BAM contracts differ and the tools were not all rerun in the 2026-08-29 timing series. Sparse BAMs can omit ambiguous or unmapped pairs, so precision, recall, and output completeness must be interpreted together.
The retained tool identities are BitMapperBS commit
699a931a143f7cc001660d379832f3aabfcad061, HISAT-3N
2.2.1-3n-0.0.3 at commit
f5dda37bd1340f74ab91deace470aebc66e87a2d, Bismark 3.1.0 with Bowtie2
2.5.5, and BISCUIT 1.10.2 at commit
327fe0ca4a02d95b8f3cbe75b6bb5ffa86994a8a.
| Aligner/profile | Proper pairs | Within-5 correct | Precision | Recall | F1 |
|---|---|---|---|---|---|
| bsbit default | 4,942,069 | 4,837,150 | 97.877020% | 96.743000% | 97.306710% |
bsbit --sensitive |
4,991,738 | 4,867,358 | 97.508283% | 97.347160% | 97.427655% |
| BitMapperBS | 4,708,478 | 4,705,094 | 99.928130% | 94.101880% | 96.927531% |
| HISAT-3N | 4,797,333 | 4,768,385 | 99.396581% | 95.367700% | 97.340470% |
| Bismark | 4,819,603 | 4,811,092 | 99.823409% | 96.221840% | 97.989542% |
| BISCUIT auto insert | 4,973,804 | 4,853,160 | 97.574412% | 97.063200% | 97.318135% |
Bismark has the highest all-proper F1 in this table; BitMapperBS has the highest all-proper precision; sensitive bsbit has the highest correct-pair count and recall. There is no single winner across every metric.
Runtime and memory¶
All formal promotion runs were sequential in one WSL2 instance on an Intel
Core i7-14700K. Mapping used physical CPUs 0,2,4,6,8,10,12,14, BAM/I/O used
16,18, and the 100 ms process-tree sampler used CPU 20. Reference, indexes,
FASTQ, and BAM lived on the same WSL ext4 filesystem. Inputs remained gzip
compressed. Each method received a 10,000-pair warm-up and a complete input
read before its timed run; timing covered process launch through BAM
finalization.
Contemporaneous sensitive promotion¶
Five randomized two-method blocks compare promoted sensitive bsbit with
unmodified BitMapperBS commit
699a931a143f7cc001660d379832f3aabfcad061.
| Method | Wall observations (s) | Median | Pairs/s | Peak tree RSS | Logical read/write |
|---|---|---|---|---|---|
bsbit --sensitive |
49.332 / 46.053 / 47.850 / 47.232 / 48.022 | 47.850 s | 104,493 | 7.976 GiB | 6.087 / 0.681 GiB |
| BitMapperBS | 50.529 / 51.291 / 51.450 / 49.502 / 49.399 | 50.529 s | 98,954 | 6.773 GiB | 7.170 / 0.638 GiB |
Sensitive bsbit had lower wall time in all five blocks; its median wall was 94.70% of BitMapperBS. It used about 1.203 GiB more peak RSS. The machine-readable rows are published as sensitive-promotion-speed-20260829.tsv.
Default mode¶
Default v2 measured 19.76 seconds, 253,036 pairs/s, and 8.105 GiB peak RSS as
the median of five same-source paired runs. Those runs used GNU time rather
than the process-tree logical-I/O sampler, so their I/O columns are not merged
with the sensitive series. Exact run rows and the adapter-recovery controls are
in default-adapter-recovery-speed-20260829.tsv.
Same-corpus comprehensive supplement¶
BSBolt release 1.6.0 at commit
ea4870e975c546d1eb9cbb4d7021537e3cca05bd was freshly indexed from a
sequence-identical copy of the shared 194-contig primary assembly and run on
both layouts. BitMapperBS, HISAT-3N,
Bismark, and BISCUIT rows are the immediately preceding same-host runs. Every
paired accuracy row uses the same 5-million-pair truth denominator.
| Paired aligner / preset | Wall | Peak RSS | Logical I/O | Within-5 correct recall | Q40-qualified correct recall |
|---|---|---|---|---|---|
| bsbit default | 19.73 s | 7.95 GiB | 1.34 GiB | 96.743% | not qualified |
bsbit --sensitive |
35.76 s | 7.91 GiB | 1.34 GiB | 97.358% | 92.532% |
| BSBolt | 977.28 s | 13.40 GiB | 22.71 GiB | 95.965% | 92.354% |
| BitMapperBS | 30.51 s | 6.77 GiB | 7.81 GiB | 94.102% | not qualified |
| HISAT-3N | 238.45 s | 7.93 GiB | 46.58 GiB | 95.368% | not qualified |
| Bismark | 774.04 s | 11.18 GiB | 60.16 GiB | 96.222% | 86.838% |
| BISCUIT | 957.54 s | 11.77 GiB | 20.66 GiB | 97.060% | not qualified |
Default has the lowest paired wall and logical I/O and produces the most correct pairs per second. Sensitive has the highest proper-pair rate, correct count, all-proper recall, and Q30/Q40-qualified correct yield. PR-AUC on the score-bearing-unique scope is reported separately and must not be substituted with the all-proper value. Its Q40 set contains 4,626,592 correct pairs and 263 errors; the one-sided 95% upper error bound is 62.95 ppm, below the literal 100-ppm Q40 target. BSBolt has a lower Q40 empirical error (7.80 ppm) but retains 8,904 fewer correct pairs. Thus the advantage is calibrated yield, not the lowest error in every scalar.
For R1 only under the same eight-CPU allowance, bsbit takes 18.66 seconds versus BitMapperBS at 18.96 seconds; bsbit's within-5-bp-correct throughput is 1.9% higher. With eight mapping CPUs plus the same two-CPU streaming allocation used by the paired protocol, bsbit takes 15.98 seconds, 15.7% less wall than BitMapperBS, and its correct-read throughput is 19.0% higher. BitMapperBS still uses less RSS (6.77 versus 7.30 GiB). bsbit has by far the lowest logical I/O (0.68 GiB). Its all-mapped within-5 result is 4,798,553 correct reads, 95.971% recall, and 96.182% precision. BISCUIT finds 5,114 more correct reads but takes 425.43 seconds and 11.46 GiB peak RSS.
Single-end unique placements now use numeric Q10/Q15/Q20/Q30/Q40 evidence tiers. At within 5 bp, Q10, Q20, Q30, and Q40 all pass their literal one-sided 95% error gates. Q40 retains 1,819,377 reads, with 121 errors (66.51 ppm) and a 77.34-ppm upper bound. Exact-coordinate Q40 has 289 errors and does not pass; exact Q10/Q20/Q30 do pass. BSBolt and BISCUIT retain more Q40-correct reads and have lower Q40 empirical error, while bsbit supplies the lowest logical I/O and the second-lowest wall time. The primary unique-scope within-5 PR-AUC is 93.645%, computed only over score-bearing unique reads.
Complete runtime/resource, output-completeness, exact/+/-1/+/-5/+/-10, error-category, exhaustive native classification, exact/within-5 unique-scope PR-AUC, and Q10/Q20/Q30/Q40 error tables, together with BAMs, commands, and source provenance, are retained in that local benchmark archive.
Reproduction protocol¶
Build the audited executable with the release script and verify its recorded provenance:
cargo build --locked --release -p bsbit-cli --bin bsbit
scripts/build-bsbit.sh
sha256sum build/bsbit/bsbit
After building the opaque index with bsbit index as described in the
alignment guide, the bsbit side of the timed
command has this public shape:
build/bsbit/bsbit align \
-x GRCh38.bsbit \
--read1 sim_wgbs_10m.R1.fastq.gz \
--read2 sim_wgbs_10m.R2.fastq.gz \
--output sensitive.bam \
--sensitive \
--threads 8 \
--compression-threads 2 \
--min-template-span 0 \
--max-template-span 500 \
--metrics > sensitive.summary.tsv
For a defensible comparison, retain warm-up commands, randomized block order, host/kernel/WSL configuration, full command lines, binary and input hashes, sampler output, BAM hashes, and truth-evaluator version. A single warm run is useful for local sizing but is not interchangeable with this qualification.
Robustness and adapter evidence¶
The sensitive rule was frozen before evaluation on nine corpora spanning read length, sequencing error, conversion rate, insert distribution, seed, and adapter tails. The complete machine-readable matrix is sensitive-promotion-condition-matrix-20260829.tsv.
Paired Default v2 is byte-identical to its predecessor on the clean 5M-pair corpus because that corpus contains no paired adapter trigger. Independent adapter-tail truth sets exercise the recovery rule; their results are default-adapter-recovery-truth-20260829.tsv.
Limits of this evidence¶
- Directional PE150 simulated WGBS on GRCh38 owns the primary speed and truth-accuracy claim.
- The frozen matrix broadens regression evidence, but it does not qualify every read length, organism, protocol, or error distribution.
- Non-directional paired-end and single-end have functional coverage, not this large-corpus performance/MAPQ qualification. Standard directional single-end has one controlled 5M-R1 performance/truth regression run, but no replicated MAPQ qualification; that run does not qualify non-directional single-end. Single-end exact-adapter recovery has focused functional regression coverage and is not included in those retained timing claims.
- Calling and cohort outputs are outside this aligner scorecard and require study-specific validation.
- The benchmark is not an independent third-party study.
Read this page with the scientific contract, sequencing data support, and limitations and roadmap.