Releases

• Please run lexicmap version to check update !!!
• Please run lexicmap autocompletion to update shell autocompletion script !!!

v0.5.0 - 2015-12-18

This version is compatible with indexes created by LexicMap v0.4.0, but rebuilding the index is recommended for more accurate results.

• New commands:
  • lexicmap utils remerge: Rerun the merging step for an unfinished index.
• lexicmap index:
  • Big genomes with thousands of contigs (big yet fragmented assemblies) are automatically split into multiple chunks, and alignments from these chunks will be merged.
  • Change the default value of --partitions from 1024 to 4096, which increases the seed-matching speed at the cost of 2 GiB more memory occupation. For existing lexicmap indexes, just run lexicmap utils reindex-seeds --partitions 4096 to re-create seed indexes.
  • Do not save seeds of low-complexity.
  • Fix high memory usage in writing seed data.
  • Change the default value of -c/--chunks from all available CPUs to the value of -j/--threads.
  • Change the default value of --max-open-files from 512 to 1024.
  • Add a new flag --debug.
• lexicmap search:
  • Improving chaining, pseudoalignment, and alignment for highly repetitive sequences.
  • More accurate chaining score with better chaining of overlapped anchors, this produces more accurate results with -n/--top-n-genomes:
    • Merging two overlapped non-gapped anchors into a longer one.
    • For these with gaps, only the non-overlapped part of the second anchor is used to compute the weight.
    • Using the score of the best chain (rather than the sum) for sorting genomes when using -n.
  • Fix positions and alignment texts for queries with highly repetitive sequences in end regions. #9
  • Skip seeds of low-complexity.
  • Change the default value of --max-open-files from 512 to 1024.
  • Change the default value of --align-band from 50 to 100.
  • Improve the speed of anchor deduplication, genome information extraction, and result ordering.
  • Improve the speed of chaining for long queries.
  • Improve the speed of seed matching when using -w/--load-whole-seeds.
  • Improve the speed of alignment, and reduce the memory usage.
  • Remain compatible after the change of lexicmap index.
  • Add a new flag --debug.
• lexicmap utils genomes:
  • Do not sort genome ids.
  • Add a header line and add another column to show if the reference genome is chunked.
• lexicmap utils subseq:
  • Remain compatible after the change of lexicmap index.
• lexicmap utils seed-pos:
  • Remain compatible after the change of lexicmap index, while histograms are plotted separately for multiple genome chunks.
• lexicmap utils reindex-seeds:
  • Change the default value of --partitions from 1024 to 4096.

v0.4.0 - 2024-08-15

• New commands:
  • lexicmap utils 2blast: Convert the default search output to blast-style format.
• lexicmap index:
  • Support suffix matching of seeds, now seeds are immune to any single SNP!!!, at the cost of doubled seed data.
  • Better sketching desert filling for highly-repetitive regions.
  • Change the default value of --seed-max-desert from 900 to 200 to increase alignment sensitivity.
  • Mask gap regions (N’s).
  • Fix skipping interval regions by further including the last k-1 bases of contigs.
  • Fix a bug in indexing small genomes.
  • Change the default value of -b, --batch-size from 10,000 to 5,000.
  • Improve lexichash data structure.
  • Write and merge seed data in parallel, new flag -J/--seed-data-threads.
  • Improve the log.
• lexicmap search:
  • Fix chaining for highly-repetitive regions.
  • Perform more accurate alignment with WFA.
  • Use buffered reader for seeds file reading.
  • Fix object recycling and reduce memory usage.
  • Fix alignment against genomes with many short contigs.
  • Fix early quit when meeting a sequence shorter than k.
  • Add a new option -J/--max-query-conc to limit the miximum number of concurrent queries, with a default valule of 12 instead of the number of CPUs, which reduces the memory usage in batch searching.
  • Result format:
    • Cluster alignments of each target sequence.
    • Remove the column seeds.
    • Add columns gaps, cigar, align, which can be reformated with lexicmap utils 2blast.
• lexicmap utils kmers:
  • Fix the progress bar.
  • Fix a bug where some masks do not have any k-mer.
  • Add a new column prefix to show the length of common prefix between the seed and the probe.
  • Add a new column reversed to indicate if the k-mer is reversed for suffix matching.
• lexicmap utils masks:
  • Add the support of only outputting a specific mask.
• lexicmap utils seed-pos:
  • New columns: sseqid and pos_seq.
  • More accurate seed distance.
  • Add histograms of numbers of seed in sliding windows.
• lexicmap utils subseq:
  • Fix a bug when the given end position is larger than the sequence length.
  • Add the strand ("+" or “-”) in the sequence header.

v0.3.0 - 2024-05-14

• lexicmap index:
  • Better seed coverage by filling sketching deserts.
  • Use longer (1000bp N’s, previous: k-1) intervals between contigs.
  • Fix a concurrency bug between genome data writing and k-mer-value data collecting.
  • Change the format of k-mer-value index file, and fix the computation of index partitions.
  • Optionally save seed positions which can be outputted by lexicmap utils seed-pos.
• lexicmap search:
  • Improved seed-chaining algorithm.
  • Better support of long queries.
  • Add a new flag -w/--load-whole-seeds for loading the whole seed data into memory for faster search.
  • Parallelize alignment in each query, so it’s faster for a single query.
  • Optional outputing matched query and subject sequences.
  • 2-5X searching speed with a faster masking method.
  • Change output format.
  • Add output of query start and end positions.
  • Fix a target sequence extracting bug.
  • Keep indexes of genome data in memory.
• lexicmap utils kmers:
  • Fix a little bug, wrong number of k-mers for the second k-mer in each k-mer pair.
• New commands:
  • lexicmap utils gen-masks for generating masks from the top N largest genomes.
  • lexicmap utils seed-pos for extracting seed positions via reference names.
  • lexicmap utils reindex-seeds for recreating indexes of k-mer-value (seeds) data.
  • lexicmap utils genomes for list genomes IDs in the index.

v0.2.0 - 2024-02-02

• Software architecture and index formats are redesigned to reduce searching memory occupation.
• Indexing: genomes are processed in batches to reduce RAM usage, then indexes of all batches are merged.
• Searching: seeds matching is performed on disk yet it’s ultra-fast.

v0.1.0 - 2024-01-15

• The first release.
• Seed indexing and querying are performed in RAM.
• GTDB r214 with 10k masks: index size 75GB, RAM: 130GB.