MMseqs2: Error in expandaln module when applying MMseqs to custom database.

To whom it may concern,

I want to use MMseqs to search MSAs in my own database (a .fasta file about 50G), but it stuck in the ‘expandaln’ module.

Pipeline to setup database

I use MMseqs-GPU so the brief pipeline is: (1) setup CPU database (2) setup GPU database (3) make the GPU database expandable (4) create index for GPU database for faster speed.

DB_NAME="v1rep_renum"
FASTA_NAME="v1rep_renum.fasta"
BASE_DIR="/Database"
TMP_DIR="./tmp"

# Directory settings
INPUT_FASTA=${BASE_DIR}/${FASTA_NAME}
OUTPUT_DIR=${BASE_DIR}/${DB_NAME}_mmseqs2
CPU_TEMP=${BASE_DIR}/${DB_NAME}_tmp

# ====================================================

# (1) CPU Database
mkdir -p $CPU_TEMP
chmod 754 $CPU_TEMP
mmseqs createdb $INPUT_FASTA ${CPU_TEMP}/${DB_NAME}

# (2) GPU Database
mkdir -p $OUTPUT_DIR
chmod 754 $OUTPUT_DIR
mmseqs makepaddedseqdb ${CPU_TEMP}/${DB_NAME} ${OUTPUT_DIR}/${DB_NAME}

# Remove CPU tmp
mmseqs rmdb ${CPU_TEMP}/${DB_NAME}
rm -rf ${CPU_TEMP}

# (3) Expandable Search Database
mmseqs cluster \
  ${OUTPUT_DIR}/${DB_NAME} \
  ${OUTPUT_DIR}/${DB_NAME}_clu \
  $TMP_DIR \
  --min-seq-id 0.3 -c 0.8

# disable E-value threshold with -e inf, accept everything that was clustered
mmseqs align \
  ${OUTPUT_DIR}/${DB_NAME} \
  ${OUTPUT_DIR}/${DB_NAME} \
  ${OUTPUT_DIR}/${DB_NAME}_clu \
  ${OUTPUT_DIR}/${DB_NAME}_aln \
  -a -e inf

mmseqs result2profile \
  ${OUTPUT_DIR}/${DB_NAME} \
  ${OUTPUT_DIR}/${DB_NAME} \
  ${OUTPUT_DIR}/${DB_NAME}_aln \
  ${OUTPUT_DIR}/${DB_NAME}_prof

mmseqs profile2consensus \
  ${OUTPUT_DIR}/${DB_NAME}_prof \
  ${OUTPUT_DIR}/${DB_NAME}_cons

mmseqs prefixid ${OUTPUT_DIR}/${DB_NAME}_cons ${OUTPUT_DIR}/${DB_NAME}.tsv --tsv --threads 32
mmseqs prefixid ${OUTPUT_DIR}/${DB_NAME} ${OUTPUT_DIR}/${DB_NAME}_seq.tsv --tsv --threads 32
mmseqs prefixid ${OUTPUT_DIR}/${DB_NAME}_h ${OUTPUT_DIR}/${DB_NAME}_h.tsv --tsv --threads 32
mmseqs prefixid ${OUTPUT_DIR}/${DB_NAME}_aln ${OUTPUT_DIR}/${DB_NAME}_aln.tsv --tsv --threads 32
mmseqs tsv2exprofiledb ${OUTPUT_DIR}/${DB_NAME} ${OUTPUT_DIR}/${DB_NAME}

# (4) Index
mmseqs createindex ${OUTPUT_DIR}/${DB_NAME} $TMP_DIR --remove-tmp-files 1 --threads 32

Pipeline to do the search job

After the setup process (also after I make sure that the above pipeline is finished without any error), I run the following command:
mmseqs createdb query.fasta ./outdir/qdb --shuffle 0
mmseqs search ./outdir/qdb ./Database/v1rep_renum_mmseqs2/v1rep_renum ./outdir/result ./tmp --threads 16 --num-iterations 3 --db-load-mode 2 -a -e 0.1 --max-seqs 10000 --gpu 1 --prefilter-mode 1
mmseqs mvdb ./tmp/latest/profile_1 ./outdir/prof_res
mmseqs lndb ./outdir/qdb_h ./outdir/prof_res_h
mmseqs expandaln ./outdir/qdb ./Database/v1rep_renum_mmseqs2/v1rep_renum.idx ./outdir/result ./Database/v1rep_renum_mmseqs2/v1rep_renum.idx ./outdir/res_exp --db-load-mode 2 --threads 16 --expansion-mode 0 -e inf --expand-filter-clusters 1 --max-seq-id 0.95

Error information

I0506 15:15:57.246535 22734209480448 mmseqs2.py:41] Running /xcfhome/jzlu/Software/MMseqs2/build/bin/mmseqs expandaln /tmp/tmpt0dhojz7/qdb /xcfhome/jzlu/Database/v1rep_renum_mmseqs2/v1rep_renum.idx /tmp/tmpt0dhojz7/result /xcfhome/jzlu/Database/v1rep_renum_mmseqs2/v1rep_renum.idx /tmp/tmpt0dhojz7/res_exp --db-load-mode 2 --threads 16 --expansion-mode 0 -e inf --expand-filter-clusters 1 --max-seq-id 0.95
expandaln /tmp/tmpt0dhojz7/qdb /xcfhome/jzlu/Database/v1rep_renum_mmseqs2/v1rep_renum.idx /tmp/tmpt0dhojz7/result /xcfhome/jzlu/Database/v1rep_renum_mmseqs2/v1rep_renum.idx /tmp/tmpt0dhojz7/res_exp --db-load-mode 2 --threads 16 --expansion-mode 0 -e inf --expand-filter-clusters 1 --max-seq-id 0.95 

Index version: 16
Generated by:  0889c78b502c5380616cd60f1b197ed62f35af52
ScoreMatrix:  VTML80.out
Index version: 16
Generated by:  0889c78b502c5380616cd60f1b197ed62f35af52
ScoreMatrix:  VTML80.out
[=================================================================terminate called after throwing an instance of 'std::logic_error'
  what():  basic_string: construction from null is not valid

Environment:

MMseqs version: 0889c78b502c5380616cd60f1b197ed62f35af52 (version 16) Submit job with SLRUM system on a node with both CPU and GPU, memory is about 400G.

Thanks for your reply. Hope that I have clarified my problem.

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 15 (4 by maintainers)

Most upvoted comments

I was also able to run the example above without issues. The --gpu flag is critical for tsv2exprofiledb. Thank you @camel2000 for helping here.

@jianzhang-lu try this one, when you build the database

mmseqs createdb ${FASTA} ${DATA_DIR}/seqdb

# parameter choice is very important here, generally you want to cluster to a low sequence identity however keep a high coverage.
# Without a high coverage, we might lose a domain in the representative sequence and then not be able to find the domain in any of the members anymore, since we always first need to match the cluster representative
mmseqs cluster ${DATA_DIR}/seqdb ${DATA_DIR}/clu ${DATA_DIR}/tmp --min-seq-id 0.3 -c 0.8
# disable E-value threshold with -e inf, accept everything that was clustered
mmseqs align ${DATA_DIR}/seqdb ${DATA_DIR}/seqdb ${DATA_DIR}/clu ${DATA_DIR}/aln -a -e inf
mmseqs result2profile ${DATA_DIR}/seqdb ${DATA_DIR}/seqdb ${DATA_DIR}/aln ${DATA_DIR}/prof
mmseqs profile2consensus ${DATA_DIR}/prof ${DATA_DIR}/cons
mmseqs prefixid ${DATA_DIR}/cons ${DATA_DIR}/${DBNAME}.tsv --tsv --threads 1
mmseqs prefixid ${DATA_DIR}/seqdb ${DATA_DIR}/${DBNAME}_seq.tsv --tsv --threads 1
mmseqs prefixid ${DATA_DIR}/seqdb_h ${DATA_DIR}/${DBNAME}_h.tsv --tsv --threads 1
mmseqs prefixid ${DATA_DIR}/aln ${DATA_DIR}/${DBNAME}_aln.tsv --tsv --threads 1
mmseqs tsv2exprofiledb ${DBNAME} ${DATA_DIR}/${DBNAME} --gpu 1 # Pay attention here.


mmseqs createindex ${DATA_DIR}/${DBNAME} ${DATA_DIR}/tmp  

# # build GPU padded database
mmseqs makepaddedseqdb ${DATA_DIR}/${DBNAME} ${DATA_DIR}/${DBNAME}_gpu

# # build GPU database index
mmseqs createindex ${DATA_DIR}/${DBNAME}_gpu ${DATA_DIR}/tmp --index-subset 2 --split 1 


# nohup mmseqs gpuserver ${DATA_DIR}/${DBNAME}_gpu > ${DATA_DIR}/gpuserver.log 2>&1 &