BioHPC Cloud:
: User Guide

Use cbsugpu05, 06, 07 to run alphapulldown. cbsugpu07 has the faster GPU, but cbsugpu05 and 06 each have two GPU units. At the end of this page, there are instructions how to use both GPU units on the same server.

Step 1: generate msa with colabfold (much faster than the alphafold2 based alignment pipeline)

#This step normally takes a few minutes for dozens of proteins. The actual computing happens on the colabfold cloud server, it could be busy sometimes. 

#Combine the bait and candidate protein sequences into a single fasta file, and put it under the current directory.

#Run the following command to generate msa files. In this command, my.fasta is the input fasta file name, msas is the output directory name. 

cd /workdir/$USER

singularity run --bind /local/local_data/colabfold_cache:/cache --bind $PWD --pwd $PWD /programs/colabfold-1.5.5/colabfold.sif colabfold_batch my.fasta msas --msa-only

Step 2: run create_individual_features.py on cbsugpu05-07

#This step runs on cbsugpu05-7. It does not require GPU, but requires the database on the GPU. It should take <1hour for dozens of proteins.

#run this command from the parental directory of msas directory created from step1. 

singularity exec --bind /local/local_data/alphafold-2.3.2:/db --bind $PWD --pwd $PWD /programs/alphapulldown-1.0.4/alphapulldown.sif \
create_individual_features.py \
  --fasta_paths=my.fasta \
  --data_dir=/db \
  --output_dir=msas \
  --use_precomputed_msas=True \
  --max_template_date=2024-01-01 \
  --use_mmseqs2=True \
  --skip_existing=False

Step 3: run run_multimer_jobs.pyon GPU (instructions of using two GPU units are at the end of the page)

#create two text file bait.txt,candidates.txt. The bait.txt has one line: bait protein name; the condidates.txt file has one protein name per line.

#run the command from the current directory, which contains: bait.txt,candidates.txt, msas directory, and an empty outputdir. it could take hours or days to finish depending on number of candidate proteins.

mkdir outputdir
#set environment variable to enable GPU to use system memory, so that large proteins can be processed
export TF_FORCE_UNIFIED_MEMORY=1
export XLA_PYTHON_CLIENT_MEM_FRACTION=4.0

singularity exec --nv --bind /local/local_data/alphafold-2.3.2:/db --bind $PWD --pwd $PWD /programs/alphapulldown-1.0.4/alphapulldown.sif \
run_multimer_jobs.py --mode=pulldown \
--num_cycle=3 \
--num_predictions_per_model=1 \
--output_path=./outputdir \
--data_dir=/db \
--protein_lists=bait.txt,candidates.txt \
--monomer_objects_dir=msas

Step 4: generate a summary table

#this step can be run on any server. you just need the outputdir from the previous step

#run the command from the current directory, which has the outputdir under it

singularity exec --bind $PWD --pwd $PWD /programs/alphapulldown-1.0.4/alpha-analysis_jax_0.4.sif run_get_good_pae.sh --cutoff=10 --output_dir=./outputdir 

Output is a csv file that you can open in Excel: predictions_with_good_interpae.csv

Step 5: create a jupyter notebook

#run the command in the outputdir

cd outputdir

singularity exec --bind $PWD --pwd $PWD /programs/alphapulldown-1.0.4/alphapulldown.sif create_notebook.py --cutoff=5.0 --output_dir=./

#after this step, you should see a file output.ipynb in the directory.

#start jupyter in the outputdir

singularity exec --bind $PWD --pwd $PWD /programs/alphapulldown-1.0.4/alphapulldown.sif jupyter lab --ip=0.0.0.0 --port=8017 --no-browser

#you should see a URL http://cbsuxxxxx.biohpc.cornell.edu:8017/labtoken=xxxxxxxxxxxxxxxxxxxxx, copy paste the URL into a web browser, open the output.ipynb in the left panel, and execute every step in the book

#QC plot can also be produced by alphapickle.

Execute run_multimer_jobs.py in parallel on two GPU or more units on the same server:
1. Calculate the number of jobs for run_multimer, which is "number_of_baits x number_of candidates"

2. Divide the jobs into two batches, and create two shell scripts. The job_index should be between 1 and number of jobs:
shell script 1: 
singularity exec --nv --env CUDA_VISIBLE_DEVICES=0 ... --job_index=1
singularity exec --nv --env CUDA_VISIBLE_DEVICES=0 ... --job_index=3
singularity exec --nv --env CUDA_VISIBLE_DEVICES=0 ... --job_index=5
...

shell script 2:
singularity exec --nv --env CUDA_VISIBLE_DEVICES=1 ... --job_index=2
singularity exec --nv --env CUDA_VISIBLE_DEVICES=1 ... --job_index=4
singularity exec --nv --env CUDA_VISIBLE_DEVICES=1 ... --job_index=6
...

3. Run both shell scripts simultaneously in "screen" session
sh script1.sh >& log1 &
sh script2.sh >& log2 &

4. monitor whether both GPUs are used:
nvidia-smi