BioHPC Cloud:
: User Guide

Cornell Library has the license to Repbase. Please DO NOT share the database files with groups outside Cornell.

Download the database files using a computer on campus. From outside Cornell campus (including with VPN), you can access the website through the library proxy server (a valid Cornell NetID is required) .

Repbase provides two sets of data files, both can be used for RepeatMasker.

1. Repbase RepeatMasker Edition (latest version 10/26/2018)

• Includes modification by the RepeatMasker team, with TE classifications formatted for RepeatMasker/RepeatClassifier;
• No update since 2018;

2. New Repbase release in embl or fasta format (latest version as checked on 5/18/2021: v26.04 04/22/2021)

• More species included;
• Up to date;

How to use the Repbase RepeatMasker Edition with RepeatMasker. 

Scenario 1. Your species is covered in the Repbase. No need to create a de novo custom database. 

a) Setup the RepeatMasker library from Repbase (You only need to do it once)

#Download the file RepBaseRepeatMaskerEdition-20181026.tar.gz from Repbase

#download the tetools docker image and convert it to a singularity image file tetools.sif  
cd /workdir/$USER
singularity pull tetools.sif docker://dfam/tetools:latest

# Make a copy of RepeatMasker's Libraries directory here
singularity exec --bind $PWD --pwd $PWD tetools.sif cp -r /opt/RepeatMasker/Libraries ./

# Extract RepBase (replace "/work/path/to" with path to the file)
tar -x -f /work/path/to/RepBaseRepeatMaskerEdition-20181026.tar.gz 

# Run the 'addRepBase.pl' script (part of the RepeatMasker package) to merge the databases,
# After this step, you should see a new RepeatMaskerLib.h5 which combines the Dfam and Repbase
singularity exec  --bind $PWD --pwd $PWD tetools.sif addRepBase.pl -libdir Libraries/

b) Next time you run RepeatMasker, you just need to set environment variable LIBDIR to the Libraries directory. RepeatMasker will use the Repbase.

# Run RepeatMasker with the LIBDIR environment variable set, using Insecta as an example
export LIBDIR=/workdir/$USER/Libraries

singularity exec --bind $LIBDIR  --bind $PWD --pwd $PWD /work/path/to/tetools.sif RepeatMasker -species Insecta [add other RepeatMasker options] genome.fa

Scenario 2. Your species is not well covered in the Repbase. You need to create a de novo custom repeat library from the genome assembly.

It is recommended that you create the de novo library using RepeatModeler, you may want to combine the library from RepeatModeler with the Repbase library. To merge the two libraries, people have used two different approaches,

• Concatenate the two repeat libraries into one file, optionally remove the redudancy (probably not needed). Then run RepeatMasker with the "-lib myCombined.lib" option.
• Alternatively you can run RepeatMasker sequencially, first use "-species mySpecies" option against Repbase and create a masked genome, then run  RepeatMasker on the masked genome with "-lib myCustom.lib" option.

Here are the instructions how to combine the two libraries.

#create a Repbase fasta file from "RepeatMaskerLib.h5" created in the previous step, using insect as an example 
singularity exec --bind $PWD --pwd $PWD /work/path/to/tetools.sif famdb.py -i RepeatMaskerLib.h5 families --format fasta_name --include-class-in-name --ancestors --descendants 'Insecta' > insect.lib

#then combine with custom library
cat insect.lib myCustom.lib > myCombined.lib

How to use the Repbase latest release

• The full release file is RepBaseXX.XX.embl.tar.gz, which includes separate files for each taxonomy clade (here is the documentation for the file names);
• You can also download data file from one clade only (From the Repbase web site, first click the directory link RepBaseXX.XX.embl to access these files .); 

Convert the Repbase embl file to RepeatMasker library file (a fasta file, the sequence title include class/subclass of the TE elements). embl2rm.pl in PATH of BioHPC computers. It can also be downloaded from https://bitbucket.org/cornell_bioinformatics/embl2rm . Do not include the simple.ref file from Repbase for RepeatMasker. 

embl2rm.pl invrep.ref invrep.fasta

If you have a custom library built with RepeatModeler, you can optionally combine the custom library with the Repbase library file. When running RepeatMasker, use the parameter "-lib" to point to this library file.