below in the code there are a couple of errors that wordpress don’t let me fix. The > and < are replaced by their literal html enconding. I am working on it, sorry

#!/usr/bin/env python

import fasta
import sys
from collections import defaultdict

def choose(n, k):
    if 0 <= k <= n:
        ntok = 1
        ktok = 1
        for t in xrange(1, min(k, n - k) + 1):
            ntok *= n
            ktok *= t
            n -= 1
        #print ntok // ktok
        return ntok // ktok
    else:
        return 0

def get_quorums(seqs, mlen):
    """
    add seq id_no to a set
    use explicit counter to create seq_no
    """
    quorum = defaultdict(set)
    id_no = 0
    for seq in seqs:
        id_no += 1
        for n in range(len(seq) - mlen):
            quorum[seq[n:n + mlen]].add(id_no)
    return quorum

input_seqs = fasta.read_seqs(open(sys.argv[1]).readlines())
input_seqs2 = fasta.read_seqs(open(sys.argv[2]).readlines())

foreground = get_quorums(input_seqs, 10)
background = get_quorums(input_seqs2, 10)

N = len(input_seqs) + len(input_seqs2)

for i in foreground:
    term1 = choose(len(background[i]), len(foreground[i]))
    term2 = choose((N - len(background[i])), len(input_seqs)-1)
    term3 = choose(N, len(input_seqs))
    p = (float(term1) * float(term2)) / term3
    if 0 < p <= 0.0001:
        print i, len(foreground[i]), len(background[i]), p

We already defined choose in the last post (more information in the link from the Python’s cookbook) and earlier Mike sent us a series of quorum-determination functions and one of the best was portrayed and explained here. We also need our fasta module to read the sequences (and only the sequences) in order to use it in the quorum function.

Basically we use the foreground and background files as input, determine the quorum of the different words (width 10) and then we iterate over the results, calculating the p value for each motif found in the foreground set. The tree terms of the Hypergeometric Distribution are calculated separately and we test for a p value smaller that 0.0001 (this can be modified) and we only print the results that fall in this category.>

So, we have the dnatranslate.py and the fasta.py that we are going to import into our script. And that’s basically it: calling function already created, stored in modules that can be reused anytime. In the end our script that translates DNA sequences to proteins takes a little bit more than a handful of lines.

#!/usr/bin/env python

import dnatranslate
import sys
import fasta

dna = fasta.read_fasta(open(sys.argv[1], 'r').readlines())

for item in dna:
    protein = dnatranslate.translate_dna(item.sequence)
    print item.name
    print protein

That’s it. A good example of reusable code, that once created fits everywhere and handles most type of data. We read the FASTA file in the first line, and iterate over the items created translating them as we go. As an extra exercise, we can include the output formatting function. First we need to update the fasta.py module (already on the repository) and slightly change the formatting function, that ends up looking like this

def format_output(sequence, length):
    temp = []
    for j in range(0,len(sequence),length):
        temp.append(sequence[j:j+length])
    return '\n'.join(temp)

For this case the ideal formatting function would go through the “longer” route mentioned before, because the final printing should be done by the main script and not by the imported module. This gives us more control on what we want to do with the resulting string. The format_output function receives two arguments: the first is the actual DNA/protein sequence to be formatted and the length we want to output it. We had to remove the loop too, so only one sequence can be sent to the function and, as pointed, the function returns a string with the formatted sequence. In the end our post’s initial sequence has one modification only

#!/usr/bin/env python

import dnatranslate
import sys
import fasta

dna = fasta.read_fasta(open(sys.argv[1], 'r').readlines())

for item in dna:
    protein = dnatranslate.translate_dna(item.sequence)
    print item.name
    print fasta.format_output(protein, 60)

the last line, that instead of printing directly the result of the translation, sends the sequence to the formatting function before.

Basically, our FASTA reader strips all the carriage returns/new lines (if existent) from the input file, so the sequence itself is stored continuously in a string. Of course we could do it differently and store the sequence as is in the file, but then they might differ on the number of characters per line and other features. Our goal then is to break the sequence in chunks, making it look like a justified paragraph. We use the code below to obtain this, back after it.

#!/usr/bin/env python

import fasta
import sys

sequences = fasta.read_fasta(open(sys.argv[1], 'r').readlines())

temp = []
for i in sequences:
    print i.name
    for j in range(0,len(i.sequence),80):
        temp.append(i.sequence[j:j+80])
    print '\n'.join(temp)
    temp = []

The script, as always, is very simple and does the job. A new feature of the range method is introduced: the step. From previous posts we have seen that function range generates a sequence of integers based either on one value

>>>range(5)
>>>[0, 1, 2, 3, 4]

or two values

>>>range(10,15)
>>>[10, 11, 12, 13, 14]

The step parameter makes the generated to jump a certain number of values.

>>>range(10, 20, 2)
>>>[10, 12, 14, 16, 18]

This is similar to the for loop in C/C++ where the loop is define by three parameters on a construction like this

for (i = 0; i < 10; i++)

where the first parameter sets the initial value of i, the second value sets the maximum value of i and the last one sets the step that i will be incremented.

In our small script above, we read the sequence (this time putting everything in one line), get the sequences and then iterate along the instances of our class, printing first the name of the sequence. Then we iterate on the whole length of each sequence, this time generating a range that starts on zero and goes up to the final nucleotide/amino acid and has a step that defines the number of characters we want to print in one line.

To print the sequence to the screen we had two options: directly print using

print i.sequence[j:j+80]

or use the longer route shown in our script. Trying this script will output the sequences from the original file with 80 characters along the line. Next following the book we will write a script to translate DNA to protein using our FASTA reading function and modify our FASTA module to include the formatted output function.

Please notice that the code uploaded to the repository contains the shortest route, just to show the differences

class Fasta:
    def __init__(self, name, sequence):
        self.name = name
        self.sequence = sequence

def read_fasta(file):
    items = []
    index = 0
    for line in file:
        if line.startswith(">"):
           if index >= 1:
               items.append(aninstance)
           index+=1
           name = line[:-1]
           seq = ''
           aninstance = Fasta(name, seq)
        else:
           seq += line[:-1]
           aninstance = Fasta(name, seq)

    items.append(aninstance)
    return items

We are going to reuse this code several times, and now we only need to import by using

import fasta

and we are done. This will make everything easier. Please, be aware in the following posts the use of this simple module and download it from the repository if needed.

class Fasta:
    def __init__(self, name, sequence):
        self.name = name
        self.sequence = sequence

That’s all we need for now. Of course we could include sequence length, sequence type, or any other relevant information as a class attribute. But we will stick with those for the time being. NOw that we have the class declared, we need to read the file and create instances of the Fasta class to store all our sequences.

We already know how to open and read a file and create functions. We put everything together in a simple script:

import sys
#class declaration with both attributes we need
class Fasta:
    def __init__(self, name, sequence):
        #this will store the sequence name
        self.name = name
        #this  will store the sequence itself
        self.sequence = sequence

#this function will receive the list with the file
#contents, create instances of the Fasta class as
#it scans the list, putting the sequence name on the
#first attribute and the sequence itself on the second
#attribute
def read_fasta(file):
    #we declare an empty list that will store all
    #Fasta class instances generated
    items = []
    index = 0
    for line in file:
    #we check to see if the line starts with a > sign
        if line.startswith(">"):
           #if so and our counter is large than 1
           #we add the created class instance to our list
           #a counter larger than 1 means we are reading
           #from sequences 2 and above
           if index >= 1:
               items.append(aninstance)
           index+=1
           #we add the line contents to a string
           name = line[:-1]
           #and initialize the string to store the sequence
           seq = ''
           #this creates a class instance and we add the attributes
           #which are the strings name and seq
           aninstance = Fasta(name, seq)
        else:
           #the line does not start with > so it has to be
           #a sequence line, so we increment the string and
           #add it to the created instance
            seq += line[:-1]
            aninstance = Fasta(name, seq)

    #the loop before reads everything but the penultimate
    #sequence is added at the end, so we need to add it
    #after the loop ends
    items.append(aninstance)
    #a list with all read sequences is returned
    return items

fastafile = open(sys.argv[1], 'r').readlines()
mysequences = read_fasta(fastafile)

print mysequences

for i in mysequences:
    print i.name

At first, it looks scary. But is not. There are many ways to create this loop and to read the sequences, and many ways to make this loop shorter. We will get there, eventually, but for starters this is OK. Basically, in the above script we have a Fasta class, a read_fasta function and a couple of lines to read a print the results. The read_fasta function basically checks all items of a list and see which is the first character of each item: a > sign indicates that the item should be parsed to the temporary name string, another character redirects the item to the sequence string. Instances are created on the fly and the attributes assigned as what was file content is being scanned.

In the end, just to be sure of what we accomplished with the function we print the list and loop through it and print the name of each sequence read. Notice that the loop counter/index is the one that receives the class attribute.

The biggest advantage of this way of reading FASTA files is that the class and the function are reusable. Basically we can create a file (for instance called readfasta.py) and import into another script and we have a FASTA reader ready to rock. We would need to tweak a little bit in order to catch exceptions, but with consistent FASTA files this would work fine.

A great part of bioinformatics is to store data, how to store it and which format to use. Anyone working in a wetlab or on a pure bioinformatics lab had had problems with file formats one day or another. In fact, you don’t have to be in a bioinformatics environment to have such problems, but in biology people tend to create a file format for everyt program they write. We can call it a lack of standards, sometimes. But one very well established format is the FASTA (pronounced FAST-Aye, according to the EMBl-EBI page for the software with identical name. This format is really simple and easy to manipulate with most computer languages and being text-based adds an extra advantage of portability for the files. Usually a FASTA file has a structure like this

>title/name/extra information about the sequence
sequence in one or many lines

There is no limit on the number of sequences that can be stored in a file, neither on the size of each sequence. Usually sequences that are larger than 70-80 nucleotides/amino acids are displayed in multiple lines.

>Q15465|SHH_HUMAN Sonic hedgehog protein – Homo sapiens (Human).
MLLLARCLLLVLVSSLLVCSGLACGPGRGFGKRRHPKKLTPLAYKQFIPNVAEKTLGASG
RYEGKISRNSERFKELTPNYNPDIIFKDEENTGADRLMTQRCKDKLNALAISVMNQWPGV
KLRVTEGWDEDGHHSEESLHYEGRAVDITTSDRDRSKYGMLARLAVEAGFDWVYYESKAH
IHCSVKAENSVAAKSGGCFPGSATVHLEQGGTKLVKDLSPGDRVLAADDQGRLLYSDFLT
FLDRDDGAKKVFYVIETREPRERLLLTAAHLLFVAPHNDSATGEPEASSGSGPPSGGALG
PRALFASRVRPGQRVYVVAERDGDRRLLPAAVHSVTLSEEAAGAYAPLTAQGTILINRVL
ASCYAVIEEHSWAHRAFAPFRLAHALLAALAPARTDRGGDSGGGDRGGGGGRVALTAPGA
ADAPGAGATAGIHWYSQLLYQIGTWLLDSEALHPLGMAVKSS

So, let’s write a function and a small script to read a typical FASTA file and display the output. Later we will see how to manipulate the file and in the end we will have created a list of scripts that will be useful for the everyday laboratory life.

But before we get to our final goal, we need to learn some new features of Python. In our case Classes. Classes in Python are very similar to classes in C++, and they are the building blocks of Object Oriented Programming (OOP). This guide will only scratch the surface of OOP, as it is a very complex subject, requiring a guide of its own. There are plenty of good introductory material online and any Google search will return dozens of links.

Let’s focus on some basic concepts, what will be exactly what we will need here. Classes are basically objects with associated properties (attributes) and methods. A traditional introductory exampple of classes is the employee list. In a company all employees are registered and they basic information are stored in the human resources file system. Let’s call the main class Employee with three attributes: name, room_number and favourite_colour. This defines a class. Let’s see how to do that in Python:

class Employee:
    def __init__(self, name, room, colour):
        self.name = name
        self.room = room
        self.colour = colour

Let’s dissect this piece of code. First we declare a class and give a name to it Employee. The next thing we need to do is to define the initiation (constructor) method of the class and give it the attributes we need, that’s the __init__ method. Everytime we create a copy of the main class object the copy will be initialized with the values we are passing to the method. According to Dive into Python “The first argument of every class method, including __init__, is always a reference to the current instance of the class.”. That’s the self on the method definition, which is followed by what we want to store in the object: name, room and favourite colour. The lines in the method are the ones assigning the received values to the different class’ attributes. Again from Dive into Python we have “To reference this attribute from code outside the class, you qualify it with the instance name, instance.data, in the same way that you qualify a function with its module name. To reference a data attribute from within the class, you use self as the qualifier.”. In other words the attributes of the class are seen internally by it by the use of self while from the outside (another part of the script) it is seen by the name given to the instance of the class. It is a little bit confusing, so let’s see an example below.

class Employee:
    def __init__(self, name, room, colour):
        self.name = name
        self.room = room
        self.colour = colour

employeename = "Paulo"
roomnumber = "21"
colour = "blue"

newemployee = Employee(employeename, roomnumber, colour)
print newemployee

If we run this simple code we will get something like this

<__main__.Employee instance at 0x7379cc>

And that’s exactly what we need here: that’s a class instance that was created by

newemployee = Employee(employeename, roomnumber, colour)

Printing this information is not very useful. We don’t want to know the memory address of the object, we want to get the attributes of it, so we need to change print newemployee by these lines:

print newemployee.name
print newemployee.room
print newemployee.colour

There they are, run it and see what is being printed to the screen. Of course, as is this script does not accomplish anything useful, but imagine that you need to read a file with 1000 new employees everyt month. That’s what we need to do with FASTA files everyday and we will see how to do that on the next post.

Last time we have seen a dictionary structure that contained our genetic code. We are going to see a script that will translate DNA into proteins and we are also going to see how to create a module in Python that can be imported in any script. This module will contain the function that translates the DNA, and for the time being only that.

Let’s see how our module looks like:

def translate_dna(sequence):

    gencode = {
    'ATA':'I', 'ATC':'I', 'ATT':'I', 'ATG':'M',
    'ACA':'T', 'ACC':'T', 'ACG':'T', 'ACT':'T',
    'AAC':'N', 'AAT':'N', 'AAA':'K', 'AAG':'K',
    'AGC':'S', 'AGT':'S', 'AGA':'R', 'AGG':'R',
    'CTA':'L', 'CTC':'L', 'CTG':'L', 'CTT':'L',
    'CCA':'P', 'CCC':'P', 'CCG':'P', 'CCT':'P',
    'CAC':'H', 'CAT':'H', 'CAA':'Q', 'CAG':'Q',
    'CGA':'R', 'CGC':'R', 'CGG':'R', 'CGT':'R',
    'GTA':'V', 'GTC':'V', 'GTG':'V', 'GTT':'V',
    'GCA':'A', 'GCC':'A', 'GCG':'A', 'GCT':'A',
    'GAC':'D', 'GAT':'D', 'GAA':'E', 'GAG':'E',
    'GGA':'G', 'GGC':'G', 'GGG':'G', 'GGT':'G',
    'TCA':'S', 'TCC':'S', 'TCG':'S', 'TCT':'S',
    'TTC':'F', 'TTT':'F', 'TTA':'L', 'TTG':'L',
    'TAC':'Y', 'TAT':'Y', 'TAA':'_', 'TAG':'_',
    'TGC':'C', 'TGT':'C', 'TGA':'_', 'TGG':'W',
    }

    print sequence
    proteinseq = ''
    for n in range(0,len(sequence),3):
        if gencode.has_key(sequence[n:n+3]) == True:
            proteinseq += gencode[sequence[n:n+3]]

    return proteinseq

One function, called translate_dna, that receives a DNA sequence and outputs a protein sequence. It is a “long” function because we have in the middle the genetic code in Python’s dictionary format. Our translation loop is very simple, it reads the DNA sequence three nucleotides at a time

for n in range(0,len(sequence),3):

means that it will loop from 0 to the size of the DNA sequence in steps of 3, so basically we start at 0 and then jumping directly to 3. This is done to obey the translation structure based on codons of three nucleotides. Sometimes the DNA sequence entered does not have a size multiple of three and that’s the reason we use an error checking before accessing the dictionary

if gencode.has_key(sequence[n:n+3]) == True:

This will test for any possible error, or codons that are smaller than three nucleotides. If the key exists it is returned and addes to our protein string. The code that will use this function is this:

#!/usr/bin/env python

import dnatranslate

dnafile = open("AY162388.seq", 'r').readlines()

sequence = ''
for line in dnafile:
    sequence += line.strip()

protein = dnatranslate.translate_dna(sequence)

print sequence, len(sequence)
print
print protein, len(protein)

No secrets or new things here. Just notice that we import a module which is not part of the common Python modules, but was created by us. In this case the identification in the import will be the name of the .py file that contains the function(s) we are going to use. This file also needs to be located in the same directory of the script, if not installed in the Python libraries/modules directory. Notice that to use the function we need to call

protein = dnatranslate.translate_dna(sequence)

as we would do with the sys or the re modules. We can now create any modules that will contain different functions that can be reused anytime without need of extra coding. For instance, someone can create a module that would read a FASTA file and return sequences and sequence names in string or a list and send it to anyone interested in the same functionality. Everything we would need to do is to have this file installed in Python or in the same directory of our script and we would take advantage of all functionality contained in the module. That easy.

first_dictionary = {}

inside the curly braces we first assign a key and separated by a colon (:), while multiple pairs should be separated by comma. Both key and value have to be between single or double quotes. Let’s see how we will represent the genetic code in a Python dictionary, assigning values to keys

gencode = {
    'ATA':'I',    #Isoleucine
    'ATC':'I',    #Isoleucine
    'ATT':'I',    # Isoleucine
    'ATG':'M',    # Methionine
    'ACA':'T',    # Threonine
    'ACC':'T',    # Threonine
    'ACG':'T',    # Threonine
    'ACT':'T',    # Threonine
    'AAC':'N',    # Asparagine
    'AAT':'N',    # Asparagine
    'AAA':'K',    # Lysine
    'AAG':'K',    # Lysine
    'AGC':'S',    # Serine
    'AGT':'S',    # Serine
    'AGA':'R',    # Arginine
    'AGG':'R',    # Arginine
    'CTA':'L',    # Leucine
    'CTC':'L',    # Leucine
    'CTG':'L',    # Leucine
    'CTT':'L',    # Leucine
    'CCA':'P',    # Proline
    'CCC':'P',    # Proline
    'CCG':'P',    # Proline
    'CCT':'P',    # Proline
    'CAC':'H',    # Histidine
    'CAT':'H',    # Histidine
    'CAA':'Q',    # Glutamine
    'CAG':'Q',    # Glutamine
    'CGA':'R',    # Arginine
    'CGC':'R',    # Arginine
    'CGG':'R',    # Arginine
    'CGT':'R',    # Arginine
    'GTA':'V',    # Valine
    'GTC':'V',    # Valine
    'GTG':'V',    # Valine
    'GTT':'V',    # Valine
    'GCA':'A',    # Alanine
    'GCC':'A',    # Alanine
    'GCG':'A',    # Alanine
    'GCT':'A',    # Alanine
    'GAC':'D',    # Aspartic Acid
    'GAT':'D',    # Aspartic Acid
    'GAA':'E',    # Glutamic Acid
    'GAG':'E',    # Glutamic Acid
    'GGA':'G',    # Glycine
    'GGC':'G',    # Glycine
    'GGG':'G',    # Glycine
    'GGT':'G',    # Glycine
    'TCA':'S',    # Serine
    'TCC':'S',    # Serine
    'TCG':'S',    # Serine
    'TCT':'S',    # Serine
    'TTC':'F',    # Phenylalanine
    'TTT':'F',    # Phenylalanine
    'TTA':'L',    # Leucine
    'TTG':'L',    # Leucine
    'TAC':'Y',    # Tyrosine
    'TAT':'Y',    # Tyrosine
    'TAA':'_',    # Stop
    'TAG':'_',    # Stop
    'TGC':'C',    # Cysteine
    'TGT':'C',    # Cysteine
    'TGA':'_',    # Stop
    'TGG':'W',    # Tryptophan
}

Simple, yet efficient. But this is the type of functionality that would be great to have at hand everytime you write a script to translate DNA into proteins. And it is not something that you would like to type (or even copy-and-paste) all the time. On the next post we will create the translation script and will also create our first Python module.