

Simlis 2.0 examples: simgen.txt
-------------------------------

As commonly known there is a Human Genome Project to explore the
contents of all human chromosomes as sequences of nitrogenous bases.
There are 4 nitrogenous bases possible abbreviated with the letters A, C, G, T.

Therefore the results as published are (huge) files consisting
only of these A C G T sequences, e.g. the human chromosome 1 
is described in a file of 18.64 megabytes (as a pre-release) starting with:

GATCAATGAGGTGGACACCAGAGGCGGGGACTTGTAAATAACACTGGGCTGTAGGAGTGA
TGGGGTTCACCTCTAATTCTAAGATGGCTAGATAATGCATCTTTCAGGGTTGTGCTTCTA
followed by 315343 of similar looking lines.

These chromosome files are publicly available and can be downloaded
from ncbi.nlm.nih.gov/genome/seq or http://promo.net/pg.

When doing more research on this "raw data" questions may arise as:
What are the longest multiply occuring substring within the same chromosome,
within different chromosomes, even between chromosomes of different species?
Or, how often occuring structures look like ?

Here Simlis could be of some help.

Example 1: 
----------
Look up the longest common substrings with multiple occurances within
chromosome 1, as can be found by examinating 2% of the whole file and
by splitting into pseudo records of 200 bytes length:

simlis -rv -a9999 -g200,200,2 <chromo1.txt >result
where
-r show byte position of input data
-v case sensitive (here to avoid conversions as only letters ACGT used in input
-g200,200,2  from the logically unstructured input generate pseudo records
             of length 200 (range 200..200) selecting 2% of data picked
	     regularly from the whole input, before evaluation.
	     Ignore physical line break characters on input.
-a9999 sort on longest substrings of consecutive letters within the records

Looking through the result we find within 1954 output lines:	     
.
.	     
#17569800# GCGATCTCGGCTCACTGCAAGCTCCGCCTCCC
#19489800# TACAGGCGCCCGCTACCACGCCCGGCTAATTTTTTGTATTTTTAGTAGAGACGGGGTTTCACCG-
           TGTTAGCCAGGATGGTCTCGATCTCCTGACCTCGTGATCCACCCGCCT
.
.
above the longest sequence (112 symbols) occuring twice next to
input byte positions 17569800 and 19489800.
Note: The FIRST line of a sequence of lines containing the same substring
(here address 17569800) cannot show that substring because that string shown
there is related to a previous line (not indicated here). Just the subsequent
(here address 19489800) line(s) can show it.

Example 2: 
----------
We sort 2% of the input, splitted into pseudo records of 70 bytes length
each, on the similarity of all substrings contained within these records
ordering them into output clusters.

simlis -dvp -a10 -g70,70,2 <chromo1.txt >result
where
-d   show similarity score on output.
-v   case sensitive (here to avoid conversions as only 4 capital letters used).
-p   work substring position sensitive.
-a10 apply pattern length correlation with median value 10.
-g70,70,2  from the logically unstructured input generate pseudo records
             of constant length 70 (range 70..70) selecting 2% of data picked
	     regularly from the whole input, prior to evaluation.

.
.
 669 CCTGTAATCCCAACACTTTGGGAGGCCGAGGCTTGCGGATCATGAGG  TCAGGAGATCGAGACCATCCT
 904 GTAATCCCAGCCCTTTGGGAGGCCGAGGTGGGCAGATCACGAGGTCAGGAGA  TCGAGACCATCCTGGC
1022 AATCCCAGCACTTTGGGAGGCCGAGGTGGGCAGATCACCTGAGGTCAGGAGTTCGAGACC  ACCCTGGC
1022 CCCAGCACTTTGGGAGGCCGAGGTGGGTGGATCACCTGAGGTCAGGAATTC  GAGGCCAGCCTGGCCAA
1007 CAGCACTTTGGGAGGCCGAGGTGGGT  GGATCACAAGGTCAAGAGATCAAGACCATCCTGGCCAACATG
 852 CAGCACTTTGGGAAGCTGAGGTGGGCAGATCACAAGGTCAGGAGATGGAGACCATCC  TGGCCAACATG
 735 CACTTTGGG  AGGCTGAGGTGGGCAGATCATTTGAGGCCAGGAGTTTGAGACTAGCCTGGCCAACATGG
.
(Note: The gaps -2 blanks- are input line breaks, to be ignored).

As the differences between the lines above are very hard to recognize
visually the matching substrings are marked (manually) in the following 
paragraph:

 669 CCTGTAATCCCAACACTTTGGGAGGCCGAGGCTTGCGGATCATGAGGTCAGGAGATCGAGACCATCCT  
        111111111   2222222222222222            3333333333333333333333333
     111111111   2222222222222222            3333333333333333333333333
 904 GTAATCCCAGCCCTTTGGGAGGCCGAGGTGGGCAGATCACGAGGTCAGGAGATCGAGACCATCCTGGC  
       111111111 222222222222222222222222222233333333333 444444444 555555   
     111111111 2222222222222222222222222222  33333333333 444444444 555555
1022 AATCCCAGCACTTTGGGAGGCCGAGGTGGGCAGATCACCTGAGGTCAGGAGTTCGAGACCACCCTGGC  
        111111111111111111111111111  222222222222222222 333333 444 555555
     111111111111111111111111111  222222222222222222 333333 444 555555
1022 CCCAGCACTTTGGGAGGCCGAGGTGGGTGGATCACCTGAGGTCAGGAATTCGAGGCCAGCCTGGCCAA  
       111111111111111111111111111111111   222222       333 444 555555555
     111111111111111111111111111111111 222222 333       444 555555555
1007 CAGCACTTTGGGAGGCCGAGGTGGGTGGATCACAAGGTCAAGAGATCAAGACCATCCTGGCCAACATG  
     1111111111111    22222222  3333333333333 44444  55555555555555555555
     1111111111111    22222222  3333333333333 44444  55555555555555555555
 852 CAGCACTTTGGGAAGCTGAGGTGGGCAGATCACAAGGTCAGGAGATGGAGACCATCCTGGCCAACATG  
        1111111111 222222222222222222  333 444444   55555   6666666666666
     1111111111 222222222222222222    333 444444   55555   6666666666666
 735 CACTTTGGGAGGCTGAGGTGGGCAGATCATTTGAGGCCAGGAGTTTGAGACTAGCCTGGCCAACATGG  


In contrast to the closely matching lines above here are some
lines of less correspondence to each other:
 345 CACCACTGACTCACAGGATGGCAGGGCCAGCTCACTTCCCCTTCTGATGTTTTTGTTCCTTAGTGTAC 
        4444 aaa5555 22222  8888 111113333  bbb7777  6666   999           
      3333 4444  5555111116666 7777   999 22222 aaa8888 bbb               
 191 TACTTACACTAACACAAGCTCGTTTATCTGCAATTCAGATGGAACTGGGCACCTGCCTCTATCTGCTA 
     999    111111111 888 22222777  3333 6666aaa     5555   4444 bbb  ccc
      3333 111111111777   888   22222 999   aaa 4444 5555 bbb  6666 ccc   
 330 TAATTGACTAACACATCTCCCGCTTTGGTTTAATACAAAGGATCCTCAGCACGATCCTAGATCCTACA 

