Difference between revisions of "SHORE v0.7 file formats"
From SHORE wiki
m (moved SHORE file formats to SHORE v0.7 file formats) |
|||
| (3 intermediate revisions by the same user not shown) | |||
| Line 3: | Line 3: | ||
Typing ''shore fmt'' will display a quick reference for many of SHORE’s file formats. | Typing ''shore fmt'' will display a quick reference for many of SHORE’s file formats. | ||
| − | This page only describes SHORE's read and alignment file formats; other files formats | + | This page only describes SHORE's read and alignment file formats; other files formats are described on the page of the respective [[subprogram]] that generates them. |
[[shore convert]] may be used to convert SHORE files into various third-party file formats, or vice versa. | [[shore convert]] may be used to convert SHORE files into various third-party file formats, or vice versa. | ||
| Line 16: | Line 16: | ||
{| | {| | ||
|----valign="top" | |----valign="top" | ||
| + | |1 | ||
| '''id''' | | '''id''' | ||
| A unique identifier for the read or read pair | | A unique identifier for the read or read pair | ||
|----valign="top" | |----valign="top" | ||
| + | |2 | ||
| '''sequence''' | | '''sequence''' | ||
| DNA sequence | | DNA sequence | ||
|----valign="top" | |----valign="top" | ||
| + | |3 | ||
| '''index''' | | '''index''' | ||
|''0'' for a single end read, ''n''≥''1'' for read ''n'' of a read pair or other multi part fragment. | |''0'' for a single end read, ''n''≥''1'' for read ''n'' of a read pair or other multi part fragment. | ||
|----valign="top" | |----valign="top" | ||
| + | |4 | ||
| '''Sanger quality values''' | | '''Sanger quality values''' | ||
| String of sanger calibrated quality values | | String of sanger calibrated quality values | ||
| Line 30: | Line 34: | ||
Encoding: ASCII 33 (''''!'''', quality 0) to ASCII 73 (''''I'''', quality 40); extended range ASCII 93 ('''']'''', quality 60) | Encoding: ASCII 33 (''''!'''', quality 0) to ASCII 73 (''''I'''', quality 40); extended range ASCII 93 ('''']'''', quality 60) | ||
|----valign="top" | |----valign="top" | ||
| + | |5 | ||
| ['''Chastity values'''] | | ['''Chastity values'''] | ||
(Optional column) | (Optional column) | ||
| Line 84: | Line 89: | ||
| Alignment start offset into the read (local alignments, first base of the read is ''1'') | | Alignment start offset into the read (local alignments, first base of the read is ''1'') | ||
* ''0'': no offset, the read/query is fully aligned (''soft clip'' is still possible) | * ''0'': no offset, the read/query is fully aligned (''soft clip'' is still possible) | ||
| − | * ≥''1'': The read/query is only locally aligned and the alignment starts at the specified position. Unaligned | + | * ≥''1'': The read/query is only locally aligned and the alignment starts at the specified position. Unaligned ends are not represented in the alignment string. |
|----valign="top" | |----valign="top" | ||
| '''pe flag''' | | '''pe flag''' | ||
Latest revision as of 18:54, 21 February 2013
Any output generated by SHORE will usually be written to various text files that contain a number of tab-delimited columns.
Typing shore fmt will display a quick reference for many of SHORE’s file formats.
This page only describes SHORE's read and alignment file formats; other files formats are described on the page of the respective subprogram that generates them.
shore convert may be used to convert SHORE files into various third-party file formats, or vice versa.
Read file format
Read files can be found in the LengthFolders or ReadFolders. These files are created by shore import and are named reads_0.fl.
reads_0.fl files are usually sorted on the id field in numerical order.
The tab delimited entries are:
| 1 | id | A unique identifier for the read or read pair |
| 2 | sequence | DNA sequence |
| 3 | index | 0 for a single end read, n≥1 for read n of a read pair or other multi part fragment. |
| 4 | Sanger quality values | String of sanger calibrated quality values
Encoding: ASCII 33 ('!', quality 0) to ASCII 73 ('I', quality 40); extended range ASCII 93 (']', quality 60) |
| 5 | [Chastity values]
(Optional column) |
String of illumina chastity values (defined as <math>Intensity(max)/ (Intensity(max) + Intensity(second))</math>).
Encoding: ASCII 40 ('(', chastity of 0.5) to ASCII 90 ('Z', chastity of 1.0) |
Note: Earlier versions of SHORE used a read file format featuring three different quality types per entry. This file format is still supported for reading, but is no longer written.
Alignment file format
SHORE alignment files are typically named map.list, map.list.1 or map.list.2. They are stored in the LengthFolders or ReadFolders.
map.list files are sorted in numerical order, either on the fields chr id and pos, or on the field read id.
The tab delimited entries are:
| chr id | Each chromosome has an internal id, simply enumerated according to their occurrence within the reference sequence file, starting from 1. Translation to the native chromosome name can be found in the *.shore.ref and *.shore.trans file in the IndexFolder, or in the ref.txt files created by shore mapflowcell. |
| pos | Left-most position of the alignment relative to the forward strand of the reference sequence. The first position of a chromosome is 1. |
| alignment | String representation of the read alignment. The sequence is always reported with respect to the forward strand of the reference, i.e. the sequence of reads matching to the reverse strand is reverse complemented.
Extensions, not supported by all tools:
|
| read id | A unique identifier for the read or read pair |
| strand | D for forward and P for reverse hits (direct and palindromic, respectively). |
| mismatches | The number of mismatches+gaps in the alignment. |
| hits | The total number of genomic positions the read is aligned to. |
| read length | Length of the read ('soft clipped' nucleotides excluded). |
| offset | Alignment start offset into the read (local alignments, first base of the read is 1)
|
| pe flag | Paired-end information
A library ID <math>L</math> may be encoded in the pe flag for the flags with value <math>>2</math>; in this case the flag is calculated as <math>pe\_flag = pe\_flag + (L * 6)</math> |
| Sanger quality values | String of sanger calibrated quality values.
Encoding: ASCII 33 ('!', quality 0) to ASCII 73 ('I', quality 40); extended range ASCII 93 (']', quality 60) |
| [Chastity values]
(Optional column) |
String of illumina chastity values defined as the highest intensity divided by the sum of the highest and the second highest intensity of a single base.
Encoding: ASCII 40 ('(', chastity of 0.5) to ASCII 90 ('Z', chastity of 1.0) |
Note: Earlier versions of SHORE used an alignment file format featuring three different quality types per entry. This file format is still supported for reading, but is no longer written.
