AsciiDataTable Example

The AsciiDataTable class is meant to deal with data in the form of a table with a header and footer. It has the ability to change formating, load a file, and do simple manipulations.

Some key elements of the AsciiDataTable class:

  1. It can be created by opening a file with the right options. To preserve the options of a given file use the save_schema method and the read_schema function.
  2. It can save, or print.
  3. It has a large amount of formatting flexability

To make a new data table:

new_table=AsciiDataTable()
#or
new_table=AsciiDataTable(header=["My Header","Line2" ],column_names=["a","b","c"],data=[[1,2,3]])

To open an existing table:

#in default format (csv with no #)
existing_table=AsciiDataTable("My_Table.txt")
# for an existing format with a saved schema
schema=read_schema("My_Schema")
existing_table=AsciiDataTable("My_Table.txt",**schema)
In [1]:
# As an example
import pyMez.Code.DataHandlers.GeneralModels as GeneralModels
Importing pyMez, this should take roughly 30 seconds
The module smithplot was not found,please put it on the python path
The module smithplot was not found,please put it on the python path
In [2]:
#Lets make a table and put it into the AsciiDataTable 
new_table=GeneralModels.AsciiDataTable()
In [3]:
# The data is a list of lists with the first dimension # of rows and the second dimension # of columns
table_data=[[i+j*3 for i in range(3)]for j in range(3)]
print table_data
[[0, 1, 2], [3, 4, 5], [6, 7, 8]]
In [4]:
#input the data directly
new_table.data=table_data
In [5]:
# assign the column names
new_table.column_names=['x1','x2','x3']
In [6]:
# when a string related command is called it uses build string to output the string print(new_table) also works
print new_table.build_string()
x1,x2,x3
0,1,2
3,4,5
6,7,8
In [7]:
# the attribute options stores the schema for the data
new_table.options['column_names_begin_token']='#'
In [8]:
print new_table.build_string()
#x1,x2,x3
0,1,2
3,4,5
6,7,8
In [9]:
# changing a formating option doesn't change the data
new_table.options['column_names_begin_token']='['
new_table.options['column_names_end_token']=']'
In [10]:
print new_table.build_string()
[x1,x2,x3]
0,1,2
3,4,5
6,7,8
In [11]:
# There is the ability to control the output format in many ways
new_table.options['data_begin_token']='*'*80+'\n'
new_table.options['data_end_token']="\n"+'*'*80
In [12]:
print new_table.build_string()
[x1,x2,x3]
********************************************************************************
0,1,2
3,4,5
6,7,8
********************************************************************************
In [13]:
# the header is where we would store metadata
new_table.header=['My First Header Line','My Second Header Line']
In [14]:
print new_table
My First Header Line
My Second Header Line
[x1,x2,x3]
********************************************************************************
0,1,2
3,4,5
6,7,8
********************************************************************************
In [15]:
new_table.options['comment_begin']='#'
new_table.options['comment_end']='\n'
In [16]:
print new_table
#My First Header Line
#My Second Header Line

[x1,x2,x3]
********************************************************************************
0,1,2
3,4,5
6,7,8
********************************************************************************
In [17]:
# get_options returns all options printed in a nice form 
new_table.get_options()
inline_comments = None
data_begin_line = 4
column_names_begin_line = 3
footer_line_types = None
general_descriptor = Table
header_end_token = None
header_end_line = 2
column_names_end_line = 4
data_table_element_separator = 

escape_character = None
data = None
inline_comment_begin = None
column_descriptions = None
inline_comment_end = None
data_begin_token = ********************************************************************************

comment_end = 

header_begin_line = 0
row_end_token = None
column_names_end_token = ]
block_comment_begin = None
header_begin_token = None
metadata = None
specific_descriptor = Data
empty_value = None
column_names_begin_token = [
footer_end_token = None
footer_begin_token = None
column_names = None
metadata_key_value_delimiter = None
column_types = None
comment_begin = #
treat_header_as_comment = None
column_units = None
column_names_delimiter = ,
treat_footer_as_comment = None
extension = txt
header = None
footer = None
data_end_line = None
block_comment_end = None
footer_end_line = None
data_delimiter = ,
footer_begin_line = None
data_end_token = 
********************************************************************************
row_begin_token = None
directory = None
header_line_types = None
row_formatter_string = None
metadata_delimiter = None
In [18]:
# when we created the file it autonamed itself. This is the place the file will save to using new_table.save()
new_table.path
Out[18]:
'Data_Table_20170429_001.txt'
In [19]:
# Each formatting option can be turned off
new_table.options['data_begin_token']=None
new_table.options['data_end_token']=None
In [20]:
print new_table
#My First Header Line
#My Second Header Line

[x1,x2,x3]
0,1,2
3,4,5
6,7,8
In [21]:
# We can build the table this way or using 
# AsciiDataTable(header=new_table.header,column_names=new_table.column_names)
new_table2=GeneralModels.AsciiDataTable()
new_table2.header=new_table.header
new_table2.column_names=new_table.column_names
new_table2.options=new_table.options
In [22]:
print new_table2
#My First Header Line
#My Second Header Line

[x1,x2,x3]
In [23]:
new_table2.data=[[2*(i+j*3) for i in range(3)]for j in range(3)]
In [24]:
print new_table2
#My First Header Line
#My Second Header Line

[x1,x2,x3]
0,2,4
6,8,10
12,14,16
In [25]:
# Adding the tables, if column names are the same
new_table+new_table2
Out[25]:
<Code.DataHandlers.GeneralModels.AsciiDataTable instance at 0x00000000112179C8>
In [26]:
print new_table
print new_table2
#My First Header Line
#My Second Header Line

[x1,x2,x3]
0,1,2
3,4,5
6,7,8
0,2,4
6,8,10
12,14,16
#My First Header Line
#My Second Header Line

[x1,x2,x3]
0,2,4
6,8,10
12,14,16
In [27]:
print new_table
#My First Header Line
#My Second Header Line

[x1,x2,x3]
0,1,2
3,4,5
6,7,8
0,2,4
6,8,10
12,14,16
In [28]:
print new_table2.build_string()
#My First Header Line
#My Second Header Line

[x1,x2,x3]
0,2,4
6,8,10
12,14,16
In [29]:
str(new_table2).count("\n")
Out[29]:
6
In [30]:
test_string=str(new_table2)
In [31]:
test_string
Out[31]:
'#My First Header Line\n#My Second Header Line\n\n[x1,x2,x3]\n0,2,4\n6,8,10\n12,14,16'
In [32]:
test_list=test_string.split('\n')
In [33]:
test_list
Out[33]:
['#My First Header Line',
 '#My Second Header Line',
 '',
 '[x1,x2,x3]',
 '0,2,4',
 '6,8,10',
 '12,14,16']
In [34]:
print new_table
#My First Header Line
#My Second Header Line

[x1,x2,x3]
0,1,2
3,4,5
6,7,8
0,2,4
6,8,10
12,14,16
In [35]:
new_table.options['inline_comment_begin']='(*'
new_table.options['inline_comment_end']='*)'
new_table.inline_comments=[["Hello, This  is an inline comment",0,3]]
print new_table
#My(*Hello, This  is an inline comment*) First Header Line
#My Second Header Line

[x1,x2,x3]
0,1,2
3,4,5
6,7,8
0,2,4
6,8,10
12,14,16
In [36]:
import random
In [37]:
data_delimiters=[",",".",":",";","-"," ","\t","::","~"," | "]
begin_end_pairs=[["<",">"],["(*","*)"],["[","]"],["/*","*/"],["(",")"],["<","/>"],["#",""]]
random_int=random.randint(0,len(data_delimiters)-1)
random_list=[random.randint(0,len(data_delimiters)-1) for i in range(20)]
print random_list
[0, 7, 6, 8, 0, 3, 4, 1, 3, 9, 5, 8, 3, 5, 8, 1, 4, 6, 6, 8]
In [38]:
random_delimiter_list_1=[data_delimiters[random.randint(0,len(data_delimiters)-1)] for i in range(20)]
random_delimiter_list_2=[data_delimiters[random.randint(0,len(data_delimiters)-1)] for i in range(20)]
random_pairs_1=[begin_end_pairs[random.randint(0,6)] for i in range(20)]
random_pairs_2=[begin_end_pairs[random.randint(0,6)] for i in range(20)]
In [39]:
print begin_end_pairs
new_table.inline_comments=[["Hello, This  is an inline comment",0,-1]]
[['<', '>'], ['(*', '*)'], ['[', ']'], ['/*', '*/'], ['(', ')'], ['<', '/>'], ['#', '']]
In [40]:
# An example of all the formats randomized
for index in range(len(random_delimiter_list_1)):
    new_table.options["data_delimiter"]=random_delimiter_list_1[index]
    new_table.options["column_names_delimiter"]=random_delimiter_list_2[index]
    new_table.options["comment_begin"]=random_pairs_1[index][0]
    new_table.options["comment_end"]=random_pairs_1[index][1]+"\n"
    new_table.options["column_names_begin_token"]=random_pairs_2[index][0]
    new_table.options["column_names_end_token"]=random_pairs_2[index][1]    
    print "\n{0}\n".format(new_table)
(*My First Header Line*)(*Hello, This  is an inline comment*)
(*My Second Header Line*)

/*x1:x2:x3*/
0 1 2
3 4 5
6 7 8
0 2 4
6 8 10
12 14 16


[My First Header Line](*Hello, This  is an inline comment*)
[My Second Header Line]

/*x1.x2.x3*/
0-1-2
3-4-5
6-7-8
0-2-4
6-8-10
12-14-16


/*My First Header Line*/(*Hello, This  is an inline comment*)
/*My Second Header Line*/

(*x1;x2;x3*)
0::1::2
3::4::5
6::7::8
0::2::4
6::8::10
12::14::16


[My First Header Line](*Hello, This  is an inline comment*)
[My Second Header Line]

(x1::x2::x3)
0	1	2
3	4	5
6	7	8
0	2	4
6	8	10
12	14	16


<My First Header Line>(*Hello, This  is an inline comment*)
<My Second Header Line>

<x1::x2::x3/>
0,1,2
3,4,5
6,7,8
0,2,4
6,8,10
12,14,16


[My First Header Line](*Hello, This  is an inline comment*)
[My Second Header Line]

<x1 | x2 | x3>
0.1.2
3.4.5
6.7.8
0.2.4
6.8.10
12.14.16


/*My First Header Line*/(*Hello, This  is an inline comment*)
/*My Second Header Line*/

[x1:x2:x3]
0:1:2
3:4:5
6:7:8
0:2:4
6:8:10
12:14:16


[My First Header Line](*Hello, This  is an inline comment*)
[My Second Header Line]

<x1.x2.x3>
0	1	2
3	4	5
6	7	8
0	2	4
6	8	10
12	14	16


/*My First Header Line*/(*Hello, This  is an inline comment*)
/*My Second Header Line*/

(x1.x2.x3)
0::1::2
3::4::5
6::7::8
0::2::4
6::8::10
12::14::16


#My First Header Line(*Hello, This  is an inline comment*)
#My Second Header Line

<x1,x2,x3/>
0 1 2
3 4 5
6 7 8
0 2 4
6 8 10
12 14 16


(*My First Header Line*)(*Hello, This  is an inline comment*)
(*My Second Header Line*)

#x1.x2.x3
0.1.2
3.4.5
6.7.8
0.2.4
6.8.10
12.14.16


(My First Header Line)(*Hello, This  is an inline comment*)
(My Second Header Line)

[x1:x2:x3]
0:1:2
3:4:5
6:7:8
0:2:4
6:8:10
12:14:16


/*My First Header Line*/(*Hello, This  is an inline comment*)
/*My Second Header Line*/

[x1~x2~x3]
0 1 2
3 4 5
6 7 8
0 2 4
6 8 10
12 14 16


<My First Header Line>(*Hello, This  is an inline comment*)
<My Second Header Line>

<x1 x2 x3/>
0::1::2
3::4::5
6::7::8
0::2::4
6::8::10
12::14::16


<My First Header Line>(*Hello, This  is an inline comment*)
<My Second Header Line>

(x1 x2 x3)
0.1.2
3.4.5
6.7.8
0.2.4
6.8.10
12.14.16


/*My First Header Line*/(*Hello, This  is an inline comment*)
/*My Second Header Line*/

<x1 | x2 | x3/>
0~1~2
3~4~5
6~7~8
0~2~4
6~8~10
12~14~16


#My First Header Line(*Hello, This  is an inline comment*)
#My Second Header Line

(*x1 x2 x3*)
0~1~2
3~4~5
6~7~8
0~2~4
6~8~10
12~14~16


[My First Header Line](*Hello, This  is an inline comment*)
[My Second Header Line]

[x1~x2~x3]
0.1.2
3.4.5
6.7.8
0.2.4
6.8.10
12.14.16


(My First Header Line)(*Hello, This  is an inline comment*)
(My Second Header Line)

(*x1;x2;x3*)
0 | 1 | 2
3 | 4 | 5
6 | 7 | 8
0 | 2 | 4
6 | 8 | 10
12 | 14 | 16


(My First Header Line)(*Hello, This  is an inline comment*)
(My Second Header Line)

(x1-x2-x3)
0-1-2
3-4-5
6-7-8
0-2-4
6-8-10
12-14-16

In [41]:
GeneralModels.show_structure_script()
Printing the string representation of the table
--------------------------------------------------------------------------------
{header_begin_token}
self.header[0]
self.header[1]{inline_comment_begin}inline_comments[0][0]{inline_comment_end}
{comment_begin}self.header[2]{comment_end}

{block_comment_begin}
self.header[3]
self.header{4]
{block_comment_end}
{header_end_token}
{data_table_element_separator}
{column_names_begin_token}column_names[0]{column_names_delimiter}column_names[1]{column_names_delimiter}column_names[2]{column_names_end_token}
{data_table_element_separator}
{data_begin_token}
{row_begin_token}data[0][0]{data_delimiter}data[1][0]{data_delimiter}data[2][0]{row_end_token}
{row_begin_token}data[0][1]{data_delimiter}data[1][1]{data_delimiter}data[2][1]{row_end_token}
{data_end_token}
{data_table_element_separator}
{footer_begin_token}
{block_comment_begin}
self.footer[0]
self.footer[1]
{block_comment_end}
{footer_end_token}
Printing the lines representation of the table with line numbers
--------------------------------------------------------------------------------
0 {header_begin_token}
1 self.header[0]
2 self.header[1]{inline_comment_begin}inline_comments[0][0]{inline_comment_end}
3 {comment_begin}self.header[2]{comment_end}
4 
5 {block_comment_begin}
6 self.header[3]
7 self.header{4]
8 {block_comment_end}
9 {header_end_token}
10 {data_table_element_separator}
11 {column_names_begin_token}column_names[0]{column_names_delimiter}column_names[1]{column_names_delimiter}column_names[2]{column_names_end_token}
12 {data_table_element_separator}
13 {data_begin_token}
14 {row_begin_token}data[0][0]{data_delimiter}data[1][0]{data_delimiter}data[2][0]{row_end_token}
15 {row_begin_token}data[0][1]{data_delimiter}data[1][1]{data_delimiter}data[2][1]{row_end_token}
16 {data_end_token}
17 {data_table_element_separator}
18 {footer_begin_token}
19 {block_comment_begin}
20 self.footer[0]
21 self.footer[1]
22 {block_comment_end}
23 {footer_end_token}
--------------------------------------------------------------------------------
The result of self.lines[0:10] is :
{header_begin_token}
self.header[0]
self.header[1]{inline_comment_begin}inline_comments[0][0]{inline_comment_end}
{comment_begin}self.header[2]{comment_end}

{block_comment_begin}
self.header[3]
self.header{4]
{block_comment_end}
{header_end_token}
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
The result of self.lines[11:12] is :
{column_names_begin_token}column_names[0]{column_names_delimiter}column_names[1]{column_names_delimiter}column_names[2]{column_names_end_token}
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
The result of self.lines[13:17] is :
{data_begin_token}
{row_begin_token}data[0][0]{data_delimiter}data[1][0]{data_delimiter}data[2][0]{row_end_token}
{row_begin_token}data[0][1]{data_delimiter}data[1][1]{data_delimiter}data[2][1]{row_end_token}
{data_end_token}
--------------------------------------------------------------------------------
--------------------------------------------------------------------------------
The result of self.lines[18:None] is :
{footer_begin_token}
{block_comment_begin}
self.footer[0]
self.footer[1]
{block_comment_end}
{footer_end_token}
--------------------------------------------------------------------------------