Graph Transformation Example

Data in a table form can be transformed into numerous different formats using the class pyMez.Code.DataHandlers.GraphModels.TableGraph. The abstract data model is a table with column names.

Import needed modules

In [1]:
from pyMez.Code.DataHandlers.GraphModels import *
Importing pyMez, this should take roughly 30 seconds
Importing Code.DataHandlers.GeneralModels
It took 0.51203 s to import Code.DataHandlers.GeneralModels
Importing Code.DataHandlers.HTMLModels
It took 0.105006 s to import Code.DataHandlers.HTMLModels
Importing Code.DataHandlers.NISTModels
The module smithplot was not found,please put it on the python path
It took 0.785044 s to import Code.DataHandlers.NISTModels
Importing Code.DataHandlers.TouchstoneModels
It took 0.0 s to import Code.DataHandlers.TouchstoneModels
Importing Code.DataHandlers.XMLModels
It took 0.094006 s to import Code.DataHandlers.XMLModels
Importing Code.DataHandlers.ZipModels
It took 0.008 s to import Code.DataHandlers.ZipModels
Importing Code.InstrumentControl.Experiments
It took 2.62215 s to import Code.InstrumentControl.Experiments
Importing Code.InstrumentControl.Instruments
It took 0.0 s to import Code.InstrumentControl.Instruments
Importing Code.Utils.Names
It took 0.0 s to import Code.Utils.Names
It took 4.126236 s to import all of the active modules
C:\Users\sandersa\AppData\Local\Continuum\anaconda3\lib\site-packages\odo\backends\pandas.py:102: FutureWarning: pandas.tslib is deprecated and will be removed in a future version.
You can access NaTType as type(pandas.NaT)
  @convert.register((pd.Timestamp, pd.Timedelta), (pd.tslib.NaTType, type(None)))
In [2]:
# create the graph
graph=TableGraph()
In [3]:
# now we have access to the folowing data formats
graph.node_names
Out[3]:
['DataFrame',
 'AsciiDataTable',
 'HdfFile',
 'XmlDataTable',
 'ExcelFile',
 'OdsFile',
 'HtmlString',
 'JsonFile',
 'JsonString',
 'CsvFile',
 'MatFile',
 'XmlFile',
 'HtmlFile']
In [4]:
# to visualize the graph use the show method
graph.show()
Out[4]:
<Figure size 432x288 with 0 Axes>
<Figure size 432x288 with 0 Axes>
In [5]:
# if we want to open an ascii data table or subclass we open the schema and then open the file
schema=read_schema(os.path.join(os.getcwd(),"./TableGraph_Files/Line_4909_WR15_Wave_Parameters_20180313_003.schema"))
ascii_table=AsciiDataTable(os.path.join(os.getcwd(),"./TableGraph_Files/Line_4909_WR15_Wave_Parameters_20180313_003.w2p"),
                           **schema)
In [6]:
graph.set_state(node_name="AsciiDataTable",node_data=ascii_table)
In [7]:
# now if you want a pandas data frame just move to that node
graph.move_to_node("DataFrame")
In [8]:
#now dataframe is just
pandas_dataframe=graph.data
In [9]:
pandas_dataframe[0:10]
Out[9]:
Frequency reA1_D1 imA1_D1 reB1_D1 imB1_D1 reA2_D1 imA2_D1 reB2_D1 imB2_D1 reA1_D2 imA1_D2 reB1_D2 imB1_D2 reA2_D2 imA2_D2 reB2_D2 imB2_D2
0 50.0 -0.06771637116 -0.01816907875 -0.002698191406 0.0007358189392 0.002504944754 0.006581974472 0.004641698273 -0.02466258165 -0.002775838721 0.0007365005672 -0.009040250908 -0.01372005375 -0.0304376245 0.03904758232 -0.0006660961421 -0.001239688192
1 50.1 0.02243850646 0.06665811006 -0.002309633574 -0.001366194728 -0.007025498308 0.003140471545 0.003017988775 0.02523530152 0.002766433187 0.0008072947322 -0.01298566794 0.009946449155 -0.006614970446 0.04995893607 0.0005611848447 0.0007264105887
2 50.2 0.04527021965 -0.05256141552 -0.00173388365 -0.001927861015 -0.002513645387 -0.007837803592 -0.007693914526 -0.02422755665 -0.0004392237569 0.002661782361 0.002922748697 -0.01585399076 0.02744889107 -0.04267520686 0.000952701442 0.0002679299738
3 50.3 -0.068064164 -0.006680145887 -0.0005841158336 -0.002532520237 0.008350680994 -0.001583162367 0.01300682537 0.0216939389 -0.002352720107 -0.0004776609296 0.006113562282 0.01449721952 -0.04672627515 0.01879209235 0.0002190834271 0.001004918029
4 50.4 0.01993176055 0.06522905575 0.001363952887 -0.002947389348 -0.0007865336407 0.008783444812 -0.01435682532 -0.02113154479 0.001078082821 -0.001850086426 -0.0104808844 -0.01154704467 0.04611253162 0.02143472309 0.0002404384847 0.0006860393303
5 50.5 0.06120319388 -0.02933126305 0.002600965115 0.0002420115068 -0.00765577376 -0.004734756304 0.01142720969 0.02331471344 0.0009795936786 0.001590920773 0.01322155897 0.008355509256 -0.01542597473 -0.04949449405 -0.001176449841 0.0005619314235
6 50.6 -0.04549624588 -0.05036273752 0.002401206648 0.001194642981 0.00717132721 -0.005577364131 -0.01149386376 -0.02369062349 -0.001749184772 -0.0001509449615 -0.01486666319 -0.004184518099 -0.02583977377 0.04489485325 -2.531147057e-05 -0.0003908249213
7 50.7 -0.03622071059 0.05817229729 -3.623704135e-05 0.003032823443 0.002033359668 0.008625612246 0.009857377368 0.02480800621 0.001344404504 -0.001363835 0.01510341316 0.002440900883 0.05179807238 0.0003036752765 -0.0007968675442 -0.0007567387715
8 50.8 0.0630271502 0.02902860699 -0.001880705228 0.001355544502 -0.007952566598 -0.003059360583 -0.003294191447 -0.02718061392 0.0001941419538 0.00223786501 -0.01526013303 -0.001378560896 -0.02279080746 -0.04731256854 0.001281713799 -0.0005104444253
9 50.9 0.02000265252 -0.06661988479 -0.002146230544 -0.0004442927581 0.006057403486 -0.004937491383 -0.001895092701 0.02750249379 -0.002240929551 -0.00148763921 0.01513982274 -9.430191224e-05 -0.02918889338 0.04371010376 0.0007133279978 0.000487065628
In [10]:
# if we want to transform to json string
graph.move_to_node("JsonString")
In [11]:
json=graph.data
In [12]:
# if we would like to transform it to all possible data types -the xslx and ods require a confirmation
# first change the working directory and now all files will be put in the folder
os.chdir("./TableGraph_Files")
visit_all_nodes(graph)
C:\Users\sandersa\AppData\Local\Continuum\anaconda3\lib\site-packages\pyMez\Code\DataHandlers\Translations.py:572: FutureWarning: Method .as_matrix will be removed in a future version. Use .values instead.
  conversion_options["data"]=pandas_data_frame.as_matrix().tolist()[:]
data_nodes is [<DOM Element: Data at 0xd1b12a8>]
C:\Users\sandersa\AppData\Local\Continuum\anaconda3\lib\site-packages\pyMez\Code\DataHandlers\Translations.py:572: FutureWarning: Method .as_matrix will be removed in a future version. Use .values instead.
  conversion_options["data"]=pandas_data_frame.as_matrix().tolist()[:]
C:\Users\sandersa\AppData\Local\Continuum\anaconda3\lib\site-packages\scipy\io\matlab\miobase.py:414: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`. In the future this will be interpreted as an array index, `arr[np.array(seq)]`, which will result either in an error or a different result.
  arr[empties] = ' '
data_nodes is [<DOM Element: Data at 0xd61be88>]
data_nodes is [<DOM Element: Data at 0x805bb90>]