Schevo Database Usage Reference¶
Overview¶
A Schevo database is the combination of a schema definition, stored data, and an engine that exposes the two. This document describes in detail the public API exposed by the Schevo engine.
Schema definitions are pure-Python modules that make use of Schevo’s syntax extensions. Schema definitions can be directly imported anywhere, but typically the Schevo database engine manages the import process in order to tie a schema to an open database.
Stored data is managed by the SchevoDurus package, based on Durus. The data is kept in a general structure used by all Schevo databases, described in the internal database structures reference.
The database engine is defined in the schevo.database module, and provides read/write access to stored data using the higher-level structures defined by the database’s schema definition.
Doctest Setup¶
This document also functions as a doctest:
.. sourcecode:: pycon
>>> from schevo.test import DocTest
sys Namespaces¶
sys
(short for system) is a namespace that contains the public
methods and properties of a Schevo object that are not fields, query
methods, transaction methods, or view methods. These are kept in a
separate namespace in order to reduce the amount of reserved words.
sys
namespaces are accessed by getting the sys
attribute from
a Schevo object.
The following types of Schevo objects have a sys
namespace:
- Entity instances
- Find query instances
- Param query instances
- Transaction instances
- View instances
Database instances¶
Database labels¶
By default, a database instance is labeled “Schevo Database”:
.. sourcecode:: pycon
>>> from schevo.label import label, relabel
>>> t = DocTest("""
... class Foo(E.Entity):
... pass
... """)
>>> print label(t.db)
Schevo Database
Override the label by using relabel:
.. sourcecode:: pycon
>>> relabel(t.db, 'My Database')
>>> print label(t.db)
My Database
Extent instances¶
Relaxing and enforcing key restrictions¶
Schevo has the unique ability to temporarily relax key restrictions while still ensuring that the database remains in a consistent state.
Key restrictions may only be relaxed within the execution of a transaction.
Given an extent db.Foo that has a key specification of _key(field1, field2), relax that key restriction by calling relax_index:
.. sourcecode:: pycon
db.Foo.relax_index('field1', 'field2')
Key restrictions that are relaxed within a transaction are relaxed for the entire execution of the transaction, or until the key is enforced within that transaction, or until that transaction ends, whichever comes first.
When a key restriction is relaxed by a transaction, it remains relaxed for all sub-transactions, regardless of any request by those sub-transactions to enforce the key restriction.
To re-enforce the key restriction before the end of a transaction’s execution, call enforce_index:
.. sourcecode:: pycon
db.Foo.enforce_index('field1', 'field2')
See the tests/test_relax_index.py
test case for code examples.
Extent labels¶
By default, an extent has a singular and plural label based on the entity class name associated with the extent:
.. sourcecode:: pycon
>>> from schevo.label import label, plural
>>> t = DocTest("""
... class GreatPerson(E.Entity):
...
... name = f.string()
... """)
>>> print label(t.db.GreatPerson)
Great Person
>>> print plural(t.db.GreatPerson)
Great Persons
To override the singular and/or plural labels of an extent, assign values to the _label and/or _plural attributes of the associated entity class:
.. sourcecode:: pycon
>>> from schevo.label import label, plural
>>> t = DocTest("""
... class GreatPerson(E.Entity):
...
... name = f.string()
...
... _label = 'Great PERSON'
... _plural = 'Great PEOPLE'
... """)
>>> print label(t.db.GreatPerson)
Great PERSON
>>> print plural(t.db.GreatPerson)
Great PEOPLE
Entity instances¶
Entity sys namespace¶
links: Return entities that link to this entity¶
Use links
to find other entities that point to the calling entity.
To find all entities that link to the calling entity, use the form
links()
. The return value will be a dictionary of keys in the
form (other_extent_name, other_field_name)
, each mapping to a list
of Entity instances in that extent where that field’s value is the
calling entity:
# Doctest to go here.
To find all entities in a specific extent that link to the calling
entity, use the form links(other_extent_name)
. The return value
will be the same as that returned by a call to links()
:
# Doctest to go here.
To find all the entities in a specific extent where a specific field
value links to the calling entity, use the form
links(other_extent_name, other_field_name)
. The return value will
be a list of Entity instances in that extent where that field’s value
is the calling entity:
# Doctest to go here.
Entity labels¶
The label of an entity instance is the same as the unicode representation of the entity instance.
By default, the label is determined by the default key defined in the entity class:
.. sourcecode:: pycon
>>> from schevo.label import label
>>> t = DocTest("""
... class State(E.Entity):
...
... name = f.string()
... abbreviation = f.string()
...
... _key(name)
... _key(abbreviation)
...
... _initial = [
... ('Missouri', 'MO'),
... ]
... """)
>>> missouri = t.db.State.findone(name='Missouri')
>>> print label(missouri)
Missouri
If the key has more than one field, the values are separated by two colons:
.. sourcecode:: pycon
>>> t = DocTest("""
... class City(E.Entity):
...
... name = f.string()
... state = f.entity('State')
...
... _key(name, state)
...
... _initial = [
... ('Monett', ('Missouri',)),
... ]
...
... class State(E.Entity):
...
... name = f.string()
... abbreviation = f.string()
...
... _key(name)
... _key(abbreviation)
...
... _initial = [
... ('Missouri', 'MO'),
... ]
... """)
>>> missouri = t.db.State.findone(name='Missouri')
>>> monett = t.db.City.findone(name='Monett', state=missouri)
>>> print label(monett)
Monett :: Missouri
Override the __unicode__ method of an entity class to customize the label:
.. sourcecode:: pycon
>>> t = DocTest("""
... class City(E.Entity):
...
... name = f.string()
... state = f.entity('State')
...
... _key(name, state)
...
... def __unicode__(self):
... return u'%s, %s' % (self.name, self.state.abbreviation)
...
... _initial = [
... ('Monett', ('Missouri',)),
... ]
...
... class State(E.Entity):
...
... name = f.string()
... abbreviation = f.string()
...
... _key(name)
... _key(abbreviation)
...
... _initial = [
... ('Missouri', 'MO'),
... ]
... """)
>>> missouri = t.db.State.findone(name='Missouri')
>>> monett = t.db.City.findone(name='Monett', state=missouri)
>>> print label(monett)
Monett, MO
Entity placeholders¶
schevo.placeholder.Placeholder instances store information about an entity in a database. Schevo databases keep Placeholder instances internally, and work with them instead of actual Entity instances since the latter cannot be directly persisted.
Do not worry about using Placeholder instances directly, but be aware of their existence. At times, you may run across an Exception whose message includes the repr of a Placeholder instance, such as a schevo.error.KeyCollision error.
Field classes¶
Field instances¶
Query methods and objects¶
At their highest level, schevo.query.Query objects use criteria you specify to create sequences of results.
There are several types of queries, and some queries’ criteria consist of subqueries, from which it obtains results during its execution.
For the purposes of example, let us assume that we are working with the following schema for this section:
>>> t = DocTest("""
... from schevo.schema import *
... schevo.schema.prep(locals())
...
... class Person(E.Entity):
...
... name = f.string()
... birthdate = f.date()
...
... _key(name)
...
... _plural = u'People'
...
... def __unicode__(self):
... return u'%s, born on %s' % (self.name, self.birthdate)
...
... _sample_unittest = [
... ('Robert Jones', '1965-02-03'),
... ('Roberto Gonzales', '1976-04-05'),
... ('Roberta Smith', '1987-06-07'),
... ('Rance Thomas', '1954-12-01'),
... ]
...
... class Location(E.Entity):
...
... name = f.string()
...
... _key(name)
...
... _sample_unittest = [
... ('Bank',),
... ('Grocery Store',),
... ('Library',),
... ]
...
... class Sighting(E.Entity):
...
... person = f.entity('Person')
... location = f.entity('Location')
... when = f.date()
...
... def __unicode__(self):
... return u'%s at %s on %s' % (
... self.person.name,
... self.location,
... self.when,
... )
...
... _sample_unittest = [
... (('Robert Jones',), ('Library',), '2004-01-01'),
... (('Roberto Gonzales',), ('Library',), '2004-01-04'),
... (('Robert Jones',), ('Grocery Store',), '2004-01-02'),
... (('Roberto Gonzales',), ('Bank',), '2004-01-05'),
... (('Robert Jones',), ('Bank',), '2004-01-03'),
... (('Roberto Gonzales',), ('Grocery Store',), '2004-01-06'),
... (('Roberta Smith',), ('Grocery Store',), '2004-01-07'),
... (('Rance Thomas',), ('Grocery Store',), '2004-01-08'),
... ]
... """)
>>> db = t.db
Both schema modules and database engines expose all available Query
classes in its Q
namespace. For brevity, we will use an alias:
>>> Q = db.Q
Match queries¶
The simplest query is a match on a field. Use the
schevo.query.Match
class:
>>> person_name = Q.Match(db.Person, 'name', 'startswith')
To set the value of the match query, set the query’s value:
>>> person_name.value = u'Robert'
If you already have a search in mind, you can specify it in the query constructor:
>>> person_name = Q.Match(db.Person, 'name', 'startswith', u'Robert')
It is worth noting that at any point, you can get a human language version of the query by converting it to a string:
>>> print person_name
People where Name starts with Robert
Retrieving results¶
To retrieve the results of a query, call the Query instance to get an
iterator over the results. For example, if we are using the
person_name
query defined above:
>>> results = person_name()
>>> for person in results:
... print person
Robert Jones, born on 1965-02-03
Roberto Gonzales, born on 1976-04-05
Roberta Smith, born on 1987-06-07
Query results, at the minimum, must support iteration:
>>> results = person_name()
>>> i = iter(results)
>>> i
<generator object ...>
To cache the query results, use whatever tools you like. For example, you can get a list of results in order to retrieve the length or perform slices:
>>> results = list(person_name())
>>> len(results)
3
>>> r0 = results[0]
>>> r0
<Person entity oid:1 rev:0>
>>> results[1:3]
[<Person entity oid:2 rev:0>, <Person entity oid:3 rev:0>]
>>> results.index(r0)
0
Compound queries¶
Continuing the example, let us also search for persons who have a birthdate before 1980-01-01, and who were most recently sighted at the grocery store.
Create the birthdate query:
>>> person_birthdate = Q.Match(db.Person, 'birthdate', '<', '1980-01-01')
>>> print person_birthdate
People where Birthdate < 1980-01-01
>>> for person in sorted(person_birthdate()):
... print person
Rance Thomas, born on 1954-12-01
Robert Jones, born on 1965-02-03
Roberto Gonzales, born on 1976-04-05
Combine the results of these queries using an intersection, so that we get the Person entities that match both queries:
>>> persons = Q.Intersection(person_name, person_birthdate)
>>> print persons
the intersection of (People where Name starts with Robert,
People where Birthdate < 1980-01-01)
>>> for person in sorted(persons()):
... print person
Robert Jones, born on 1965-02-03
Roberto Gonzales, born on 1976-04-05
Get all the sightings for each person:
>>> person_sightings = Q.Match(db.Sighting, 'person', 'in', persons)
>>> print person_sightings
Sightings where Person in the intersection of (People where Name
starts with Robert, People where Birthdate < 1980-01-01)
>>> for sighting in sorted(person_sightings()):
... print sighting
Robert Jones at Library on 2004-01-01
Roberto Gonzales at Library on 2004-01-04
Robert Jones at Grocery Store on 2004-01-02
Roberto Gonzales at Bank on 2004-01-05
Robert Jones at Bank on 2004-01-03
Roberto Gonzales at Grocery Store on 2004-01-06
Group the sightings by person. Since we are grouping the results of a query, which may be heterogeneous, we also specify a field class:
>>> by_person = Q.Group(person_sightings, 'person', db.Sighting.f.person)
>>> print by_person
Sightings where Person in the intersection of (People where Name
starts with Robert, People where Birthdate < 1980-01-01), grouped
by Person
>>> for group in sorted(by_person()):
... print '----'
... for result in group:
... print result
----
Robert Jones at Library on 2004-01-01
Robert Jones at Grocery Store on 2004-01-02
Robert Jones at Bank on 2004-01-03
----
Roberto Gonzales at Library on 2004-01-04
Roberto Gonzales at Bank on 2004-01-05
Roberto Gonzales at Grocery Store on 2004-01-06
The results of a Group query is a list of results. Reduce each group
to the result that has the maximum value for the when
field.
Again, we specify a field class since the query source may be
heterogeneous:
>>> most_recent = Q.Max(by_person, 'when', db.Sighting.f.when)
>>> print most_recent
results that have the maximum When in each (Sightings where Person
in the intersection of (People where Name starts with Robert, People
where Birthdate < 1980-01-01), grouped by Person)
>>> for sighting in sorted(most_recent()):
... print sighting
Robert Jones at Bank on 2004-01-03
Roberto Gonzales at Grocery Store on 2004-01-06
Finally, filter most_recent
by the grocery store Location:
>>> grocery_store = db.Location.findone(name=u'Grocery Store')
>>> last_seen_shopping = Q.Match(
... most_recent, 'location', '==', grocery_store, db.Sighting.f.location)
>>> print last_seen_shopping
results that have the maximum When in each (Sightings where Person
in the intersection of (People where Name starts with Robert, People
where Birthdate < 1980-01-01), grouped by Person) where Location ==
Grocery Store
>>> for sighting in sorted(last_seen_shopping()):
... print sighting
Roberto Gonzales at Grocery Store on 2004-01-06
Default queries¶
The default query for an extent is an Intersection of Match queries
against all non-calculated fields of an extent, with a default
operator of any
(which means “is any value”), and a default value
of UNASSIGNED
(which is ignored when using the any
operator).
This query is the same as would be returned when calling
db.Sighting.q.default()
:
>>> query = Q.Intersection(
... Q.Match(db.Sighting, 'person', 'any'),
... Q.Match(db.Sighting, 'location', 'any'),
... Q.Match(db.Sighting, 'when', 'any'),
... )
>>> print query
all Sightings
Without changing any of the operators, iterating over the results of the default query for an extent yields the same thing as iterating over the extent itself.
The Intersection query optimizes its execution plan to use the
extent’s find
method when all of the Match
queries match on
the same extent, when there is only one of any given field name, when
the only operators used are either any
or ==
, and when all
values are not queries.
As seen above, it also optimizes its label for brevity if all values
have the any
operator.
Field Containers¶
The following types of objects are field containers:
- Entity instances
- Param query instances
- Transaction instances
- View instances
A field container obj
implements the following:
They have a namespace,
obj.f
, that contains field objects.The value of
obj.f.field_name
is a Field instance associated withobj
, with the namefield_name
.The value of
obj.f['field_name']
is the same asobj.f.field_name
.Iterating over
obj.f
yields the names of the fields, in the order they were defined.If allowed by the field container, you can remove a field named
field_name
fromobj
using the following:.. sourcecode:: pycon del obj.f.field_name
If allowed by the field container, you can add a new field named
field_name
and of typeFieldClass
toobj
using either of the following:.. sourcecode:: pycon obj.f.field_name = existing_field_instance obj.f.field_name = f.field_class(...)
The field is added to the end of the definition order. To re-order fields, you can get them from
obj.f
, delete them fromobj.f
, then reassign them toobj.f
, as in this example:.. sourcecode:: pycon # Assume original declaration order was: # foo = f.string() # bar = f.string() # baz = f.string() bar, baz = obj.f.bar, obj.f.baz del obj.f.bar, obj.f.baz obj.f.baz = baz obj.f.bar = bar # Now the order is the following: # foo = f.string() # baz = f.string() # bar = f.string()
They have a method,
obj.s.fields()
, that returns a schevo.fieldspec.FieldMap ordered dictionary for the fields inobj
.- Certain types of field decorators accept additional arguments to
the call to
obj.s.fields()
.
- Certain types of field decorators accept additional arguments to
the call to
Iterators¶
Many objects in a Schevo database provide iterators.
Extents¶
Iterating over an extent results in a sequence of all entities in the extent, ordered by OID.
Query, view, and transaction namespaces¶
Iterating over a f
, q
, t
, or v
namespace results in a
sequence of the names in that namespace, in no defined order.
Example:
>>> sorted(db.Person.q)
['by_example', 'exact']
>>> sorted(db.Person[1].t)
['clone', 'delete', 'update']
Entity sys namespaces¶
Iterating over an entity’s sys
namespace has no defined result.
It may become part of the Objects without iterators.
Objects without iterators¶
- Database instances.
- Entity instances.
- Query instances.
- Transaction instances.
- View instances.
sys
namespaces.