Internal API reference
Request
Request class used to handle HTTP.
- process_cookies(raw_cookies, pconfig)[source]
Process the raw cookies and replace with the real data.
- Return type
Dict
[str
,str
]
- process_headers(raw_headers, pconfig)[source]
Process the raw headers and replace with the real data.
- Return type
Dict
[str
,str
]
- process_data(raw_data, pconfig)[source]
Process the raw HTTP data and replace with the real data.
- Return type
Dict
[str
,str
]
- class Request(function, url, method, **kwargs)[source]
Class holding the elements of the HTTP request.
When a Flow object is created, it defines a Request object with the information necessary to create a HTTP request. The “method” and “url” attributes are required. Everything else is optional.
The Request object can contain Plugins which will be evaluated and its value replaced in the HTTP request.
- method
A string with the HTTP request method. Only GET and POST is supported for now.
- url
A string with the URL of the HTTP request.
- cookies
A list of Cookie objects to be sent with the HTTP request.
- headers
A list of Header objects to be sent with the HTTP request.
- data
A dictionary of Any objects. Can contain strings and Plugins. When a key or a value of the dictionary is a Plugin, it will be evaluated and its value will be used in the HTTP request. If the “method” is GET those values will be put inside the URL parameters, and if the “method” is POST they will be inside the POST request body.
- send(pconfig)[source]
Sends the HTTP request.
With the given user information, replaces the input plugins with their values, and sends the HTTP request. Returns the response.
- Parameters
user – A User object with the user specific data to be used when processing inputs.
pconfig – A Config object with the global Raider configuration.
- Return type
Optional
[Response
]- Returns
A requests.models.Response object with the HTTP response received after sending the generated request.
- class Template(method, url=None, cookies=None, headers=None, data=None)[source]
Template class to hold requests.
It will initiate itself with a
Request
parent, and when called will return a copy of itself with the modified parameters.
Structures
Data structures used in Raider.
- class DataStore(data)[source]
Class defining a dictionary-like data structure.
This class was created to hold information relevant to Raider in a structure similar to Python dictionaries.
- class HeaderStore(data)[source]
Class storing the HTTP headers.
This class inherits from DataStore, and converts the values into Header objects.
- __init__(data)[source]
Initializes the HeaderStore object.
Creates a HeaderStore object out of the given Header list.
- Parameters
data (
Optional
[List
[Header
]]) – A list of Header objects to store.
- set(header)[source]
Sets the value of a Header.
Given a Header object, add or update its value in the HeaderStore.
- Parameters
header (
Header
) – A Header object to be added to the HeaderStore.- Return type
None
- classmethod from_dict(data)[source]
Creates a HeaderStore object from a dictionary.
Given a dictionary with header values, creates a HeaderStore object and returns it.
- Parameters
data (
Optional
[Dict
[str
,str
]]) – A dictionary with header values. Those will be mapped in Header objects.- Return type
- Returns
A HeaderStore object containing the headers created from the supplied dictionary.
- class CookieStore(data)[source]
Class storing the HTTP cookies.
This class inherits from DataStore, and converts the values into Cookie objects.
- __init__(data)[source]
Initializes a CookieStore object.
Given a list of Cookie objects, create the CookieStore containing them.
- Parameters
data (
Optional
[List
[Cookie
]]) – A list of Cookies to be added to the CookieStore.
- set(cookie)[source]
Sets the value of a Cookie.
Given a Cookie object, add or update its value in the CookieStore.
- Parameters
cookie (
Cookie
) – A Cookie object to be added to the CookieStore- Return type
None
- classmethod from_dict(data)[source]
Creates a CookieStore object from a dictionary.
Given a dictionary with cookie values, creates a CookieStore object and returns it.
- Parameters
data (
Optional
[Dict
[str
,str
]]) – A dictionary with cookie values. Those will be mapped in Cookie objects.- Return type
- Returns
A CookieStore object containing the cookies created from the supplied dictionary.
User
Classes used for handling users.
- class User(username=None, password=None, **kwargs)[source]
Class holding user related information.
User
objects are created inside theUsers
. EachUser
object contains at least theusername
and thepassword
. Every time aPlugin
generates an output, it is saved in theUser
object. If thePlugin
is aCookie
or aHeader
, the output will be stored in the thecookies
andheaders
attributes respectively. Otherwise they’ll be saved insidedata
.- username
A string containing the user’s email or username used to log in.
- password
A string containing the user’s password.
- cookies
A
CookieStore
object containing all of the collected cookies for this user. TheCookie
plugin only writes here.
- headers
A
HeaderStore
object containing all of the collected headers for this user. TheHeader
plugin only writes here.
- __init__(username=None, password=None, **kwargs)[source]
Initializes a
User
object.Creates an object for easy access to user specific information. It’s used to store the
username
,password
,cookies
,headers
, and otherdata
extracted from thePlugin
objects.- Parameters
username (
Optional
[str
]) – A string with the username used for the login process.password (
Optional
[str
]) – A string with the password used for the login process.**kwargs (
Dict
[str
,str
]) – A dictionary with additional data about the user.
- set_cookie(cookie)[source]
Sets the
cookies
for the user.Given a
Cookie
object, update the user’scookies
attribute to include thisCookie's
value.
- set_cookies_from_dict(data)[source]
Set user’s
cookies
from a dictionary.Given a dictionary of cookie values as strings, convert them to
Cookie
objects, and load them in theUser
object respectively.- Parameters
data (
Dict
[str
,str
]) – A dictionary of strings corresponding to cookie keys and values.- Return type
None
- set_header(header)[source]
Sets the
headers
for the user.Given a
Header
object, update the user’sheaders
attribute to include this header value.
- set_headers_from_dict(data)[source]
Set user’s
headers
from a dictionary.Given a dictionary of header values as strings, convert them to
Header
objects, and load them in theUser
object respectively.- Parameters
data (
Dict
[str
,str
]) – A dictionary of strings corresponding to header keys and values.- Return type
None
- set_data(data)[source]
Sets the
data
for the user.Given a
Plugin
, update the user’sdata
attribute to include this data.
- class Users(users=None, active_user='DEFAULT')[source]
Class holding all the users of the application.
Users inherits from
DataStructure
, and contains the users set up in hyfiles. Each user is anUser
object. The data from aUsers
object can be accessed same way like from theDataStore
.
utils
Functions that are used within Raider.
- default_user_agent()[source]
Gets the default user agent.
Gets the current version of Raider and creates the user agent string.
- Return type
str
- Returns
A string with the user agent.
- get_config_dir()[source]
Gets the configuration directory.
Returns the path of the directory with the Raider configuration files.
- Return type
str
- Returns
A string with the path of the configuration directory.
- get_config_file(filename)[source]
Gets the configuration file.
Given the file name, it returns the path of this file in the Raider configuration directory.
- Parameters
filename (
str
) – A string with the name of the file to look up for in the main configuration directory.- Return type
str
- Returns
A string with the path of the file.
- get_project_dir(project)[source]
Gets the directory of the project.
Given the name of the project, returns the path to the directory containing the configuration files for this project.
- Parameters
project (
str
) – A string with the name of the project.- Return type
str
- Returns
A string with the path of the directory where the config files for the project are located.
- get_project_file(project, filename)[source]
Gets a file from a project.
Given the project name and the file name, it returns the path to that file.
- Parameters
project (
str
) – A string with the name of the project.filename (
str
) – A string with the file name.
- Return type
str
- Returns
The path of the file in the project directory.
- import_raider_objects()[source]
Imports Raider objects to use inside hy configuration files.
To make Raider objects visible inside hy files without using separate imports, this function does the imports and returns the locals() which is later used when evaluating hy files.
- Return type
Dict
[str
,Any
]- Returns
A dictionary with the locals() containing all the Raider objects that can be used in hy files.
- hy_dict_to_python(hy_dict)[source]
Converts a hy dictionary to a python dictionary.
When creating dictionaries in hylang using :parameters they become hy.models.Keyword objects. This function converts them to normal python dictionaries.
- Parameters
hy_dict (
Dict
[Keyword
,Any
]) – A dictionary created in hy, which uses hy.models.Keyword instead of simple strings as keys.- Return type
Dict
[str
,Any
]- Returns
A dictionary with the same elements only with hy.models.Keyword keys converted into normal strings.
- py_dict_to_hy_list(data)[source]
Converts a python dictionary to a hylang list.
In hy, dictionaries are created out of lists, and this function converts a normal python dictionary to a list made out of hy symbols that will be later used to create the hy dictionary.
- Parameters
data (
Dict
[str
,Any
]) – A python dictionary with the data to convert.- Return type
List
[Union
[String
,Dict
,Symbol
]]- Returns
A list with hy objects that can be used to create a hy dictionary.
- create_hy_expression(variable, value)[source]
Creates a hy expression.
Raider configuration is saved in hy format, and this function creates the assignments in this format.
- Parameters
variable (
str
) – A string with the name of the variable to be created.value (
Union
[str
,Dict
[Any
,Any
],List
[Any
]]) – The value of the variable.
- Return type
str
- Returns
A string with the valid hy expression.
- serialize_hy(form)[source]
Serializes hy expression.
This function serializes the supplied hy expression and returns it in a string format, so that it can be later saved in a file.
- Parameters
form (
Union
[Expression
,Dict
,List
,Symbol
,Integer
,Keyword
,String
]) – A hy expression to convert to a string.- Return type
str
- Returns
A string with the serialized form.
- eval_file(filename, shared_locals=None)[source]
Evaluate hy file.
This function evaluates all the content inside the supplied hy file, and returns the created locals() so that it can be later used for other files.
- Parameters
filename (
str
) – A string with the file name to be evaluated.shared_locals (
Optional
[Dict
[str
,Any
]]) – A dictionary with the locals() that will be considered when evaluating the file.
- Return type
Dict
[str
,Any
]- Returns
A dictionary with the updated locals() after evaluating the hy file.
- eval_project_file(project, filename, shared_locals)[source]
Evaluate a hy file from a project.
This function evaluates the specified file inside the project and returns the locals() which are updated after evaluating the file.
- Parameters
project (
str
) – A string with the name of the project.filename (
str
) – A string with the file name to be evaluated.shared_locals (
Dict
[str
,Any
]) – A dictionary of locals() to be included when evaluating the file.
- Return type
Dict
[str
,Any
]- Returns
A dictionary of locals() updated after evaluating the file.
- list_projects()[source]
List existing projects.
This function returns the list of projects that have been configured in Raider.
- Return type
List
[str
]- Returns
A list with the strings of the project found in the configuration directory.
- list_hyfiles(project)[source]
List hyfiles for a project.
This function returns the list of hyfiles that have been configured in Raider for the provided project.
- Parameters
project (
str
) – A string with the project name.- Return type
List
[str
]- Returns
A list with the strings of the project found in the configuration directory.
- match_tag(html_tag, attributes)[source]
Tells if a tag matches the search.
This function checks whether the supplied tag matches the attributes. The attributes is a dictionary, and the values are treated as a regular expression, to allow checking for tags that don’t have a static value.
- Parameters
html_tag (
Tag
) – A bs4.element.Tag object with the tag to be checked.attributes (
Dict
[str
,str
]) – A dictionary of attributes to check whether they match with the tag.
- Return type
bool
- Returns
A boolean saying whether the tag matched with the attributes or not.