yaql(1)

YAQL(1)

yaql

YAQL(1)

NAME

yaql - yaql

YAQL: YET ANOTHER QUERY LANGUAGE

YAQL (Yet Another Query Language) is an embeddable and extensible query language, that allows performing complex queries against arbitrary objects. It has a vast and comprehensive standard library of frequently used querying functions and can be extend even further with user-specified functions. YAQL is written in python and is distributed via PyPI.

Quickstart

Install the latest version of yaql:

pip install yaql>=1.0.0

Run yaql REPL:

yaql

Load a json file:

yaql> @load my_file.json

Check it loaded to current context, i.e. $:

yaql> $

Run some queries:

yaql> $.customers ... yaql> $.customers.orders ... yaql> $.customers.where($.age > 18) ... yaql> $.customers.groupBy($.sex) ... yaql> $.customers.where($.orders.len() >= 1 or name = "John")

Project Resources

Official Documentation
Project status, bugs, and blueprints are tracked on Launchpad

License

Apache License Version 2.0 http://www.apache.org/licenses/LICENSE-2.0

WHAT IS YAQL

YAQL is a general purpose query language, that is designed to operate on objects of arbitrary complexity. YAQL has a large standard library of functions for filtering, grouping and aggregation of data. At the same time YAQL allows you to extend it by defining your own functions.

WHY YAQL?

So why bother and create another solution for a task, that has been addressed by many before us? Obviously because we were not satisfied with flexibility and/or quality of any existing solution. Most notably we needed a tool for json data, that would support some complex data transformations. YAQL is a pure-python library and therefore is easily embeddable in any python application. YAQL is designed to be human-readable and has a SQL-like feel and look. It is inspired in part by LINQ for .NET. Since YAQL is extensible and embeddable it makes a perfect choice for becoming the basis for your DSLs.

GETTING STARTED WITH YAQL

Introduction to YAQL

YAQL (Yet Another Query Language) is an embeddable and extensible query language that allows performing complex queries against arbitrary data structures. Embeddable means that you can easily integrate a YAQL query processor in your code. Queries come from your DSLs (domain specific language), user input, JSON, and so on. YAQL has a vast and comprehensive standard library of functions that can be used to query data of any complexity. Also, YAQL can be extended even further with user-specified functions. YAQL is written in Python and is distributed through PyPI.

YAQL was inspired by Microsoft LINQ for Objects and its first aim is to execute expressions on the data in memory. A YAQL expression has the same role as an SQL query to databases: search and operate the data. In general, any SQL query can be transformed to a YAQL expression, but YAQL can also be used for computational statements. For example, 2 + 3*4 is a valid YAQL expression.

Moreover, in YAQL, the following operations are supported out of the box:

Complex data queries
Creation and transformation of lists, dicts, and arrays
String operations
Basic math operations
Conditional expression
Date and time operations (will be supported in yaql 1.1)

An interesting thing in YAQL is that everything is a function and any function can be customized or overridden. This is true even for built-in functions. YAQL cannot call any function that was not explicitly registered to be accessible by YAQL. The same is true for operators.

YAQL can be used in two different ways: as an independent CLI tool, and as a Python module.

Installation

You can install YAQL in two different ways:

1.: Using PyPi:

pip install yaql

2.: Using your system package manager (for example Ubuntu):

sudo apt-get install python-yaql

HowTo: Use YAQL in Python

You can operate with YAQL from Python in three easy steps:

Create a YAQL engine
Parse a YAQL expression
Execute the parsed expression

NOTE:

The engine should be created once for a set of operators and parser rules. It can be reused for all queries.

Here is an example how it can be done with the YAML file which looks like:

customers_city:


  - city: New York


    customer_id: 1


  - city: Saint Louis


    customer_id: 2


  - city: Mountain View


    customer_id: 3
customers:


  - customer_id: 1


    name: John


    orders:


      - order_id: 1


        item: Guitar


        quantity: 1


  - customer_id: 2


    name: Paul


    orders:


      - order_id: 2


        item: Banjo


        quantity: 2


      - order_id: 3


        item: Piano


        quantity: 1


  - customer_id: 3


    name: Diana


    orders:


      - order_id: 4


        item: Drums


        quantity: 1

import yaql
import yaml
data_source = yaml.load(open('shop.yaml', 'r'))
engine = yaql.factory.YaqlFactory().create()
expression = engine(


    '$.customers.orders.selectMany($.where($.order_id = 4))')
order = expression.evaluate(data=data_source)

Content of the order will be the following:

[{u'item': u'Drums', u'order_id': 4, u'quantity': 1}]

YAQL grammar

YAQL has a very simple grammar:

Three keywords as in JSON: true, false, null
Numbers, such as 12 and 34.5
Strings: 'foo' and "bar"
Access to the data: $variable, $
Binary and unary operators: 2 + 2, -1, 1 != 2, $list[1]

Data access

Although YAQL expressions may be self-sufficient, the most important value of YAQL is its ability to operate on user-passed data. Such data is placed into variables which are accessible in a YAQL expression as $<variable_name>. The variable_name can contain numbers, English alphabetic characters, and underscore symbols. The variable_name can be empty, in this case you will use $. Variables can be set prior to executing a YAQL expression or can be changed during the execution of some functions.

According to the convention in YAQL, function parameters, including input data, are stored in variables like $1, $2, and so on. The $ stands for $1. For most cases, all function parameters are passed in one piece and can be accessed using $, that is why this variable is the most used one in YAQL expressions. Besides, some functions are expected to get a YAQL expression as one of the parameters (for example, a predicate for collection sorting). In this case, passed expression is granted access to the data by $.

Strings

In YAQL, strings can be enclosed in " and '. Both types are absolutely equal and support all standard escape symbols including unicode code-points. In YAQL, both types of quotes are useful when you need to include one type of quotes into the other. In addition, ` is used to create a string where only one escape symbol ` is possible. This is especially suitable for regexp expressions.

If a string does not start with a digit or __ and contains only digits, _, and English letters, it is called identifier and can be used without quotes at all. An identifier can be used as a name for function, parameter or property in $obj.property case.

Functions

A function call has syntax of functionName(functionParameters). Brackets are necessary even if there are no parameters. In YAQL, there are two types of parameters:

•

Positional parameters: foo(1, 2, someValue)

•

Named parameters: foo(paramName1 => value1, paramName2 => 123)

Also, a function can be called using both positional and named parameters: foo(1, false, param => null). In this case, named arguments must be written after positional arguments. In name => value, name must be a valid identifier and must match the name of parameter in function definition. Usually, arguments can be passed in both ways, but named-only parameters are supported in YAQL since Python 3 supports them.

Parameters can have default values. Named parameters is a good way to pass only needed parameters and skip arguments which can be use default values, also you can simply skip parameters in function call: foo(1,,3).

In YAQL, there are three types of functions:

Regular functions: max(1,2)

Method-like functions, which are called by specifying an object for which the: function is called, followed by a dot and a function call: stringValue.toUpper()

•: Extension methods, which can be called both ways: len(string), string.len()

YAQL standard library contains hundreds of functions which belong to one of these types. Moreover, applications can add new functions and override functions from the standard library.

Operators

YAQL supports the following types of operators out of the box:

Arithmetic: +. -, *, /, mod
Logical: =, !=, >=, <=, and, or, not
Regexp operations: =~, !~
Method call, call to the attribute: ., ?.
Context pass: ->
Indexing: [ ]
Membership test operations: in

Data structures

YAQL supports these types out of the box:

•: Scalars

YAQL supports such types as string, int. boolean. Datetime and timespan will be available after yaql 1.1 release.

•: Lists

List creation: [1, 2, value, true] Alternative syntax: list(1, 2, value, true) List elemenets can be accesessed by index: $list[0]

•: Dictionaries

Dict creation: {key1 => value1, true => 1, 0 => false} Alternative syntax: dict(key1 => value1, true => 1, 0 => false) Dictionaries can be indexed by keys: $dict[key]. Exception will be raised if the key is missing in the dictionary. Also, you can specify value which will be returned if the key is not in the dictionary: dict.get(key, default).

NOTE:

During iteration through the dictionary, key can be called like: $.key

•: (Optional) Sets

Set creation: set(1, 2, value, true)

NOTE:

YAQL is designed to keep input data unchanged. All the functions that look as if they change data, actually return an updated copy and keep the original data unchanged. This is one reason why YAQL is thread-safe.

Basic YAQL query operations

It is obvious that we can compare YAQL with SQL as they both are designed to solve similar tasks. Here we will take a look at the YAQL functions which have a direct equivalent with SQL.

We will use YAML from HowTo: use YAQL in Python as a data source in our examples.

Filtering

NOTE:

Analog is SQL WHERE

The most common query to the data sets is filtering. This is a type of query which will return only elements for which the filtering query is true. In YAQL, we use where to apply filtering queries.

yaql> $.customers.where($.name = John)

- customer_id: 1


  name: John


  orders:


    - order_id: 1


      item: Guitar


      quantity: 1

Ordering

NOTE:

Analog is SQL ORDER BY

It may be required to sort the data returned by some YAQL query. The orderBy clause will cause the elements in the returned sequence to be sorted according to the default comparer for the type being sorted. For example, the following query can be extended to sort the results based on the profession property.

yaql> $.customers.orderBy($.name)

- customer_id: 3


  name: Diana


  orders:


    - order_id: 4


      item: Drums


      quantity: 1
- customer_id: 1


  name: John


  orders:


    - order_id: 1


      item: Guitar


      quantity: 1
- customer_id: 2


  name: Paul


  orders:


    - order_id: 2


      item: Banjo


      quantity: 2


    - order_id: 3


      item: Piano


      quantity: 1

Grouping

NOTE:

Analog is SQL GROUP BY

The groupBy clause allows you to group the results according to the key you specified. Thus, it is possible to group example json by gender.

yaql> $.customers.groupBy($.name)

- Diana:


  - customer_id: 3


    name: Diana


    orders:


      - order_id: 4


        item: Drums


        quantity: 1
- Paul:


  - customer_id: 2


    name: Paul


    orders:


      - order_id: 2


        item: Banjo


        quantity: 2


      - order_id: 3


        item: Piano


        quantity: 1
- John:


  - customer_id: 1


    name: John


    orders:


      - order_id: 1


        item: Guitar


        quantity: 1

So, here you can see the difference between groupBy and orderBy. We use the same parameter name for both operations, but in the output for groupBy name is located in additional place before everything else.

Selecting

NOTE:

Analog is SQL SELECT

The select method allows building new objects out of objects of some collection. In the following example, the result will contain a list of name/orders pairs.

yaql> $.customers.select([$.name, $.orders])

- John:


  - order_id: 1


    item: Guitar


    quantity: 1
- Paul:


  - order_id: 2


    item: Banjo


    quantity: 2


  - order_id: 3


    item: Piano


    quantity: 1
- Diana:


  - order_id: 4


    item: Drums


    quantity: 1

Joining

NOTE:

Analog is SQL JOIN

The join method creates a new collection by joining two other collections by some condition.

yaql> $.customers.join($.customers_city, $1.customer_id = $2.customer_id, {customer=>$1.name, city=>$2.city, orders=>$1.orders})

- customer: John


  city: New York


  orders:


    - order_id: 1


      item: Guitar


      quantity: 1
- customer: Paul


  city: Saint Louis


  orders:


    - order_id: 2


      item: Banjo


      quantity: 2


    - order_id: 3


      item: Piano


      quantity: 1
- customer: Diana


  city: Mountain View


  orders:


    - order_id: 4


      item: Drums


      quantity: 1

Take an element from collection

YAQL supports two general methods that can help you to take elements from collection skip and take.

yaql> $.customers.skip(1).take(2)

- customer_id: 2


  name: Paul


  orders:


    - order_id: 2


      item: Banjo


      quantity: 2


    - order_id: 3


      item: Piano


      quantity: 1
- customer_id: 3


  name: Diana


  orders:


    - order_id: 4


      item: Drums


      quantity: 1

First element of collection

The first method will return the first element of a collection.

yaql> $.customers.first()

- customer_id: 1


  name: John


  orders:


    - order_id: 1


      item: Guitar


      quantity: 1

USAGE

This section is not ready yet.

Embedding YAQL

REPL utility

LANGUAGE REFERENCE

YAQL is a single expression language and as such does not have any block constructs, line formatting, end of statement marks or comments. The expression can be of any length. All whitespace characters (including newline) that are not enclosed in quote marks are stripped. Thus, the expressions may span multiple lines.

Expressions consist of:

Literals
Keywords
Variable access
Function calls
Binary and unary operators
List expressions
Dictionary expressions
Index expressions
Delegate expressions

Terminology

YAQL - the name of the language - acronym for Yet Another Query Language
yaql - Python implementation of the YAQL language (this package)
expression - a YAQL query that takes context as an input and produces result value
context - an object that (directly or indirectly) holds all the data available to expression and all the function implementations accessible to expression
host - the application that hosts the yaql interpreter. The host uses yaql to evaluate expressions, provides initial data, and decides which functions are going to be available to the expression. The host has ultimate power to customize yaql - provide additional functions, operators, decide not to use standard library or use only parts of it, override function and operator behavior
variable - any data item that is available through the context
function - a Python callable that is exposed to the YAQL expression and can be called either explicitly or implicitly
delegate - a Python callable that is available as a context variable (in expression data rather than registered in context)
operator - a form of implicit function on one (unary operator) or two (binary operator) operands
alphanumeric - consists of latin letters and digits (A-Z, a-z, 0-9)

Literals

Literals refer to fixed values in expressions. YAQL has the following literals:

Integer literals: 123
Floating point literals: 1.23, 1.0
Boolean and null literals represented by keywords (see below)
String literals enclosed in either single (') or double (") quotes: "abc", 'def'. The backslash () character is used to escape characters that otherwise have a special meaning, such as newline, backslash itself, or the quote character
Verbatim strings enclosed in back quote characters, for example `abc`, are used to suppress escape sequences. This is equivalent to r'strings' in Python and is especially useful for regular expressions

Keywords

Keyword is a sequence of characters that conforms to the following criteria:

Consists of non-zero alphanumeric characters and an underscore (_)
Doesn't start with a digit
Doesn't start with two underscore characters (__)
Is not enclosed in quote marks of any type

YAQL has only three predefined keywords: true, false, and null that have the value of similar JSON keywords.

There are also four keyword operators: and, or, not, in. However, this list is not fixed. The yaql host may decide to have additional keyword operators or not to have any of the four aforementioned keywords.

All other keywords have the value of their string representation. Thus, except for the predefined keywords and operators they can be considered as string literals and can be used anywhere where string is expected. However the opposite is not true. That is, keywords can be used as string literals but string literals cannot be used where a token is expected.

Examples:

John + Snow - the same as "John" + "Snow"
true + love - syntactically valid, but cannot be evaluated because there is no plus operator that accepts boolean and string (unless you define one)
not true - evaluates to false, not is an operator
"foo"() - invalid expression because the function name must be a token
John Snow - invalid expression - two tokens with no operator between them

Variable access

Each YAQL expression is a function that takes inputs (arguments) and produces the result value (usually by doing some computations on those inputs). Expressions get the input through a context - an object that holds all the data and a list of functions, available for expression.

Besides the argument values, expressions may populate additional data items to the context. All these data are collectively known as a variables and available to all parts of an expression (unless overwritten with another value).

The syntax for accessing variable values is $variableName where variableName is the name of the variable. Variable names may consist of alphanumeric and underscore characters only. Unlike tokens, variable names may start with digit, any number of underscores and even be an empty string. By convention, the first (usually the single) function parameter is accessible through $ expression (i.e. empty string variable name) which is an alias for $1. The usual case is to pass the main expression data in a single structure (document) and access it through the $ variable.

If the variable with given name is not provided, it is assumed to be null. There is no built-in syntax to check if a variable exists to distinguish cases where it does not and when it is just set to null. However in the future such a function might be added to yaql standard library.

When the yaql parser encounters the $variable expression, it automatically translates it to the #get_context_data("$variable") function call. By default, the #get_context_data function returns a variable value from the current context. However the yaql host may decide to override it and provide another behavior. For example, the host may try to look up the value in an external data source (database) or throw an exception due to a missing variable.

Function calls

The power of YAQL comes from the fact that almost everything in YAQL is a function call (explicit or implicit) and any function may be overridden by the host. In YAQL there are two types of functions:

explicit function - those that can be called from expressions
implicit (system) functions - functions with predefined names that get called upon some operations. For example, 2 + 3 is translated to #operator_+(2, 3). In this case, #operator_+ is the name of the implicit function. However, because #operator_+(2, 3) is not a valid YAQL expression (because of #), implicit functions cannot be called explicitly but still can be redefined by the host.

The syntax for explicit function is:

call                 ::=  funcName "(" [parameters] ")"
funcName             ::=  token
parameters           ::=  positionalParameters |


                          keywordParameters |


                          positionalParameters "," keywordParameters
positionalParameters ::=  parameter ("," parameter)*
parameter            ::=  expression | empty-string
keywordParameters    ::=  keywordParameter ("," keywordParameter)
keywordParameter     ::=  parameterName "=>" expression
parameterName        ::=  token

In simple words:

The function name must be a token.
Parameters may be positional, keyword or both. But keyword parameters may not come before positional.
Positional parameters can be skipped if they have a default value, for example, foo(1,,3).
Keyword arguments must have a token name that must match the parameter name in the function declaration. Therefore, you must know the function signature for the right name.

Examples:

foo(2 + 3)
bar(hello, world)
baz(a,b, kwparam1 => c, kwparam2 => d)

Functions have ultimate control over how they can be called. In particular:

Each parameter may (and usually does) have an associated type check. That is, the function may specify that the expected parameter type and if it can be null.
Usually, any parameters can be passed either by positional or keyword syntax. However, function declaration may force one particular way and make it positional-only or keyword-only.
A function may have a variable number of positional (aka *args) and/or keyword (aka **kwarg) arguments.
In most languages, function arguments are evaluated prior to function invocation. This is not always true in YAQL. In YAQL, a function may declare a lazy argument. In this case, it is not evaluated and the function implementation receives a passed value as a callable or even as an AST, depending on how the parameter was declared. Thus in YAQL there is no special syntax for lambdas. foo($ + 1) may mean either "call foo with value of $ + 1" or "call foo with expression $ + 1 as a parameter". In the latter case it corresponds to foo(lambda *args, **kwargs: args[0] + 1) in Python. Actual argument interpretation depends on the parameter declaration.
Function may decide to disable keyword argument syntax altogether. For such functions, the name => expr expression will be interpreted as a positional parameter yaql.language.utils.MappingRule(name, expr) and the left side of => can be any expression and not just a keyword. This allows for functions like switch($ > 0 => 1, $ < 0 => -1, $ = 0 => 0).

Additionally, there are three subtypes of explicit functions. Suppose that there is a declared function foo(string, int). By default, the syntax to call it will be foo(something, 123). But it can be declared as a method. In this case, the syntax is going to be something.foo(123). Because of the type checking, something.foo(123) will work since something is a string, but not the 123.foo(456). Thus foo becomes a method of a string type.

A function may also be declared as being an extension method. If foo were to be declared as an extension method it could be called both as a function (foo(string, int)) and as a method (something.foo(123)).

YAQL makes use of a full function signature to determine which function implementation needs to be executed. This allows several overloads of the same function as long as they differ by parameter count or parameter type, or anything else that allows unambiguous identification of the right overload from the function call expression. For example, something.foo(123) may be resolved to a completely different implementation of foo from that in foo(something, 123) if there are two functions with the name foo present in the context, but one of them was declared as a function while the other as a method. If several overloads are equally suitable for the call expression, an AmbiguousFunctionException or AmbiguousMethodException exception gets raised.

Operators

YAQL has both binary and unary operators, like most other languages do. Parentheses and => sequence are not considered as operators and handled internally by the yaql parser. However, it is possible to configure yaql to use sequence other than => for that purpose.

The list of available operators is not fixed and can be modified by the host. The following operators are available by default:

Binary operators:

Group	Operators
math operators	+, -, , /, mod*
comparision operators	>, <, >=, <=, =, !=
logical operators	and, or
method/member access	., ?.
regex operators	=~, !~
membership operator	in
context passing operator	->

Unary operators:

Group	Operators
math operators	+, -
logical operators	not

YAQL supports for both prefix and suffix unary operators. However, only the prefix operators are provided by default.

In YAQL there are no built-in operators. The parser is given a list of all possible operator names (symbols), their associativity, precedence, and type, but it knows nothing about what operators are applicable for what operands. Each time a parser recognizes the X OP Y construct and OP is a known binary operator name, it translates the expression to #operator_OP(X, Y). Thus. 2 + 3 becomes #operator_+(2, 3) where #operator_+ is an implicit function with several implementations including the one for number addition and defined in standard library. The host may override it and even completely disable it. For unary operators, OP X (or X OP for suffix unary operators) becomes #unary_operator_OP(X).

Upon yaql parser initialization, an operator might be given an alias name. In such cases, X OP Y is translated to *ALIAS(X, Y) and OP X to *ALIAS(X). This decouples the operator implementation from the operator symbol. For example, the = operator has the equal alias. The host may configure yaql to have the == operator instead of = keeping the same alias so that operator implementation and all its consumers work equally well for the new operator symbol. In default configuration only = and != operators have alias names.

For information on default operators, see the YAQL standard library reference.

List expressions

List expressions have the following form:

listExpression ::=  "[" [expressions] "]"
expressions    ::=  expression ("," expression)*

When a yaql parser encounters an expression of the form [A, B, C], it translates it into #list(A, B, C) (for arbitrary number of arguments).

Default #list function implementation in standard library produces a list (tuple) comprised of given elements. However, the host might decide to give it a different implementation.

Map expressions

Map expressions have the following form:

mapExpression ::=  "{" [mappings] "}"
mappings      ::=  mapping ("," mapping)*
mapping       ::=  expression "=>" expression

When a yaql parser encounters an expression of the form {A => X, B => Y}, it translates it into #map(A => X, B => Y).

The default #map implementation disables the keyword arguments syntax and thus receives a variable length list of mappings, which allows dictionary keys to be expressions rather than a keyword. It returns a (frozen) dictionary that itself can be used as a key in another map expression. For example, {{a => b} => {[2 + 2, 2 * 2] => 4}} is a valid YAQL expression though yaql REPL utility will fail to display its output due to the fact that it is not JSON-compatible.

Index expressions

Index expressions have the following form:

indexExpression ::=  expression listExpression

Examples:

[1, 2, 3][0]
$arr[$index + 1]
{foo => 1, bar => 2}[foo]

When a yaql parser encounters such an expression, it translates it into #indexer(expression, index).

The standard library provides a number of #indexer implementations for different types.

The right side of the index expression is a list expression. Therefore, an expression like $foo[1, x, null] is also a valid YAQL expression and will be translated to #indexer($foo, 1, x, null). However, any attempt to evaluate such expression will result in NoMatchingFunctionException exception because there is no #indexer implementation that accepts such arguments (unless the host defines one).

Delegate expressions

Delegate expressions is an optional language feature that is disabled by default. It makes possible to pass delegates (callables) as part of the context data and invoke them from the expression. It has the same syntax as explicit function calls with the only difference being that instead of function name (keyword) there is a non-keyword expression that must produce the delegate.

Examples:

$foo(1, arg => 2) - call delegate returned by $foo with parameters (1, arg => 2)
[$foo, $bar][0](x) - the same as $foo(x)
foo()() - can be written as (foo())() - foo() must return a delegate

Delegate expressions are translated into #call(callable, arguments). Thus $foo(1, 2) becomes #call($foo, 1, 2).

The default implementation of #call invokes the result of the evaluation of its first arguments with the given arguments.

CUSTOMIZING AND EXTENDING YAQL

Configuring yaql parser

yaql has two main points of customization:

yaql engine settings allow one to configure the query language and execution flags shared by all queries that are processed by the same YAQL parser. This includes the list of available operators, yaql resources quotas, and other engine parameters.
By customizing the yaql context object, one can change the list of available functions (add new, override existing) and change naming conventions.

Engine options are supplied to the yaql.language.factory.YaqlFactory class. YaqlFactory is used to create instances of the YaqlEngine, that is the YAQL parser. This is done by calling the create method of the factory. Once the engine is created, it captures all the factory options so that they cannot be changed for that particular parser any longer. In general, it is recommended to have one yal engine instance per application, because construction of the parser is an expensive operation and the parser has no internal state and thus can be reused for several queries, including in different threads. However, the host may have several YAQL parsers for different option sets or dialects.

On the contrary, the context object is cheap to create and is mutable by design, since it holds the input data for the query. In most cases it is a good idea to execute each query in its own context, although all such contexts might be the children of some other, fixed context that is created just once.

Customizing operators

YaqlFactory object holds an operator table that is recognized by the parser produced by it. By default, it is prepopulated with standard operators and most applications never need to do anything here. However, if the host wants to have some custom operator symbol available in its expressions, this table needs to be modified. YaqlFactory holds the operator symbols and other information about the operator that is relevant to the parser, but not the implementations. The implementations (what operators actually do) are put in the context and can be configured for each expression, but the list of available operator symbols cannot be changed for the parser once it has been built.

Each operator in the table is represented by the tuple (op_symbols, op_type, op_alias):

op_symbols are the operator symbols. There are no limitations on how the operators can be called as long as they do not contain whitespaces. It can be one symbol (like +), several symbols (like =~) or even a word (like not). List/index and dictionary expressions require [] and {} binary left associative operators to be present in the table. Otherwise corresponding constructions will not work (and can be disabled by removing corresponding operators from the table)
op_type is one of the values in yaql.language.factory.OperatorType enumeration: BINARY_LEFT_ASSOCIATIVE and BINARY_RIGHT_ASSOCIATIVE for binary operators, PREFIX_UNARY and SUFFIX_UNARY for unary operators, NAME_VALUE_PAIR for the keyword/mapping pseudo-operator (that is =>, by default).
op_alias is the alias name for the operator. See YAQL language reference on how operator aliases are used. Aliases are optional and most operators do not have it and thus are represented by a tuple of two elements.

Operators are grouped by their precedence. Operators with a higher precedence come first in the operator table. Operators within the same group have the same precedence. Groups are separated by an empty tuple (()).

The operator table, which is a list of tuples, is available through the operators attribute of the factory and is open for modification. To simplify the editing, YaqlFactory provides the insert_operator helper method to insert an operator before of after some other existing operator to get the desired precedence.

Execution options

Execution options are the settings and flags that affect execution of each query and are accessible and processed by both yaql runtime and standard library functions.

Options are passed to the create method of the YaqlFactory class in a plain key-value dictionary. The factory does not process the dictionary but rather attaches the options to the constructed engine (YAQL parser) after which they cannot be changed. However, the engine provides a copy method that can be used to clone the engine with different execution options.

The options that are honored by the yaql are:

"yaql.limitIterators": <INT> limit iterators by the given number of elements. When set, each time any function declares its parameter to be iterator, that iterator is modified to not produce more than a given number of items. Also, upon the expression evaluation, all the output collections and iterators are limited as well. If not set (or set to -1) the result data is allowed to contain endless iterators that would cause errors if the result where to be serialized (to JSON or any other format). Default is -1 (do not limit).
"yaql.memoryQuota": <INT> - the memory usage quota (in bytes) for all data produced by the expression (or any part of it). Default is -1 (do not limit).
"yaql.convertTuplesToLists": <True|False>. When set to true, yaql converts all tuples in the expression result to lists. The default is True.
"yaql.convertSetsToLists": <True|False>. When set to true, yaql converts all sets in the expression result to lists. Otherwise the produced result may contain sets that are not JSON-serializable. The default is False.
"yaql.iterableDicts": <True|False>. When set to true, dictionaries are considered to be iterable and iteration over dictionaries produces their keys (as in Python and yaql 0.2). Defaults to False.

Consumers are free to use their own settings or use the options dictionary to provide some other environment information to their own custom functions.

Other engine customizations

YaqlFactory class initializer has two optional parameters that can be used to further customize the YAQL parser:

keyword_operator allows one to configure keyword/mapping symbol. The default is =>. Ability to pass named arguments can be disabled altogether if None or empty string is provided.
allow_delegates enables or disables delegate expression parsing. Default is False (disabled).

Working with contexts

Context is an interface that yaql runtime uses to obtain a list of available functions and variables. Any context object must implement yaql.language.contexts.ContextBase interface and yaql provides several such implementations ranging from the yaql.language.contexts.Context class, that is a basic context implementation, to contexts that allow one to merge several other contexts into one or link an existing context into the list of contexts.

Any context may have a parent context. Any lookup that is done in the context is also performed in its parent context, extending all the way up its chain of contexts. During expression evaluation, yaql can create a long chain of contexts that are all children of the context that was originally passed with the query.

Most of the yaql customizations are achieved by context manipulations. This includes:

Overriding YAQL functions
Building context chains and evaluating sub-expressions in different contexts
Composing context chains from pre-built contexts
Having custom ContextBase implementations and mixing them with regular contexts in the single chain

In fact, it is the context which provides the entry point for expression evaluation. And thus custom context implementations may completely change the way queries are evaluated.

There are three ways to create a context instance:

1.: Directly instantiate one of ContextBase implementations to get an empty context
2.: Call create_child_context method on any existing context object to get a child context
3.: Use yaql.create_context function to creates the root context that is prepopulated with YAQL standard library functions

yaql.create_context allows one to selectively disable standard library modules.

Naming conventions

Naming conventions define how Python functions and parameter names are translated into YAQL names. Conventions are implementations of the yaql.language.conventions.Convention interface that has just two methods: one to translate the function name and another to translate the function parameter name.

yaql has two implementations included:

yaql.language.conventions.CamelCaseConvention that translates Python conventions into camel case. For example, it will convert my_func(arg_name) into myFunc(argName). This convention is used by default.
yaql.language.conventions.PythonConvention that leaves function and parameter names intact.

Each context, either directly or indirectly through its parent context, is configured to use some convention. When a function is registered in the context, its name and parameters are translated with the convention methods. Also, regardless of convention used, all trailing underscores are stripped from the names. This makes it possible to define several Python functions that differ only by trailing underscores and get the same name in YAQL (to create several overloads of single function). Also, this allow one to have function or parameter names that would otherwise conflict with Python keywords.

Instance of convention class can be specified as a context initializer parameter or as a parameter of yaql.create_context function. Child contexts created with the create_child_context method inherit their parent convention.

Extending yaql

Extending yaql with new functions

For a function to become available to YAQL queries, it must be present in the provided context object. The default context implementation (yaql.language.contexts.Context) has a register_function method to register the function implementation.

In yaql, all functions are represented by instances of the yaql.language.specs.FunctionDefinition class. FunctionDefinition describes the complete function signature including:

Function name
List of parameters - instances of yaql.language.specs.ParameterDefinition
Function payload (Python callable)
Function type: function, method or extension method
The flag to disable the keyword arguments syntax for the function
Documentation string
Custom function metadata (dict)

register_function method can accept either an instance of the FunctionDefinition class or a regular Python function. In the latter case, it constructs a FunctionDefinition instance from the declaration of the function using Python introspection. Because a YAQL function signature has much more information than the Python one, yaql provides a number of function decorators that can be used to fill the missing properties.

The decorators are located in the yaql.language.specs module. Below is the list of available function decorators:

@name(function_name): set function name to be function_name rather than its Python name
@parameter(...) is used to declare the type of one of the function parameters
@inject(...) is used to declare a hidden function parameter
@method declares function to be YAQL method
@extension_method declares function to be YAQL extension method
@no_kwargs disables the keyword arguments syntax for the function
@meta(name, value) appends the name attribute with the given value to the function metadata dictionary

Specifying function parameter types

When yaql constructs FunctionDefinition, it collects all possible information about its parameters. For each parameter, it records its name, position, whether it is a keyword-only argument (available in Python 3), whether it is an *args or **kwargs, and its default parameter value.

The only parameter attribute that cannot be obtained through retrospection is the parameter type. For that purpose, yaql has a @parameter(name, type) decorator that can be used to explicitly declare the parameter type. name must match the name of one of the function parameters, and type must be of the yaql.language.yaqltypes.SmartType type.

SmartType is the base class for all yaql type descriptors - classes that check if the value is compatible with the desired type and can do type conversion between compatible types.

YAQL type system slightly differs from Python's:

Strings are not considered to be collections of characters
Booleans are not integers
Dictionaries are not iterable
For most of the types one can specify if the null (None) value is acceptable

yaql.language.yaqltypes module has many useful smart-type classes. The most generic smart-type for primitive types is the PythonType class, that validates if the value is instance of a given Python type. Due to the mentioned differences between YAQL and Python type systems and because Python types have a lot of nuances (several string types, differences between Python 2 and Python 3, separation between mutable and immutable type versions: list-tuple, set-frozenset, dict-FrozenDict, which is missing in Python and provided by the yaql instead), yaql provides specialized smart-types for most primitive types:

String - str and unicode
Integer
Number - integer of float
DateTime
Sequence - fixed-size iterable collection, except for the dictionary
Iterable - any iterable or generator
Iterator - iterator over the iterable

And several specialized variants that enforce particular representation in the YAQL syntax:

Keyword
BooleanConstant
NumericConstant
StringConstant

It is also possible to aggregate several smart-types so that the value can be of any given type or conform to all of them:

AnyOf
Chain
NotOfType

These three smart-types accept other smart-type(s) as their initializer parameter(s).

In addition to the smart-types, the second parameter of the @parameter can be a Python type. For example, @parameter("name", unicode) or @parameter("name", unicode, nullable=True). In this case the Python type is automatically wrapped in the PythonType smart-type. If nullability is not specified, yaql tries to infer it from the parameter declaration - it is nullable only if the parameter has its default value set to None.

Lazy evaluated function parameters

All the smart-types from the previous section are for parameters that are evaluated before the function gets invoked. But sometimes the function might need the parameter to remain unevaluated so that it can be evaluated by the function itself, possibly with additional parameters or in a different context.

There are two possible representations of non-evaluated arguments:

Get it as a Python callable that the function can call to do the evaluation
Get it as a YAQL expression (AST), that can be analyzed

The first method is available through the Lambda smart-type. The parameter, which is declared as a Lambda(), has an *args/**kwargs signature and can be called from the function: parameter(arg1, arg2). If it was declared as Lambda(with_context=True) the function may invoke it in a context, other than that which is used for the function: parameter(new_context, arg1, arg2). Lambda(method=True) specifies that the parameter must be a method and the caller can specify the receiver object for it: parameter(receiver, arg1, arg2). Parameters can also be combined: Lambda(with_context=True, method=True) so the callable is invoked as parameter(receiver, new_context, arg1, arg2). All supplied callable arguments are automatically published to the $1 ($), $2 and so on context variables for the context in which the callable will be executed.

The second method is available through the YaqlExpression smart-type. It also allows one to request the parameter to be of a particular expression type rather than an arbitrary YAQL expression.

Auto-injected function parameters

Besides regular parameters, yaql also supports auto-injected (hidden) parameters. This is also known as a function parameter dependency injection. The values of injected parameters come from the yaql runtime rather than from the caller. Functions use injected parameters to get information on their execution environment.

Auto-injected parameters are declared using the @inject(...) decorator, which has exactly the same signature as @parameter with the only difference being that @inject checks that that the supplied smart-type is an instance of the yaql.language.yaqltypes.HiddenParameterType class (in addition to SmartType), whereas the @parameter decorator checks that it is not. This difference exists to clearly distinguish explicitly passed parameters from those that are injected by the system.

yaql has the following hidden parameter smart types:

Context - injects the current function context object
Engine - injects YaqlEngine object that was used to parse the expression. Engine object may be used to access execution options or to parse some other expression
FunctionDefinition - FunctionDefinition object of the function. May be used to obtain function metadata and doc-string
Delegate - injects a Python callable to some other YAQL function by its name. This is a convenient way to call one YAQL function from another without depending on its Python implementation signature and location. The syntax is very similar to Lambda smart-type
Super - similar to Delegate - injects callable to an overload of itself from the parent context. Useful when the function overload wants to call its base implementation (analogous to Python's super())
Receiver - injects a method receiver object if the function was called as a method and None otherwise. Can be used in an extension method to distinguish the case, when it was invoked as a method rather than as a function. Do not do it without a good reason!
YaqlInterface - injects a convenient wrapper (YaqlInterface) around yaql functionality, which also encapsulates many of the values above

Auto-injected parameters may appear anywhere in the function signature as they do not affect caller syntax. Implementations can add additional hidden parameters without breaking existing queries. However, it is important to call YAQL function implementations through the yaql mechanisms (such as Delegate), rather than to call their Python implementations directly.

Automatic parameters

In some cases there is no need to declare the parameter at all. yaql uses parameter name and default value to guess the parameter type if it was not declared.

If the parameter name is context or __context it will automatically be treated as if it was declared as a Context. engine/__engine is considered as an Engine, and yaql_interface/__yaql_interface is considered as a YaqlInterface.

The host can override this logic by providing a callable to Context's register_function method through the parameter_type_func parameter. When yaql encounters an undeclared parameter, it calls this function, passing the parameter name as an argument, and expects it to return a smart-type for the parameter.

If the parameter_type_func callable returned None, yaql would assume that the smart type should be PythonType(object), that is anything, except for the None value, unless the parameter had the default value None.

Function resolution rules

Function resolution rules are used to determine the correct overload of the function when more than one overload is present in the context. Each time a function with a given list of parameters is called yaql does the following:

1.: Walks through the chain of context objects and collects all the implementations with a given name and appropriate type (either functions and extension methods or methods and extensions methods, depending on the call syntax).
2.: All found overloads are organized into layers so that overloads from the same context will be put in the same layer whereas overloads from different contexts are in different layers. Overloads from contexts that are closer to the initial context have precedence over those which were obtained from the parent contexts. Also FunctionDefinition may have a flag that prevents all overload lookups in the parent contexts. If the search encounters an overload with such a flag, it does not go any further in the chain.
3.: Scan all found overloads and exclude those, that cannot be called by the given syntax. This can happen because the overload has more mandatory parameters than the arguments in the calling expression, or because it passes the argument using the keyword name and no such parameter exists.
4.: Validates laziness of overload parameters. If at least one function overload has a lazy evaluated parameter all other overloads must have it in the same position. Violation of this rule causes an exception to be thrown.
5.: All the non-lazy parameters are evaluated. The result values are validated by appropriate smart-type instances corresponding to each parameter of each overload. All the overloads that are not type-compatible with the given arguments are excluded in each layer.
6.: Take first non-empty layer. If no such layer exists (that is all the overloads were excluded) then throw an exception.
7.: If the found layer has more than one overload, then we have an ambiguity. In this case an exception is thrown since we cannot unambiguously determine the right overload.
8.: Otherwise, call the single overload with previously evaluated arguments.

Function development hints

Avoid side effects in your functions, unless you absolutely have to.
Do not make changes to the data structures coming from the parameters or the context. Functions that modify the data should return the modified copy rather than touch the original.
If you need to make changes to the context, create a child context and make them there. It is usually possible to pass the new context to other parts of the query.
Strongly prefer immutable data structures over mutable ones. Use tuple`s rather than `list`s, `frozenset instead of set. Python does not have a built-in immutable dictionary class so yaql provides one on its own - yaql.language.utils.FrozenDict.
Do not call Python implementation of YAQL functions directly. yaql provides plenty of ways to do so.
Do not reuse contexts between multiple queries unless it is intentional. However all of these contexts can be children of a single prepared context.
Do not register all the custom functions for each query. It is better to prepare all the contexts with functions at the beginning and then use child contexts for each query executed.

STANDARD YAQL LIBRARY

Comparison operators

Common module describes comparison operators for different types. Comparing with null value is considered separately.

operator !=

Returns true if left and right are not equal, false otherwise.

It is system function and can be used to override behavior of comparison between objects.

operator <

•: Returns false. This function is called when left is not null and right is null.

signature: left < right
callAs: operator
arg left: left operand
argType left: not null
arg right: right operand
argType right: null
returnType: boolean

•: Returns false. This function is called when left and right are null.

signature: left < right
callAs: operator
arg left: left operand
argType left: null
arg right: right operand
argType right: null
returnType: boolean

•: Returns true. This function is called when left is null and right is not.

signature: left < right
callAs: operator
arg left: left operand
argType left: null
arg right: right operand
argType right: not null
returnType: boolean

operator <=

•: Returns false. This function is called when left is not null and right is null.

signature: left <= right
callAs: operator
arg left: left operand
argType left: not null
arg right: right operand
argType right: null
returnType: boolean

•: Returns true. This function is called when left and right are null.

signature: left <= right
callAs: operator
arg left: left operand
argType left: null
arg right: right operand
argType right: null
returnType: boolean

•: Returns true. This function is called when left is null and right is not.

signature: left <= right
callAs: operator
arg left: left operand
argType left: null
arg right: right operand
argType right: not null
returnType: boolean

operator =

Returns true if left and right are equal, false otherwise.

It is system function and can be used to override behavior of comparison between objects.

operator >

•: Returns true. This function is called when left is not null and right is null.

signature: left > right
callAs: operator
arg left: left operand
argType left: not null
arg right: right operand
argType right: null
returnType: boolean

•: Returns false. This function is called when left and right are null.

signature: left > right
callAs: operator
arg left: left operand
argType left: null
arg right: right operand
argType right: null
returnType: boolean

•: Returns false. This function is called when left is null and right is not.

signature: left > right
callAs: operator
arg left: left operand
argType left: null
arg right: right operand
argType right: not null
returnType: boolean

operator >=

•: Returns true. This function is called when left is not null and right is null.

signature: left >= right
callAs: operator
arg left: left operand
argType left: not null
arg right: right operand
argType right: null
returnType: boolean

•: Returns true. This function is called when left and right are null.

signature: left >= right
callAs: operator
arg left: left operand
argType left: null
arg right: right operand
argType right: null
returnType: boolean

•: Returns false. This function is called when left is null and right is not.

signature: left >= right
callAs: operator
arg left: left operand
argType left: null
arg right: right operand
argType right: not null
returnType: boolean

Boolean logic functions

Whenever an expression is used in the context of boolean operations, the following values are interpreted as false: false, null, numeric zero of any type, empty strings, empty dict, empty list, empty set, zero timespan. All other values are interpreted as true.

bool

Returns true or false after value type conversion to boolean. Function returns false if value is 0, false, empty list, empty dictionary, empty string, empty set, and timespan(). All other values are considered to be true.

signature: bool(value)
callAs: function
arg value: value to be converted
argType value: any
returnType: boolean

yaql> bool(1)
true
yaql> bool([])
false

isBoolean

Returns true if value is boolean, otherwise false.

signature: isBoolean(value)
callAs: function
arg value: value to check
argType value: any
returnType: boolean

yaql> isBoolean(false)
true
yaql> isBoolean(0)
false

operator and

Returns left operand if it evaluates to false. Otherwise evaluates right operand and returns it.

signature: left and right
callAs: operator
arg left: left operand
argType left: any
arg right: right operand
argType right: any
returnType: any (left or right operand types)

yaql> 1 and 0
0
yaql> 1 and 2
2
yaql> [] and 1
[]

operator not

Returns true if arg evaluates to false. Otherwise returns false.

signature: not arg
callAs: operator
arg arg: value to be converted
argType arg: any
returnType: boolean

yaql> not true
false
yaql> not {}
true
yaql> not [1]
false

operator or

Returns left operand if it evaluates to true. Otherwise evaluates right operand and returns it.

signature: left or right
callAs: operator
arg left: left operand
argType left: any
arg right: right operand
argType right: any
returnType: any (left or right operand types)

yaql> 1 or 0
1
yaql> 1 or 2
1
yaql> [] or 1
1

Working with collections

Functions that produce or consume finite collections - lists, dicts and sets.

add

Returns a new set with added args.

signature: set.add([args])
callAs: method
receiverArg set: input set
argType set: set
arg [args]: values to be added to set
argType [args]: chain of any type
returnType: set

yaql> set(0, 1).add("", [1, 2, 3])
[0, 1, "", [1, 2, 3]]

contains

Returns true if value is contained in collection, false otherwise.

signature: collection.contains(value)
callAs: method
receiverArg collection: collection to find occurrence in
argType collection: iterable
arg value: value to be checked for occurrence
argType value: any
returnType: boolean

yaql> ["a", "b"].contains("a")
true

containsKey

Returns true if the dictionary contains the key, false otherwise.

signature: dict.containsKey(key)
callAs: method
receiverArg dict: dictionary to find occurrence in
argType dict: mapping
arg key: value to be checked for occurrence
argType key: any
returnType: boolean

yaql> {"a" => 1, "b" => 2}.containsKey("a")
true

containsValue

Returns true if the dictionary contains the value, false otherwise.

signature: dict.containsValue(value)
callAs: method
receiverArg dict: dictionary to find occurrence in
argType dict: mapping
arg value: value to be checked for occurrence
argType value: any
returnType: boolean

yaql> {"a" => 1, "b" => 2}.containsValue("a")
false
yaql> {"a" => 1, "b" => 2}.containsValue(2)
true

delete

•: Returns collection with removed [position, position+count) elements.

signature: collection.delete(position, count => 1)
callAs: method
receiverArg collection: input collection
argType collection: iterable
arg position: index to start remove
argType position: integer
arg count: how many elements to remove, 1 by default
argType position: integer
returnType: iterable

yaql> [0, 1, 3, 4, 2].delete(2, 2)
[0, 1, 2]

•: Returns dict with keys removed.

signature: dict.delete([args])
callAs: method
receiverArg dict: input dictionary
argType dict: mapping
arg [args]: keys to be removed from dictionary
argType [args]: chain of keywords
returnType: mapping

yaql> {"a" => 1, "b" => 2, "c" => 3}.delete("a", "c")
{"b": 2}

deleteAll

Returns dict with keys removed. Keys are provided as an iterable collection.

signature: dict.deleteAll(keys)
callAs: method
receiverArg dict: input dictionary
argType dict: mapping
arg keys: keys to be removed from dictionary
argType keys: iterable
returnType: mapping

yaql> {"a" => 1, "b" => 2, "c" => 3}.deleteAll(["a", "c"])
{"b": 2}

dict

•: Returns dictionary of provided keyword values.

signature: dict([args])
callAs: function
arg [args]: arguments to create a dictionary
argType [args]: mappings
returnType: dictionary

yaql> dict(a => 1, b => 2)
{ "a": 1, "b": 2}

•: Returns dictionary with keys and values built on items pairs.

signature: dict(items)
callAs: function
arg items: list of pairs [key, value] for building dictionary
argType items: list
returnType: dictionary

yaql> dict([["a", 2], ["b", 4]])
{"a": 2, "b": 4}

•: Returns dict built on collection where keys are keySelector applied to collection elements and values are valueSelector applied to collection elements.

signature: collection.toDict(keySelector, valueSelector => null)
callAs: method
receiverArg collection: collection to build dict from
argType collection: iterable
arg keySelector: lambda function to get keys from collection elements
argType keySelector: lambda
arg valueSelector: lambda function to get values from collection elements. null by default, which means values to be collection items
argType valueSelector: lambda
returnType: dictionary

yaql> [1, 2].toDict($, $ + 1)
{"1": 2, "2": 3}

difference

Return the difference of left and right sets as a new set.

signature: left.difference(right)
callAs: method
receiverArg left: left set
argType left: set
arg right: right set
argType right: set
returnType: set

yaql> set(0, 1, 2).difference(set(0, 1))
[2]

flatten

Returns an iterator to the recursive traversal of collection.

signature: collection.flatten()
callAs: method
receiverArg collection: collection to be traversed
argType collection: iterable
returnType: list

yaql> ["a", ["b", [2,3]]].flatten()
["a", "b", 2, 3]

get

Returns value of a dictionary by given key or default if there is no such key.

signature: dict.get(key, default => null)
callAs: method
receiverArg dict: input dictionary
argType dict: dictionary
arg key: key
argType key: keyword
arg default: default value to be returned if key is missing in dictionary. null by default
argType default: any
returnType: any (appropriate value type)

yaql> {"a" => 1, "b" => 2}.get("c")
null
yaql> {"a" => 1, "b" => 2}.get("c", 3)
3

insert

•: Returns collection with inserted value at the given position.

signature: collection.insert(position, value)
callAs: method
receiverArg collection: input collection
argType collection: iterable
arg position: index for insertion. value is inserted in the end if position greater than collection size
argType position: integer
arg value: value to be inserted
argType value: any
returnType: iterable

yaql> [0, 1, 3].insert(2, 2)
[0, 1, 2, 3]

•: Returns collection with inserted value at the given position.

signature: collection.insert(position, value)
callAs: method
receiverArg collection: input collection
argType collection: sequence
arg position: index for insertion. value is inserted in the end if position greater than collection size
argType position: integer
arg value: value to be inserted
argType value: any
returnType: sequence

yaql> [0, 1, 3].insert(2, 2)
[0, 1, 2, 3]

insertMany

Returns collection with inserted values at the given position.

signature: collection.insertMany(position, values)
callAs: method
receiverArg collection: input collection
argType collection: iterable
arg position: index for insertion. value is inserted in the end if position greater than collection size
argType position: integer
arg values: items to be inserted
argType values: iterable
returnType: iterable

yaql> [0, 1, 3].insertMany(2, [2, 22])
[0, 1, 2, 22, 3]

intersect

Returns set with elements common to left and right sets.

signature: left.intersect(right)
callAs: method
receiverArg left: left set
argType left: set
arg right: right set
argType right: set
returnType: set

yaql> set(0, 1, 2).intersect(set(0, 1))
[0, 1]

isDict

Returns true if arg is dictionary, false otherwise.

signature: isDict(arg)
callAs: function
arg arg: value to be checked
argType arg: any
returnType: boolean

yaql> isDict([1, 2])
false
yaql> isDict({"a" => 1})
true

isList

Returns true if arg is a list, false otherwise.

signature: isList(arg)
callAs: function
arg arg: value to be checked
argType arg: any
returnType: boolean

yaql> isList([1, 2])
true
yaql> isList({"a" => 1})
false

isSet

Returns true if arg is set, false otherwise.

signature: isSet(arg)
callAs: function
arg arg: value to be checked
argType arg: any
returnType: boolean

yaql> isSet({"a" => 1})
false
yaql> isSet(set(1, 2))
true

items

Returns an iterator over pairs [key, value] of input dict.

signature: dict.items()
callAs: method
receiverArg dict: input dictionary
argType dict: dictionary
returnType: iterator

yaql> {"a" => 1, "b" => 2}.items()
[["a", 1], ["b", 2]]

keys

Returns an iterator over the dictionary keys.

signature: dict.keys()
callAs: method
receiverArg dict: input dictionary
argType dict: dictionary
returnType: iterator

yaql> {"a" => 1, "b" => 2}.keys()
["a", "b"]

len

•: Returns size of the dictionary.

signature: dict.len()
callAs: function or method
receiverArg dict: input dictionary
argType dict: mapping
returnType: integer

yaql> {"a" => 1, "b" => 2}.len()
2

•: Returns length of the list.

signature: sequence.len()
callAs: function or method
receiverArg sequence: input sequence
argType dict: sequence
returnType: integer

yaql> [0, 1, 2].len()
3

•: Returns size of the set.

signature: set.len()
callAs: function or method
receiverArg set: input set
argType set: set
returnType: integer

yaql> set(0, 1, 2).len()
3

list

•: Returns list of provided args.

signature: list([args])
callAs: function
arg [args]: arguments to create a list
argType [args]: any types
returnType: list

yaql> list(1, "", 2)
[1, "", 2]

•: Returns list of provided args and unpacks arg element if it's iterable.

signature: list([args])
callAs: function
arg [args]: arguments to create a list
argType [args]: chain of any types
returnType: list

yaql> list(1, "", range(2))
[1, "", 0, 1]

operator *

•: Returns sequence repeated count times.

signature: left * right
callAs: operator
arg left: multiplier
argType left: integer
arg right: input sequence
argType right: sequence
returnType: sequence

yaql> 2 * [1, 2]
[1, 2, 1, 2]

•: Returns sequence repeated count times.

signature: left * right
callAs: operator
arg left: input sequence
argType left: sequence
arg right: multiplier
argType right: integer
returnType: sequence

yaql> [1, 2] * 2
[1, 2, 1, 2]

operator +

•: Returns combined left and right dictionaries.

signature: left + right
callAs: operator
arg left: left dictionary
argType left: mapping
arg right: right dictionary
argType right: mapping
returnType: mapping

yaql> {"a" => 1, b => 2} + {"b" => 3, "c" => 4}
{"a": 1, "c": 4, "b": 3}

•: Returns two iterables concatenated.

signature: left + right
callAs: operator
arg left: left list
argType left: iterable
arg right: right list
argType right: iterable
returnType: iterable

yaql> [1, 2] + [3]
[1, 2, 3]

operator .

Returns value of a dictionary by given key.

signature: left.right
callAs: operator
arg left: input dictionary
argType left: dictionary
arg right: key
argType right: keyword
returnType: any (appropriate value type)

yaql> {"a" => 1, "b" => 2}.a
1

operator <

Returns true if left set is subset of right set and left size is strictly less than right size, false otherwise.

signature: left < right
callAs: operator
arg left: left set
argType left: set
arg right: right set
argType right: set
returnType: boolean

yaql> set(0) < set(0, 1)
true

operator <=

Returns true if left set is subset of right set.

signature: left <= right
callAs: operator
arg left: left set
argType left: set
arg right: right set
argType right: set
returnType: boolean

yaql> set(0, 1) <= set(0, 1)
true

operator >

Returns true if right set is subset of left set and left size is strictly greater than right size, false otherwise.

signature: left > right
callAs: operator
arg left: left set
argType left: set
arg right: right set
argType right: set
returnType: boolean

yaql> set(0, 1) > set(0, 1)
false

operator >=

Returns true if right set is subset of left set.

signature: left >= right
callAs: operator
arg left: left set
argType left: set
arg right: right set
argType right: set
returnType: boolean

yaql> set(0, 1) >= set(0, 1)
true

operator in

Returns true if there is at least one occurrence of value in collection, false otherwise.

signature: left in right
callAs: operator
arg left: value to be checked for occurrence
argType left: any
arg right: collection to find occurrence in
argType right: iterable
returnType: boolean

yaql> "a" in ["a", "b"]
true

operator indexer

•: Returns value of a dictionary by given key.

signature: left[right]
callAs: function
arg left: input dictionary
argType left: dictionary
arg right: key
argType right: keyword
returnType: any (appropriate value type)

yaql> {"a" => 1, "b" => 2}["a"]
1

•: Returns value of a dictionary by given key or default if there is no such key.

signature: left[right, default]
callAs: function
arg left: input dictionary
argType left: dictionary
arg right: key
argType right: keyword
arg default: default value to be returned if key is missing in dictionary
argType default: any
returnType: any (appropriate value type)

yaql> {"a" => 1, "b" => 2}["c", 3]
3

•: Returns value of sequence by given index.

signature: left[right]
callAs: function
arg left: input sequence
argType left: sequence
arg right: index
argType right: integer
returnType: any (appropriate value type)

yaql> ["a", "b"][0]
"a"

remove

Returns the set with excluded values provided in arguments.

signature: set.remove([args])
callAs: method
receiverArg set: input set
argType set: set
arg [args]: values to be removed from set
argType [args]: chain of any type
returnType: set

yaql> set(0, 1, "", [1, 2, 3]).remove("", 0, [1, 2, 3])
[1]

replace

Returns collection where [position, position+count) elements are replaced with value.

signature: collection.replace(position, value, count => 1)
callAs: method
receiverArg collection: input collection
argType collection: iterable
arg position: index to start replace
argType position: integer
arg value: value to be replaced with
argType value: any
arg count: how many elements to replace, 1 by default
argType count: integer
returnType: iterable

yaql> [0, 1, 3, 4, 2].replace(2, 100, 2)
[0, 1, 100, 2]

replaceMany

Returns collection where [position, position+count) elements are replaced with values items.

signature: collection.replaceMany(position, values, count => 1)
callAs: method
receiverArg collection: input collection
argType collection: iterable
arg position: index to start replace
argType position: integer
arg values: items to replace
argType values: iterable
arg count: how many elements to replace, 1 by default
argType count: integer
returnType: iterable

yaql> [0, 1, 3, 4, 2].replaceMany(2, [100, 200], 2)
[0, 1, 100, 200, 2]

set

•: Returns dict with provided key set to value.

signature: dict.set(key, value)
callAs: method
receiverArg dict: input dictionary
argType dict: dictionary
arg key: key
argType key: keyword
arg value: value to be set to input key
argType value: any
returnType: dictionary

yaql> {"a" => 1, "b" => 2}.set("c", 3)
{"a": 1, "b": 2, "c": 3}
yaql> {"a" => 1, "b" => 2}.set("b", 3)
{"a": 1, "b": 3}

•: Returns dict with replacements keys set to replacements values.

signature: dict.set(replacements)
callAs: method
receiverArg dict: input dictionary
argType dict: dictionary
arg replacements: mapping with keys and values to be set on input dict
argType key: dictionary
returnType: dictionary

yaql> {"a" => 1, "b" => 2}.set({"b" => 3, "c" => 4})
{"a": 1, "c": 4, "b": 3}

•: Returns dict with args keys set to args values.

signature: dict.set([args])
callAs: method
receiverArg dict: input dictionary
argType dict: dictionary
arg [args]: key-values to be set on input dict
argType [args]: chain of mappings
returnType: dictionary

yaql> {"a" => 1, "b" => 2}.set("b" => 3, "c" => 4)
{"a": 1, "c": 4, "b": 3}

•: Returns set initialized with args.

signature: set([args])
callAs: function
arg [args]: args to build a set
argType [args]: chain of any type
returnType: set

yaql> set(0, "", [1, 2])
[0, "", [1, 2]]

symmetricDifference

Returns symmetric difference of left and right sets as a new set.

signature: left.symmetricDifference(right)
callAs: method
receiverArg left: left set
argType left: set
arg right: right set
argType right: set
returnType: set

yaql> set(0, 1, 2).symmetricDifference(set(0, 1, 3))
[2, 3]

toList

Returns list built from iterable.

signature: collection.toList()
callAs: method
receiverArg collection: collection to be transferred to list
argType collection: iterable
returnType: list

yaql> range(3).toList()
[0, 1, 2]

toSet

Returns set built from iterable.

signature: collection.toSet()
callAs: method
receiverArg collection: collection to build a set
argType collection: iterable
returnType: set

yaql> [0, 1, 1, 2].toSet()
[0, 1, 2]

union

Returns union of two sets.

signature: left.union(right)
callAs: method
receiverArg left: input set
argType left: set
arg right: input set
argType right: set
returnType: set

yaql> set(0, 1).union(set(1, 2))
[0, 1, 2]

values

Returns an iterator over the dictionary values.

signature: dict.values()
callAs: method
receiverArg dict: input dictionary
argType dict: dictionary
returnType: iterator

yaql> {"a" => 1, "b" => 2}.values()
[1, 2]

Querying data

Queries module.

accumulate

Applies selector of two arguments cumulatively to the items of collection from begin to end, so as to accumulate the collection to a list of intermediate values.

signature: collection.accumulate(selector, seed => NoValue)
callAs: method
receiverArg collection: input collection
argType collection: iterable
arg selector: function of two arguments to be applied on every next pair of collection
argType selector: lambda
arg seed: value to use as the first for accumulating. noValue by default
argType seed: collection elements type
returnType: list

yaql> [1, 2, 3].accumulate($1+$2)
[1, 3, 6]
yaql> [1, 2, 3].accumulate($1+$2, 100)
[100, 101, 103, 106]
yaql> [].accumulate($1+$2,1)
[1]

aggregate

Applies selector of two arguments cumulatively: to the first two elements of collection, then to the result of the previous selector applying and to the third element, and so on. Returns the result of last selector applying.

signature: collection.aggregate(selector, seed => NoValue)
callAs: method
receiverArg collection: input collection
argType collection: iterable
arg selector: function of two arguments to be applied on every next pair of collection
argType selector: lambda
arg seed: if specified, it is used as start value for accumulating and becomes a default when the collection is empty. NoValue by default
argType seed: collection elements type
returnType: collection elements type

yaql> [a,a,b,a,a].aggregate($1 + $2)
"aabaa"
yaql> [].aggregate($1 + $2, 1)
1

all

Returns true if all the elements of a collection evaluate to true. If a predicate is specified, returns true if the predicate is true for all elements in the collection.

signature: collection.all(predicate => null)
callAs: function or method
receiverArg collection: input collection
argType collection: iterable
arg predicate: lambda function to apply to every collection value. null by default, which means evaluating collections elements to boolean with no predicate
argType predicate: lambda
returnType: boolean

yaql> [1, [], ''].all()
false
yaql> [1, [0], 'a'].all()
true

any

Returns true if a collection is not empty. If a predicate is specified, determines whether any element of the collection satisfies the predicate.

signature: collection.any(predicate => null)
callAs: function or method
receiverArg collection: input collection
argType collection: iterable
arg predicate: lambda function to apply to every collection value. null by default, which means checking collection length
argType predicate: lambda
returnType: boolean

yaql> [[], 0, ''].any()
true
yaql> [[], 0, ''].any(predicate => $)
false

append

Returns a collection with appended args.

signature: collection.append([args])
callAs: function or method
receiverArg collection: input collection
argType collection: iterable
arg [args]: arguments to be appended to input collection
argType [args]: chain of any types
returnType: iterable

yaql> [1, 2, 3].append(4, 5)
[1, 2, 3, 4, 5]

concat

Returns an iterator that consequently iterates over elements of the first collection, then proceeds to the next collection and so on.

signature: collection.concat([args])
callAs: function or method
receiverArg collection: input collection
argType collection: iterable
arg [args]: iterables to be concatenated with input collection
argType [args]: chain of iterable
returnType: iterable

yaql> [1].concat([2, 3], [4, 5])
[1, 2, 3, 4, 5]

count

Returns the size of the collection.

signature: collection.count()
callAs: method
receiverArg collection: input collection
argType collection: iterable
returnType: integer

yaql> [1, 2].count()
2

cycle

Makes an iterator returning elements from the collection as if it cycled.

signature: collection.cycle()
callAs: method
receiverArg collection: value to be cycled
argType collection: iterable
returnType: iterator

yaql> [1, 2].cycle().take(5)
[1, 2, 1, 2, 1]

defaultIfEmpty

Returns default value if collection is empty.

signature: collection.defaultIfEmpty(default)
callAs: method
receiverArg collection: input collection
argType collection: iterable
arg default: value to be returned if collection size is 0
argType default: iterable
returnType: iterable

yaql> [].defaultIfEmpty([1, 2])
[1, 2]

distinct

Returns only unique members of the collection. If keySelector is specified, it is used to determine uniqueness.

signature: collection.distinct(keySelector => null)
callAs: function or method
receiverArg collection: input collection
argType collection: iterable
arg keySelector: specifies a function of one argument that is used to extract a comparison key from each collection element. The default value is null, which means elements are compared directly
argType keySelector: lambda
returnType: iterable

yaql> [1, 2, 3, 1].distinct()
[1, 2, 3]
yaql> [{'a'=> 1}, {'b'=> 2}, {'a'=> 1}].distinct()
[{"a": 1}, {"b": 2}]
yaql> [['a', 1], ['b', 2], ['c', 1], ['a', 3]].distinct($[1])
[['a', 1], ['b', 2], ['a', 3]]

enumerate

Returns an iterator over pairs (index, value), obtained from iterating over a collection.

signature: collection.enumerate(start => 0)
callAs: function or method
receiverArg collection: input collection
argType collection: iterable
arg start: a value to start with numerating first element of each pair, 0 is a default value
argType start: integer
returnType: list

yaql> ['a', 'b', 'c'].enumerate()
[[0, 'a'], [1, 'b'], [2, 'c']]
yaql> ['a', 'b', 'c'].enumerate(2)
[[2, 'a'], [3, 'b'], [4, 'c']]

first

Returns the first element of the collection. If the collection is empty, returns the default value or raises StopIteration if default is not specified.

signature: collection.first(default => NoValue)
callAs: method
receiverArg collection: input collection
argType collection: iterable
arg default: value to be returned if collection is empty. NoValue by default
argType default: any
returnType: type of collection's elements or default value type

yaql> [3, 1, 2].first()
3

generate

Returns iterator to values beginning from initial value with every next value produced with producer applied to every previous value, while predicate is true. Represents traversal over the list where each next element is obtained by the lambda result from the previous element.

signature: generate(initial, predicate, producer, selector => null, decycle => false)
callAs: function
arg initial: value to start from
argType initial: any type
arg predicate: function of one argument to be applied on every new value. Stops generating if return value is false
argType predicate: lambda
arg producer: function of one argument to produce the next value
argType producer: lambda
arg selector: function of one argument to store every element in the resulted list. none by default which means to store producer result
argType selector: lambda
arg decycle: return only distinct values if true, false by default
argType decycle: boolean
returnType: list

yaql> generate(0, $ < 10, $ + 2)
[0, 2, 4, 6, 8]
yaql> generate(1, $ < 10, $ + 2, $ * 1000)
[1000, 3000, 5000, 7000, 9000]

generateMany

Returns iterator to values beginning from initial queue of values with every next value produced with producer applied to top of queue, while predicate is true. Represents tree traversal, where producer is used to get child nodes.

signature: generateMany(initial, producer, selector => null, decycle => false, depthFirst => false)
callAs: function
arg initial: value to start from
argType initial: any type
arg producer: function to produce the next value for queue
argType producer: lambda
arg selector: function of one argument to store every element in the resulted list. none by default which means to store producer result
argType selector: lambda
arg decycle: return only distinct values if true, false by default
argType decycle: boolean
arg depthFirst: if true puts produced elements to the start of queue, false by default
argType depthFirst: boolean
returnType: list

yaql> generateMany("1", {"1" => ["2", "3"],


                         "2"=>["4"], "3"=>["5"]


                         }.get($, []))
["1", "2", "3", "4", "5"]

indexOf

Returns the index in the collection of the first item which value is item. -1 is a return value if there is no such item

signature: collection.indexOf(item)
callAs: method
receiverArg collection: input collection
argType collection: iterable
arg item: value to find in collection
argType item: any
returnType: integer

yaql> [1, 2, 3, 2].indexOf(2)
1
yaql> [1, 2, 3, 2].indexOf(102)
-1

indexWhere

Returns the index in the collection of the first item which value satisfies the predicate. -1 is a return value if there is no such item

signature: collection.indexWhere(predicate)
callAs: method
receiverArg collection: input collection
argType collection: iterable
arg predicate: function of one argument to apply on every value
argType predicate: lambda
returnType: integer

yaql> [1, 2, 3, 2].indexWhere($ > 2)
2
yaql> [1, 2, 3, 2].indexWhere($ > 3)
-1

isIterable

Returns true if value is iterable, false otherwise.

signature: isIterable(value)
callAs: function
arg value: value to be checked
argType value: any
returnType: boolean

yaql> isIterable([])
true
yaql> isIterable(set(1,2))
true
yaql> isIterable("foo")
false
yaql> isIterable({"a" => 1})
false

join

Returns list of selector applied to those combinations of collection1 and collection2 elements, for which predicate is true.

signature: collection1.join(collection2, predicate, selector)
callAs: method
receiverArg collection1: input collection
argType collection1: iterable
arg collection2: other input collection
argType collection2: iterable
arg predicate: function of two arguments to apply to every (collection1, collection2) pair, if returned value is true the pair is passed to selector
argType predicate: lambda
arg selector: function of two arguments to apply to every (collection1, collection2) pair, for which predicate returned true
argType selector: lambda
returnType: iterable

yaql> [1,2,3,4].join([2,5,6], $1 > $2, [$1, $2])
[[3, 2], [4, 2]]

last

Returns the last element of the collection. If the collection is empty, returns the default value or raises StopIteration if default is not specified.

signature: collection.last(default => NoValue)
callAs: method
receiverArg collection: input collection
argType collection: iterable
arg default: value to be returned if collection is empty. NoValue is default value.
argType default: any
returnType: type of collection's elements or default value type

yaql> [0, 1, 2].last()
2

lastIndexOf

Returns the index in the collection of the last item which value is item. -1 is a return value if there is no such item

signature: collection.lastIndexOf(item)
callAs: method
receiverArg collection: input collection
argType collection: iterable
arg item: value to find in collection
argType item: any
returnType: integer

yaql> [1, 2, 3, 2].lastIndexOf(2)
3

lastIndexWhere

Returns the index in the collection of the last item which value satisfies the predicate. -1 is a return value if there is no such item

signature: collection.lastIndexWhere(predicate)
callAs: method
receiverArg collection: input collection
argType collection: iterable
arg predicate: function of one argument to apply on every value
argType predicate: lambda
returnType: integer

yaql> [1, 2, 3, 2].lastIndexWhere($ = 2)
3

len

Returns the size of the collection.

signature: collection.len()
callAs: function or method
receiverArg collection: input collection
argType collection: iterable
returnType: integer

yaql> [1, 2].len()
2

limit

Returns the first count elements of a collection.

signature: collection.limit(count)
callAs: method
receiverArg collection: input collection
argType collection: iterable
arg count: how many first elements of a collection to return. If count is greater or equal to collection size, return value is input collection
argType count: integer
returnType: iterable

yaql> [1, 2, 3, 4, 5].limit(4)
[1, 2, 3, 4]

max

Returns max value in collection. Considers initial if specified.

signature: collection.max(initial => NoValue)
callAs: method
receiverArg collection: input collection
argType collection: iterable
arg initial: value to start with. NoValue by default
argType initial: collection's elements type
returnType: collection's elements type

yaql> [3, 1, 2].max()
3

memorize

Returns an iterator over collection and memorizes already iterated values. This function can be used for iterating over collection several times as it remembers elements, and when given collection (iterator) is too large to be unwrapped at once.

signature: collection.memorize()
callAs: method
receiverArg collection: input collection
argType collection: iterable
returnType: iterator to collection

yaql> let(range(4)) -> $.sum() + $.len()
6
yaql> let(range(4).memorize()) -> $.sum() + $.len()
10

mergeWith

Performs a deep merge of two dictionaries.

signature: dict.mergeWith(another, listMerger => null, itemMerger => null, maxLevels => null)
callAs: method
receiverArg dict: input dictionary
argType dict: mapping
arg another: dictionary to merge with
argType another: mapping
arg listMerger: function to be applied while merging two lists. null is a default which means listMerger to be distinct(lst1 + lst2)
argType listMerger: lambda
arg itemMerger: function to be applied while merging two items. null is a default, which means itemMerger to be a second item for every pair.
argType itemMerger: lambda
arg maxLevels: number which describes how deeply merge dicts. 0 by default, which means going throughout them
argType maxLevels: int
returnType: mapping

yaql> {'a'=> 1, 'b'=> 2, 'c'=> [1, 2]}.mergeWith({'d'=> 5, 'b'=> 3,


                                                  'c'=> [2, 3]})
{"a": 1, "c": [1, 2, 3], "b": 3, "d": 5}
yaql> {'a'=> 1, 'b'=> 2, 'c'=> [1, 2]}.mergeWith({'d'=> 5, 'b'=> 3,


                                                  'c'=> [2, 3]},


                                                  $1+$2)
{"a": 1, "c": [1, 2, 2, 3], "b": 3, "d": 5}
yaql> {'a'=> 1, 'b'=> 2, 'c'=> [1, 2]}.mergeWith({'d'=> 5, 'b'=> 3,


                                                  'c'=> [2, 3]},


                                                  $1+$2, $1)
{"a": 1, "c": [1, 2, 2, 3], "b": 2, "d": 5}
yaql> {'a'=> 1, 'b'=> 2, 'c'=> [1, 2]}.mergeWith({'d'=> 5, 'b'=> 3,


                                                  'c'=> [2, 3]},


                                                  maxLevels => 1)
{"a": 1, "c": [2, 3], "b": 3, "d": 5}

min

Returns min value in collection. Considers initial if specified.

signature: collection.min(initial => NoValue)
callAs: method
receiverArg collection: input collection
argType collection: iterable
arg initial: value to start with. NoValue by default
argType initial: collection's elements type
returnType: collection's elements type

yaql> [3, 1, 2].min()
1

operator .

Retrieves the value of an attribute for each element in a collection and returns a list of results.

signature: collection.attribute
callAs: operator
arg collection: input collection
argType collection: iterable
arg attribute: attribute to get on every collection item
argType attribute: keyword
returnType: list

yaql> [{"a" => 1}, {"a" => 2, "b" => 3}].a
[1, 2]

orderBy

Returns an iterator over collection elements sorted in ascending order. Selector is applied to each element of the collection to extract sorting key.

signature: collection.orderBy(selector)
callAs: method
receiverArg collection: collection to be ordered
argType collection: iterable
arg selector: specifies a function of one argument that is used to extract a comparison key from each element
argType selector: lambda
returnType: iterator

yaql> [[1, 'c'], [2, 'b'], [3, 'c'], [0, 'd']].orderBy($[1])
[[2, 'b'], [1, 'c'], [3, 'c'], [0, 'd']]

orderByDescending

Returns an iterator over collection elements sorted in descending order. Selector is applied to each element of the collection to extract sorting key.

signature: collection.orderByDescending(selector)
callAs: method
receiverArg collection: collection to be ordered
argType collection: iterable
arg selector: specifies a function of one argument that is used to extract a comparison key from each element
argType selector: lambda
returnType: iterator

yaql> [4, 2, 3, 1].orderByDescending($)
[4, 3, 2, 1]

range

•: Returns an iterator over values from 0 up to stop, not including stop, i.e. [0, stop).

signature: range(stop)
callAs: function
arg stop: right bound for generated list numbers
argType stop: integer
returnType: iterator

yaql> range(3)
[0, 1, 2]

•: Returns an iterator over values from start up to stop, not including stop, i.e [start, stop) with step 1 if not specified.

signature: range(start, stop, step => 1)
callAs: function
arg start: left bound for generated list numbers
argType start: integer
arg stop: right bound for generated list numbers
argType stop: integer
arg step: the next element in list is equal to previous + step. 1 is value by default
argType step: integer
returnType: iterator

yaql> range(1, 4)
[1, 2, 3]
yaql> range(4, 1, -1)
[4, 3, 2]

repeat

Returns collection with value repeated.

signature: value.repeat(times => -1)
callAs: method
receiverArg value: value to be repeated
argType value: any
arg times: how many times repeat value. -1 by default, which means that returned value will be an iterator to the endless sequence of values
argType times: int
returnType: iterable

yaql> 1.repeat(2)
[1, 1]
yaql> 1.repeat().take(3)
[1, 1, 1]

reverse

Returns reversed collection, evaluated to list.

signature: collection.reverse()
callAs: method
receiverArg collection: input collection
argType collection: iterable
returnType: list

yaql> [1, 2, 3, 4].reverse()
[4, 3, 2, 1]

select

Applies the selector to every item of the collection and returns a list of results.

signature: collection.select(selector)
callAs: method
receiverArg collection: input collection
argType collection: iterable
arg selector: expression for processing elements
argType selector: lambda
returnType: iterable

yaql> [1, 2, 3, 4, 5].select($ * $)
[1, 4, 9, 16, 25]
yaql> [{'a'=> 2}, {'a'=> 4}].select($.a)
[2, 4]

selectMany

Applies a selector to each element of the collection and returns an iterator over results. If the selector returns an iterable object, iterates over its elements instead of itself.

signature: collection.selectMany(selector)
callAs: method
receiverArg collection: input collection
argType collection: iterable
arg selector: function to be applied to every collection element
argType selector: lambda
returnType: iterator

yaql> [0, 1, 2].selectMany($ + 2)
[2, 3, 4]
yaql> [0, [1, 2], 3].selectMany($ * 2)
[0, 1, 2, 1, 2, 6]

sequence

Returns an iterator to the sequence beginning from start with step.

signature: sequence(start => 0, step => 1)
callAs: function
arg start: start value of the sequence. 0 is value by default
argType start: integer
arg step: the next element is equal to previous + step. 1 is value by default
argType step: integer
returnType: iterator

yaql> sequence().take(5)
[0, 1, 2, 3, 4]

single

Checks that collection has only one element and returns it. If the collection is empty or has more than one element, raises StopIteration.

signature: collection.single()
callAs: method
receiverArg collection: input collection
argType collection: iterable
returnType: type of collection's elements

yaql> ["abc"].single()
"abc"
yaql> [1, 2].single()
Execution exception: Collection contains more than one item

skip

Returns a collection without first count elements.

signature: collection.skip(count)
callAs: method
receiverArg collection: input collection
argType collection: iterable
arg count: how many elements to skip. If count is greater or equal to collection size, return value is empty list
argType count: integer
returnType: iterable

yaql> [1, 2, 3, 4, 5].skip(2)
[3, 4, 5]

skipWhile

Skips elements from the collection as long as the predicate is true. Then returns an iterator to collection of remaining elements

signature: collection.skipWhile(predicate)
callAs: method
receiverArg collection: input collection
argType collection: iterable
arg predicate: function of one argument to apply to every collection value
argType predicate: lambda
returnType: iterator

yaql> [1, 2, 3, 4, 5].skipWhile($ < 3)
[3, 4, 5]

slice

Returns collection divided into list of collections with max size of new parts equal to length.

signature: collection.slice(length)
callAs: method
receiverArg collection: input collection
argType collection: iterable
arg length: max length of new collections
argType length: integer
returnType: list

yaql> range(1,6).slice(2)
[[1, 2], [3, 4], [5]]

sliceWhere

Splits collection into lists. Within every list predicate evaluated on its items returns the same value while predicate evaluated on the items of the adjacent lists returns different values. Returns an iterator to lists.

signature: collection.sliceWhere(predicate)
callAs: method
receiverArg collection: input collection
argType collection: iterable
arg predicate: function of one argument to be applied on every element. Elements for which predicate returns true are delimiters for new list and are present in new collection as separate collections
argType predicate: lambda
returnType: iterator

yaql> [1, 2, 3, 4, 5, 6, 7].sliceWhere($ mod 3 = 0)
[[1, 2], [3], [4, 5], [6], [7]]

splitAt

Splits collection into two lists by index.

signature: collection.splitAt(index)
callAs: method
receiverArg collection: input collection
argType collection: iterable
arg index: the index of collection to be delimiter for splitting
argType index: integer
returnType: list

yaql> [1, 2, 3, 4].splitAt(1)
[[1], [2, 3, 4]]
yaql> [1, 2, 3, 4].splitAt(0)
[[], [1, 2, 3, 4]]

splitWhere

Returns collection divided into list of collections where delimiters are values for which predicate returns true. Delimiters are deleted from result.

signature: collection.splitWhere(predicate)
callAs: method
receiverArg collection: input collection
argType collection: iterable
arg predicate: function of one argument to be applied on every element. Elements for which predicate returns true are delimiters for new list
argType predicate: lambda
returnType: list

yaql> [1, 2, 3, 4, 5, 6, 7].splitWhere($ mod 3 = 0)
[[1, 2], [4, 5], [7]]

sum

Returns the sum of values in a collection starting from initial if specified.

signature: collection.sum(initial => NoValue)
callAs: method
receiverArg collection: input collection
argType collection: iterable
arg initial: value to start sum with. NoValue by default
argType initial: collection's elements type
returnType: collection's elements type

yaql> [3, 1, 2].sum()
6
yaql> ['a', 'b'].sum('c')
"cab"

takeWhile

Returns elements from the collection as long as the predicate is true.

signature: collection.takeWhile(predicate)
callAs: method
receiverArg collection: input collection
argType collection: iterable
arg predicate: function of one argument to apply to every collection value
argType predicate: lambda
returnType: iterable

yaql> [1, 2, 3, 4, 5].takeWhile($ < 4)
[1, 2, 3]

thenBy

To be used with orderBy or orderByDescending. Uses selector to extract secondary sort key (ascending) from the elements of the collection and adds it to the iterator.

signature: collection.thenBy(selector)
callAs: method
receiverArg collection: collection to be ordered
argType collection: iterable
arg selector: specifies a function of one argument that is used to extract a comparison key from each element
argType selector: lambda
returnType: iterator

yaql> [[3, 'c'], [2, 'b'], [1, 'c']].orderBy($[1]).thenBy($[0])
[[2, 'b'], [1, 'c'], [3, 'c']]

thenByDescending

To be used with orderBy or orderByDescending. Uses selector to extract secondary sort key (descending) from the elements of the collection and adds it to the iterator.

signature: collection.thenByDescending(selector)
callAs: method
receiverArg collection: collection to be ordered
argType collection: iterable
arg selector: specifies a function of one argument that is used to extract a comparison key from each element
argType selector: lambda
returnType: iterable

yaql> [[3,'c'], [2,'b'], [1,'c']].orderBy($[1]).thenByDescending($[0])
[[2, 'b'], [3, 'c'], [1, 'c']]

where

Returns only those collection elements, for which the filtering query (predicate) is true.

signature: collection.where(predicate)
callAs: method
receiverArg collection: collection to be filtered
argType collection: iterable
arg predicate: filter for collection elements
argType predicate: lambda
returnType: iterable

yaql> [1, 2, 3, 4, 5].where($ > 3)
[4, 5]

zip

Returns an iterator over collections, where the n-th iterable contains the n-th element from each of collections. Stops iterating as soon as any of the collections is exhausted.

signature: collection.zip([args])
callAs: method
receiverArg collection: input collection
argType collection: iterable
arg [args]: collections for zipping with input collection
argType [args]: chain of collections
returnType: iterator

yaql> [1, 2, 3].zip([4, 5], [6, 7])
[[1, 4, 6], [2, 5, 7]]

zipLongest

Returns an iterator over collections, where the n-th iterable contains the n-th element from each of collections. Iterates until all the collections are not exhausted and fills lacking values with default value, which is null by default.

signature: collection.zipLongest([args], default => null)
callAs: method
receiverArg collection: input collection
argType collection: iterable
arg [args]: collections for zipping with input collection
argType [args]: chain of collections
arg default: default value for lacking values, can be passed only as keyword argument. null by default
argType default: any type
returnType: iterator

yaql> [1, 2, 3].zipLongest([4, 5])
[[1, 4], [2, 5], [3, null]]
yaql> [1, 2, 3].zipLongest([4, 5], default => 100)
[[1, 4], [2, 5], [3, 100]]

Branching functions

The module describes branching functions such as switch, case, and others.

coalesce

Returns the first predicate which evaluates to non-null value. Returns null if no arguments are provided or if all of them are null.

signature: coalesce([args])
callAs: function
arg [args]: input arguments
argType [args]: chain of any types
returnType: any

yaql> coalesce(null)
null
yaql> coalesce(null, [1, 2, 3][0], "abc")
1
yaql> coalesce(null, false, 1)
false

examine

Evaluates predicates one by one and casts the evaluation results to boolean. Returns an iterator to collection of casted results. The actual evaluation is done lazily as the iterator advances, not during the function call.

signature: examine([args])
callAs: function
arg [args]: predicates to be evaluated
argType [args]: chain of predicates functions
returnType: iterator

yaql> examine("ab" > "abc", "ab" <= "abc", "ab" < "abc")
[false, true, true]

selectAllCases

Evaluates input predicates and returns an iterator to collection of zero-based indexes of predicates which were evaluated to true. The actual evaluation is done lazily as the iterator advances, not during the function call.

signature: selectAllCases([args])
callAs: function
arg [args]: predicates to check for true
argType [args]: chain of predicates
returnType: iterator

yaql> selectAllCases("ab" > "abc", "ab" <= "abc", "ab" < "abc")
[1, 2]

selectCase

Returns a zero-based index of the first predicate evaluated to true. If there is no such predicate, returns the count of arguments. All the predicates after the first one which was evaluated to true remain unevaluated.

signature: selectCase([args])
callAs: function
arg [args]: predicates to check for true
argType [args]: chain of predicates
returnType: integer

yaql> selectCase("ab" > "abc", "ab" >= "abc", "ab" < "abc")
2

switch

Returns the value of the first argument for which the key evaluates to true, null if there is no such arg.

signature: switch([args])
callAs: function
arg [args]: mappings with keys to check for true and appropriate values
argType [args]: chain of mapping
returnType: any (types of values of args)

yaql> switch("ab" > "abc" => 1, "ab" >= "abc" => 2, "ab" < "abc" => 3)
3

switchCase

Returns evaluated case-th argument. If case is less than 0 or greater than the amount of predicates, returns evaluated last argument. Returns null if no args are provided.

signature: case.switchCase([args])
callAs: method
recieverArg case: index of predicate to be evaluated
argType case: integer
arg [args]: input predicates
argType [args]: chain of any types
returnType: any

yaql> 1.switchCase('a', 1 + 1, [])
2
yaql> 2.switchCase('a', 1 + 1, [])
[]
yaql> 3.switchCase('a', 1 + 1, [])
[]
yaql> let(1) -> selectCase($ < 0, $ = 0).switchCase("less than 0",


                                                    "equal to 0",


                                                    "greater than 0")
"greater than 0"

String manipulations

The module describes which operations can be done with strings in YAQL.

characters

Returns a list of all distinct items of specified types.

signature: characters(digits => false, hexdigits => false, asciiLowercase => false, asciiUppercase => false, asciiLetters => false, letters => false, octdigits => false, punctuation => false, printable => false, lowercase => false, uppercase => false, whitespace => false)
callAs: function
arg digits: include digits in output list if true, false by default
argType digits: boolean
arg hexdigits: include hexademical digits in output list if true, false by default
argType hexdigits: boolean
arg asciiLowercase: include ASCII lowercase letters in output list if true, false by default
argType asciiLowercase: boolean
arg asciiUppercase: include ASCII uppercase letters in output list if true, false by default
argType asciiUppercase: boolean
arg asciiLetters: include both ASCII lowercase and uppercase letters in output list if true, false by default
argType asciiLetters: boolean
arg letters: include both lowercase and uppercase letters in output list if true, false by default
argType letters: boolean
arg octdigits: include digits from 0 to 7 in output list if true, false by default
argType octdigits: boolean
arg punctuation: include ASCII characters, which are considered punctuation, in output list if true, false by default
argType punctuation: boolean
arg printable: include digits, letters, punctuation, and whitespace in output list if true, false by default
argType printable: boolean
arg lowercase: include lowercase letters in output list if true, false by default
argType lowercase: boolean
arg uppercase: include uppercase letters in output list if true, false by default
argType uppercase: boolean
arg whitespace: include all characters that are considered whitespace in output list if true, false by default
argType whitespace: boolean
returnType: list

yaql> characters(digits => true)
["1", "0", "3", "2", "5", "4", "7", "6", "9", "8"]

concat

Returns concatenated args.

signature: concat([args])
callAs: function
arg [args]: values to be joined
argType [args]: string
returnType: string

yaql> concat("abc", "de", "f")
"abcdef"

endsWith

Returns true if a string ends with any of given args.

signature: string.endsWith([args])
callAs: method
receiverArg string: input string
argType string: string
arg [args]: chain of strings to check input string with
argType [args]: strings
returnType: boolean

yaql> "abcd".endsWith("cd", "xx")
true
yaql> "abcd".endsWith("yy", "xx", "zz")
false

hex

Returns a string with hexadecimal representation of num.

signature: hex(num)
callAs: function
arg num: input number to be converted to hexademical
argType num: number
returnType: string

yaql> hex(256)
"0x100"

indexOf

•: Returns an index of first occurrence sub in string beginning from start. -1 is a return value if there is no any occurrence.

signature: string.indexOf(sub, start => 0)
callAs: method
receiverArg string: input string
argType string: string
arg sub: substring to find in string
argType sub: string
arg start: index to start search with, 0 by default
argType start: integer
returnType: integer

yaql> "cabcdab".indexOf("ab")
1
yaql> "cabcdab".indexOf("ab", 2)
5
yaql> "cabcdab".indexOf("ab", 6)
-1

•: Returns an index of first occurrence sub in string beginning from start ending with start+length. -1 is a return value if there is no any occurrence.

signature: string.indexOf(sub, start, length)
callAs: method
receiverArg string: input string
argType string: string
arg sub: substring to find in string
argType sub: string
arg start: index to start search with, 0 by default
argType start: integer
arg length: length of string to find substring in
argType length: integer
returnType: integer

yaql> "cabcdab".indexOf("bc", 2, 2)
2

isEmpty

Returns true if the string with removed leading and trailing chars is empty.

signature: string.isEmpty(trimSpaces => true, chars => null)
callAs: function or method
receiverArg string: value to be checked for emptiness after trim
argType string: string
arg trimSpaces: true by default, which means string to be trimmed with chars. false means checking whether input string is empty
argType trimSpaces: boolean
arg chars: symbols for trimming. null by default, which means trim is done with whitespace characters
argType chars: string
returnType: boolean

yaql> "abaab".isEmpty(chars=>"ab")
true
yaql> "aba".isEmpty(chars=>"a")
false

isString

Returns true if arg is a string.

signature: isString(arg)
callAs: function
arg arg: input value
argType arg: any
returnType: boolean

yaql> isString("ab")
true
yaql> isString(1)
false

join

•: Returns a string with sequence elements joined by the separator.

signature: sequence.join(separator)
callAs: method
receiverArg sequence: chain of values to be joined
argType sequence: sequence of strings
arg separator: value to be placed between joined pairs
argType separator: string
returnType: string

yaql> ["abc", "de", "f"].join("")
"abcdef"
yaql> ["abc", "de", "f"].join("|")
"abc|de|f"

•: Returns a string with sequence elements joined by the separator.

signature: separator.join(sequence)
callAs: method
receiverArg separator: value to be placed between joined pairs
argType separator: string
arg sequence: chain of values to be joined
argType sequence: sequence of strings
returnType: string

yaql> "|".join(["abc", "de", "f"])
"abc|de|f"

lastIndexOf

•: Returns an index of last occurrence sub in string beginning from start. -1 is a return value if there is no any occurrence.

signature: string.lastIndexOf(sub, start => 0)
callAs: method
receiverArg string: input string
argType string: string
arg sub: substring to find in string
argType sub: string
arg start: index to start search with, 0 by default
argType start: integer
returnType: integer

yaql> "cabcdab".lastIndexOf("ab")
5

•: Returns an index of last occurrence sub in string beginning from start ending with start+length. -1 is a return value if there is no any occurrence.

signature: string.lastIndexOf(sub, start, length)
callAs: method
receiverArg string: input string
argType string: string
arg sub: substring to find in string
argType sub: string
arg start: index to start search with, 0 by default
argType start: integer
arg length: length of string to find substring in
argType length: integer
returnType: integer

yaql> "cabcdbc".lastIndexOf("bc", 2, 5)
5

len

Returns size of the string.

signature: string.len()
callAs: function or method
receiverArg string: input string
argType string: string
returnType: integer

yaql> "abc".len()
3

norm

Returns a string with the leading and trailing chars removed. If the resulting string is empty, returns null.

signature: string.norm(chars => null)
callAs: function or method
receiverArg string: value to be cut with specified chars
argType string: string
arg chars: symbols to be removed from the start and the end of input string. null by default, which means norm is done with whitespace characters
argType chars: string
returnType: string

yaql> "  abcd ".norm()
"abcd"
yaql> "aaaa".norm("a")
null

operator *

•: Returns string repeated count times.

signature: left * right
callAs: operator
arg left: left operand, how many times repeat input string
argType left: integer
arg right: right operator
argType right: string
returnType: string

yaql> 2 * "ab"
"abab"

•: Returns string repeated count times.

signature: left * right
callAs: operator
arg left: left operand
argType left: string
arg right: right operator, how many times repeat input string
argType right: integer
returnType: string

yaql> "ab" * 2
"abab"

operator <

Returns true if the left operand is strictly less than the right, ordering lexicographically, otherwise false.

signature: left < right
callAs: operator
arg left: left operand
argType left: string
arg right: right operand
argType right: string
returnType: boolean

yaql> "ab" < "abc"
true
yaql> "abb" < "abc"
true
yaql> "abc" < "abc"
false

operator <=

Returns true if the left operand is less or equal to the right, ordering lexicographically, otherwise false.

signature: left <= right
callAs: operator
arg left: left operand
argType left: string
arg right: right operand
argType right: string
returnType: boolean

yaql> "ab" <= "abc"
true
yaql> "abc" <= "abc"
true

operator >

Returns true if the left operand is strictly greater than the right, ordering lexicographically, otherwise false.

signature: left > right
callAs: operator
arg left: left operand
argType left: string
arg right: right operand
argType right: string
returnType: boolean

yaql> "abc" > "ab"
true
yaql> "abc" > "abb"
true
yaql> "abc" > "abc"
false

operator >=

Returns true if the left operand is greater or equal to the right, ordering lexicographically, otherwise false.

signature: left >= right
callAs: operator
arg left: left operand
argType left: string
arg right: right operand
argType right: string
returnType: boolean

yaql> "abc" >= "ab"
true
yaql> "abc" >= "abc"
true

operator in

Returns true if there is at least one occurrence of left string in right.

signature: left in right
callAs: operator
arg left: left operand, which occurrence is checked
argType left: string
arg right: right operand
argType right: string
returnType: boolean

yaql> "ab" in "abc"
true
yaql> "ab" in "acb"
false

replace

•: Returns a string with first count occurrences of old replaced with new.

signature: string.replace(old, new, count => -1)
callAs: method
receiverArg string: input string
argType string: string
arg old: value to be replaced
argType old: string
arg new: replacement for old value
argType new: string
arg count: how many first replacements to do. -1 by default, which means to do all replacements
argType count: integer
returnType: string

yaql> "abaab".replace("ab", "cd")
"cdacd"

•: Returns a string with all occurrences of replacements' keys replaced with corresponding replacements' values. If count is specified, only the first count occurrences of every key are replaced.

signature: string.replace(replacements, count => -1)
callAs: method
receiverArg string: input string
argType string: string
arg replacements: dict of replacements in format {old => new ...}
argType replacements: mapping
arg count: how many first occurrences of every key are replaced. -1 by default, which means to do all replacements
argType count: integer
returnType: string

yaql> "abc ab abc".replace({abc => xx, ab => yy})
"xx yy xx"
yaql> "abc ab abc".replace({ab => yy, abc => xx})
"yyc yy yyc"
yaql> "abc ab abc".replace({ab => yy, abc => xx}, 1)
"yyc ab xx"

rightSplit

Returns a list of tokens in the string, using separator as the delimiter. If maxSplits is given then at most maxSplits splits are done - the rightmost ones.

signature: string.rightSplit(separator => null, maxSplits => -1)
callAs: method
receiverArg string: value to be splitted
argType string: string
arg separator: delimiter for splitting. null by default, which means splitting with whitespace characters
argType separator: string
arg maxSplits: number of splits to be done - the rightmost ones. -1 by default, which means all possible splits are done
argType maxSplits: integer
returnType: list

yaql> "abc     de  f".rightSplit()
["abc", "de", "f"]
yaql> "abc     de  f".rightSplit(maxSplits => 1)
["abc     de", "f"]

split

Returns a list of tokens in the string, using separator as the delimiter.

signature: string.split(separator => null, maxSplits => -1)
callAs: method
receiverArg string: value to be splitted
argType string: string
arg separator: delimiter for splitting. null by default, which means splitting with whitespace characters
argType separator: string
arg maxSplits: maximum number of splittings. -1 by default, which means all possible splits are done
argType maxSplits: integer
returnType: list

yaql> "abc     de  f".split()
["abc", "de", "f"]
yaql> "abc     de  f".split(maxSplits => 1)
["abc", "de  f"]
yaql> "abcde".split("c")
["ab", "de"]

startsWith

Returns true if a string starts with any of given args.

signature: string.startsWith([args])
callAs: method
receiverArg string: input string
argType string: string
arg [args]: chain of strings to check input string with
argType [args]: strings
returnType: boolean

yaql> "abcd".startsWith("ab", "xx")
true
yaql> "abcd".startsWith("yy", "xx", "zz")
false

str

Returns a string representation of the value.

signature: str(value)
callAs: function
arg value: value to be evaluated to string
argType value: any
returnType: string

yaql> str(["abc", "de"])
"(u'abc', u'd')"
yaql> str(123)
"123"

substring

Returns a substring beginning from start index ending with start+end index.

signature: string.substring(start, length => -1)
callAs: method
receiverArg string: input string
argType string: string
arg start: index for substring to start with
argType start: integer
arg length: length of substring. -1 by default, which means end of substring to be equal to the end of input string
argType length: integer
returnType: string

yaql> "abcd".substring(1)
"bcd"
yaql> "abcd".substring(1, 2)
"bc"

toCharArray

Converts a string to array of one character strings.

signature: string.toCharArray()
callAs: method
receiverArg string: input string
argType string: string
returnType: list

yaql> "abc de".toCharArray()
["a", "b", "c", " ", "d", "e"]

toLower

Returns a string with all case-based characters lowercase.

signature: string.toLower()
callAs: method
receiverArg string: value to lowercase
argType string: string
returnType: string

yaql> "AB1c".toLower()
"ab1c"

toUpper

Returns a string with all case-based characters uppercase.

signature: string.toUpper()
callAs: method
receiverArg string: value to uppercase
argType string: string
returnType: string

yaql> "aB1c".toUpper()
"AB1C"

trim

Returns a string with the leading and trailing chars removed.

signature: string.trim(chars => null)
callAs: method
receiverArg string: value to be trimmed
argType string: string
arg chars: symbols to be removed from input string. null by default, which means trim is done with whitespace characters
argType chars: string
returnType: string

yaql> "  abcd ".trim()
"abcd"
yaql> "aababa".trim("a")
"bab"

trimLeft

Returns a string with the leading chars removed.

signature: string.trimLeft(chars => null)
callAs: method
receiverArg string: value to be trimmed
argType string: string
arg chars: symbols to be removed from start of input string. null by default, which means trim is done with whitespace characters
argType chars: string
returnType: string

yaql> "  abcd ".trimLeft()
"abcd "
yaql> "aababa".trimLeft("a")
"baba"

trimRight

Returns a string with the trailing chars removed.

signature: string.trimRight(chars => null)
callAs: method
receiverArg string: value to be trimmed
argType string: string
arg chars: symbols to be removed from end of input string. null by default, which means trim is done with whitespace characters
argType chars: string
returnType: string

yaql> "  abcd ".trimRight()
"  abcd"
yaql> "aababa".trimRight("a")
"aabab"

Math functions

The Math module describes implemented math operations on numbers.

abs

Returns the absolute value of a number.

signature: abs(op)
callAs: function
arg op: input value
argType op: number
returnType: number

yaql> abs(-2)
2

bitwiseAnd

Returns applied "bitwise and" to left and right integers. Each bit of the output is 1 if the corresponding bit of left AND right is 1, otherwise 0.

signature: bitwiseAnd(left, right)
callAs: function
arg left: left value
argType left: integer
arg right: right value
argType right: integer
returnType: integer

yaql> bitwiseAnd(6, 12)
4

bitwiseNot

Returns an integer where each bit is a reversed corresponding bit of arg.

signature: bitwiseNot(arg)
callAs: function
arg arg: input value
argType arg: integer
returnType: integer

yaql> bitwiseNot(6)
-7

bitwiseOr

Returns applied "bitwise or" to left and right numbers. Each bit of the output is 1 if the corresponding bit of left OR right is 1, otherwise 0.

signature: bitwiseOr(left, right)
callAs: function
arg left: left value
argType left: integer
arg right: right value
argType right: integer
returnType: integer

yaql> bitwiseOr(6, 12)
14

bitwiseXor

Returns applied "bitwise exclusive or" to left and right numbers. Each bit of the output is equal to the sum of corresponding left and right bits mod 2.

signature: bitwiseXor(left, right)
callAs: function
arg left: left value
argType left: integer
arg right: right value
argType right: integer
returnType: integer

yaql> bitwiseXor(6, 12)
10

float

Returns a floating number built from number, string or null value.

signature: float(value)
callAs: function
arg value: input value
argType value: number, string or null
returnType: float

yaql> float("2.2")
2.2
yaql> float(12)
12.0
yaql> float(null)
0.0

int

Returns an integer built from number, string or null value.

signature: int(value)
callAs: function
arg value: input value
argType value: number, string or null
returnType: integer

yaql> int("2")
2
yaql> int(12.999)
12
yaql> int(null)
0

isInteger

Returns true if value is an integer number, otherwise false.

signature: isInteger(value)
callAs: function
arg value: input value
argType value: any
returnType: boolean

yaql> isInteger(12.0)
false
yaql> isInteger(12)
true

isNumber

Returns true if value is an integer or floating number, otherwise false.

signature: isNumber(value)
callAs: function
arg value: input value
argType value: any
returnType: boolean

yaql> isNumber(12.0)
true
yaql> isNumber(12)
true

max

Returns max from a and b.

signature: max(a, b)
callAs: function
arg a: input value
argType a: number
arg b: input value
argType b: number
returnType: number

yaql> max(8, 2)
8

min

Returns min from a and b.

signature: min(a, b)
callAs: function
arg a: input value
argType a: number
arg b: input value
argType b: number
returnType: number

yaql> min(8, 2)
2

operator *

Returns left multiplied by right.

signature: left * right
callAs: operator
arg left: left operand
argType left: number
arg right: right operand
argType right: number
returnType: number

yaql> 3 * 2.5
7.5

operator +

Returns the sum of left and right operands.

signature: left + right
callAs: operator
arg left: left operand
argType left: number
arg right: right operand
argType right: number
returnType: number

yaql> 3 + 2
5

operator -

Returns the difference between left and right.

signature: left - right
callAs: operator
arg left: left operand
argType left: number
arg right: right operand
argType right: number
returnType: number

yaql> 3 - 2
1

operator /

Returns left divided by right.

signature: left / right
callAs: operator
arg left: left operand
argType left: number
arg right: right operand
argType right: number
returnType: number

yaql> 3 / 2
1
yaql> 3.0 / 2
1.5

operator <

Returns true if left is strictly less than right, false otherwise.

signature: left < right
callAs: operator
arg left: left operand
argType left: number
arg right: right operand
argType left: number
returnType: boolean

yaql> 3 < 2
false

operator <=

Returns true if left is less or equal to right, false otherwise.

signature: left <= right
callAs: operator
arg left: left operand
argType left: number
arg right: right operand
argType left: number
returnType: boolean

yaql> 3 <= 3
true

operator >

Returns true if left is strictly greater than right, false otherwise.

signature: left > right
callAs: operator
arg left: left operand
argType left: number
arg right: right operand
argType left: number
returnType: boolean

yaql> 3 > 2
true

operator >=

Returns true if left is greater or equal to right, false otherwise.

signature: left >= right
callAs: operator
arg left: left operand
argType left: number
arg right: right operand
argType left: number
returnType: boolean

yaql> 3 >= 3
true

operator mod

Returns left modulo right.

signature: left mod right
callAs: operator
arg left: left operand
argType left: number
arg right: right operand
argType right: number
returnType: number

yaql> 3 mod 2
1

operator unary +

Returns +op.

signature: +op
callAs: operator
arg op: operand
argType op: number
returnType: number

yaql> +2
2

operator unary -

Returns -op.

signature: -op
callAs: operator
arg op: operand
argType op: number
returnType: number

yaql> -2
-2

pow

Returns a to the power b modulo c.

signature: pow(a, b, c => null)
callAs: function
arg a: input value
argType a: number
arg b: power
argType b: number
arg c: modulo. null by default, which means no modulo is done after power.
argType c: integer
returnType: number

yaql> pow(3, 2)
9
yaql> pow(3, 2, 5)
4

random

•: Returns the next random floating number from [0.0, 1.0).

signature: random()
callAs: function
returnType: float

yaql> random()
0.6039529924951869

•: Returns the next random integer from [a, b].

signature: random(from, to)
callAs: function
arg from: left value for generating random number
argType from: integer
arg to: right value for generating random number
argType to: integer
returnType: integer

yaql> random(1, 2)
2
yaql> random(1, 2)
1

round

Returns a floating number rounded to ndigits after the decimal point.

signature: round(number, ndigits => 0)
callAs: function
arg number: input value
argType number: number
arg ndigits: with how many digits after decimal point to round. 0 by default
argType ndigits: integer
returnType: number

yaql> round(12.52)
13
yaql> round(12.52, 1)
12.5

shiftBitsLeft

Shifts the bits of value left by the number of bits bitsNumber.

signature: shiftBitsLeft(value, bitsNumber)
callAs: function
arg value: given value
argType value: integer
arg bitsNumber: number of bits
argType right: integer
returnType: integer

yaql> shiftBitsLeft(8, 2)
32

shiftBitsRight

Shifts the bits of value right by the number of bits bitsNumber.

signature: shiftBitsRight(value, bitsNumber)
callAs: function
arg value: given value
argType value: integer
arg bitsNumber: number of bits
argType right: integer
returnType: integer

yaql> shiftBitsRight(8, 2)
2

sign

Returns 1 if num > 0; 0 if num = 0; -1 if num < 0.

signature: sign(num)
callAs: function
arg num: input value
argType num: number
returnType: integer (-1, 0 or 1)

yaql> sign(2)
1

Regex functions

The module contains functions for regular expressions.

escapeRegex

Returns string with all the characters except ASCII letters, numbers, and '_' escaped.

signature: escapeRegex(string)
callAs: function
arg string: string to backslash all non-alphanumerics
argType string: string
returnType: string

yaql> escapeRegex('a.')
"a\."

isRegex

Returns true if value is a regex object.

signature: isRegex(value)
callAs: function
arg value: string to backslash all non-alphanumerics
argType value: any
returnType: boolean

yaql> isRegex(regex("a.c"))
true
yaql> isRegex(regex("a.c").matches("abc"))
false

matches

•: Returns true if string matches regexp.

signature: regexp.matches(string)
callAs: method
receiverArg regexp: regex pattern
argType regexp: regex object
arg string: string to find match in
argType string: string
returnType: boolean

yaql> regex("a.c").matches("abc")
true

•: Returns true if string matches regexp, false otherwise.

signature: string.matches(regexp)
callAs: method
receiverArg string: string to find match in
argType string: string
arg regexp: regex pattern
argType regexp: regex object
returnType: boolean

yaql> "abc".matches("a.c")
true

operator !~

•: Returns true if left doesn't match right, false otherwise.

signature: left !~ right
callAs: operator
arg left: string to find match in
argType left: string
arg right: regex pattern
argType right: regex
returnType: boolean

yaql> "acb" !~ regex("a.c")
true
yaql> "abc" !~ regex("a.c")
false

•: Returns true if left doesn't match right, false otherwise.

signature: left !~ right
callAs: operator
arg left: string to find match in
argType left: string
arg right: regex pattern
argType right: regex object
returnType: boolean

yaql> "acb" !~ regex("a.c")
true
yaql> "abc" !~ regex("a.c")
false

operator =~

•: Returns true if left matches right, false otherwise.

signature: left =~ right
callAs: operator
arg left: string to find match in
argType left: string
arg right: regex pattern
argType right: regex
returnType: boolean

yaql> "abc" =~ regex("a.c")
true

•: Returns true if left matches right, false otherwise.

signature: left =~ right
callAs: operator
arg left: string to find match in
argType left: string
arg right: regex pattern
argType right: string
returnType: boolean

yaql> "abc" =~ "a.c"
true

regex

Returns regular expression object with provided flags. Can be used for matching using matches method.

signature: regex(pattern,ignoreCase => false, multiLine => false, dotAll => false)
callAs: function
arg pattern: regular expression pattern to be compiled to regex object
argType pattern: string
arg ignoreCase: true makes performing case-insensitive matching.
argType ignoreCase: boolean
arg multiLine: true makes character '^' to match at the beginning of the string and at the beginning of each line, the character '$' to match at the end of the string and at the end of each line. false means '^' to match only at the beginning of the string, '$' only at the end of the string.
argType multiLine: boolean
arg dotAll: true makes the '.' special character to match any character (including a newline). false makes '.' to match anything except a newline.
argType dotAll: boolean
returnType: regex object

yaql> regex("a.c").matches("abc")
true
yaql> regex("A.c").matches("abc")
false
yaql> regex("A.c", ignoreCase => true).matches("abc")
true

replace

•: Returns the string obtained by replacing the leftmost non-overlapping matches of regexp in string by the replacement repl, where the latter is only string-type.

signature: regexp.replace(string, repl, count => 0)
callAs: method
receiverArg regexp: regex pattern
argType regexp: regex object
arg string: string to make replace in
argType string: string
arg repl: string to replace matches of regexp
argType repl: string
arg count: how many first replaces to do. 0 by default, which means to do all replacements
argType count: integer
returnType: string

yaql> regex("a.").replace("abcadc", "xx")
"xxcxxc"
yaql> regex("a.").replace("abcadc", "xx", count => 1)
"xxcadc"

•: Returns the string obtained by replacing the leftmost non-overlapping matches of regexp in string by the replacement repl, where the latter is only string-type.

signature: string.replace(regexp, repl, count => 0)
callAs: method
receiverArg string: string to make replace in
argType string: string
arg regexp: regex pattern
argType regexp: regex object
arg repl: string to replace matches of regexp
argType repl: string
arg count: how many first replaces to do. 0 by default, which means to do all replacements
argType count: integer
returnType: string

yaql> "abcadc".replace(regex("a."), "xx")
"xxcxxc"

replaceBy

•: Returns the string obtained by replacing the leftmost non-overlapping matches of regexp in string by repl, where the latter is an expression to get replacements by obtained matches.

signature: regexp.replaceBy(string, repl, count => 0)
callAs: method
receiverArg regexp: regex pattern
argType regexp: regex object
arg string: string to make replace in
argType string: string
arg repl: lambda function which returns string to make replacements according to input matches
argType repl: lambda
arg count: how many first replaces to do. 0 by default, which means to do all replacements
argType count: integer
returnType: string

yaql> regex("a.c").replaceBy("abcadc", switch($.value = "abc" => xx,


                                              $.value = "adc" => yy))
"xxyy"

•: Replaces matches of regexp in string with values provided by the supplied function.

signature: string.replaceBy(regexp, repl, count => 0)
callAs: method
receiverArg string: string to make replace in
argType string: string
arg regexp: regex pattern
argType regexp: regex object
arg repl: lambda function which returns string to make replacements according to input matches
argType repl: lambda
arg count: how many first replaces to do. 0 by default, which means to do all replacements
argType count: integer
returnType: string

yaql> "abcadc".replaceBy(regex("a.c"), switch($.value = "abc" => xx,


                                              $.value = "adc" => yy))
"xxyy"

search

Search substring which matches regexp. Returns selector applied to dictionary {"start" => ..., "end" => ..., "value" => ...} where appropriate values describe start of substring, its end and itself. By default, if no selector is specified, returns only substring. null is a return value if there is no substring which matches regexp.

signature: regexp.search(string, selector => null)
callAs: method
receiverArg regexp: regex pattern
argType regexp: regex object
arg string: string to find match in
argType string: string
arg selector: lambda function to be applied to resulted dictionary with keys 'start', 'end', 'value'. null by default, which means to return only substring.
argType selector: lambda
returnType: string or selector return type

yaql> regex("a.c").search("abcabc")
"abc"
yaql> regex("a.c").search("cabc", $)
{


    "start": 1,


    "end": 4,


    "value": "abc"
}
yaql> regex("a.c").search("cabc", $.start)
1

searchAll

Search all substrings which matches regexp. Returns list of applied to dictionary {"start" => ..., "end" => ..., "value" => ...} selector, where appropriate values describe start of every substring, its end and itself. By default, if no selector is specified, returns only list of substrings.

signature: regexp.searchAll(string, selector => null)
callAs: method
receiverArg regexp: regex pattern
argType regexp: regex object
arg string: string to find match in
argType string: string
arg selector: lambda function to be applied to resulted dictionary of every substring with keys 'start', 'end', 'value'. null by default, which means to return only list of substrings.
argType selector: lambda
returnType: list

yaql> regex("a.c").searchAll("abcadc")
["abc", "adc"]
yaql> regex("a.c").searchAll("abcadc", $)
[


    {


        "start": 0,


        "end": 3,


        "value": "abc"


    },


    {


        "start": 3,


        "end": 6,


        "value": "adc"


    }
]

name: searchAll

split

•: Splits string by regexp matches and returns list of strings.

signature: regexp.split(string, maxSplit => 0)
callAs: method
receiverArg regexp: regex pattern
argType regexp: regex object
arg string: string to be splitted
argType string: string
arg maxSplit: how many first splits to do. 0 by default, which means to split by all matches
argType maxSplit: integer
returnType: list

yaql> regex("a.").split("abcadc")
["", "c", "c"]
yaql> regex("a.").split("abcadc", maxSplit => 1)
["", "cadc"]

•: Splits string by regexp matches and returns list of strings.

signature: string.split(regexp, maxSplit => 0)
callAs: method
receiverArg string: string to be splitted
argType string: string
arg regexp: regex pattern
argType regexp: regex object
arg maxSplit: how many first splits to do. 0 by default, which means to split by all matches
argType maxSplit: integer
returnType: list

yaql> "abcadc".split(regex("a."))
["", "c", "c"]
yaql> "abcadc".split(regex("a."), maxSplit => 1)
["", "cadc"]

DateTime functions

The module describes which operations can be done with datetime objects.

datetime

•: Returns datetime object built on year, month, day, hour, minute, second, microsecond, offset.

signature: datetime(year, month, day, hour => 0, minute => 0, second => 0, microsecond => 0, offset => timespan(0))
callAs: function
arg year: number of years in datetime
argType year: integer between 1 and 9999 inclusive
arg month: number of months in datetime
argType month: integer between 1 and 12 inclusive
arg day: number of days in datetime
argType day: integer between 1 and number of days in given month
arg hour: number of hours in datetime, 0 by default
argType hour: integer between 0 and 23 inclusive
arg minute: number of minutes in datetime, 0 by default
argType minute: integer between 0 and 59 inclusive
arg second: number of seconds in datetime, 0 by default
argType second: integer between 0 and 59 inclusive
arg microsecond: number of microseconds in datetime, 0 by default
argType microsecond: integer between 0 and 1000000-1
arg offset: datetime offset in microsecond resolution, needed for tzinfo, timespan(0) by default
argType offset: timespan type
returnType: datetime object

yaql> let(datetime(2015, 9, 29)) -> [$.year, $.month, $.day]
[2015, 9, 29]

•: Returns datetime object built by string parsed with format.

signature: datetime(string, format => null)
callAs: function
arg string: string representing datetime
argType string: string
arg format: format for parsing input string which should be supported with C99 standard of format codes. null by default, which means parsing with Python dateutil.parser usage
argType format: string
returnType: datetime object

yaql> let(datetime("29.8?2015")) -> [$.year, $.month, $.day]
[2015, 8, 29]
yaql> let(datetime("29.8?2015", "%d.%m?%Y"))->[$.year, $.month, $.day]
[2015, 8, 29]

•: Returns datetime object built by timestamp.

signature: datetime(timestamp, offset => timespan(0))
callAs: function
arg timestamp: timespan object to represent datetime
argType timestamp: number
arg offset: datetime offset in microsecond resolution, needed for tzinfo, timespan(0) by default
argType offset: timespan type
returnType: datetime object

yaql> let(datetime(1256953732)) -> [$.year, $.month, $.day]
[2009, 10, 31]

format

Returns a string representing datetime, controlled by a format string.

signature: dt.format(format)
callAs: method
receiverArg dt: input datetime object
argType dt: datetime object
arg format: format string
argType format: string
returnType: string

yaql> now().format("%A, %d. %B %Y %I:%M%p")
"Tuesday, 19. July 2016 08:49AM"

isDatetime

Returns true if value is datetime object, false otherwise.

signature: isDatetime(value)
callAs: function
arg value: input value
argType value: any
returnType: boolean

yaql> isDatetime(now())
true
yaql> isDatetime(datetime(2010, 10, 10))
true

isTimespan

Returns true if value is timespan object, false otherwise.

signature: isTimespan(value)
callAs: function
arg value: input value
argType value: any
returnType: boolean

yaql> isTimespan(now())
false
yaql> isTimespan(timespan())
true

localtz

Returns local time zone in timespan object.

signature: localtz()
callAs: function
returnType: timespan object

yaql> localtz().hours
3.0

now

Returns the current local date and time.

signature: now(offset => timespan(0))
callAs: function
arg offset: datetime offset in microsecond resolution, needed for tzinfo, timespan(0) by default
argType offset: timespan type
returnType: datetime

yaql> let(now()) -> [$.year, $.month, $.day]
[2016, 7, 18]
yaql> now(offset=>localtz()).hour - now().hour
3

operator *

•: Returns timespan object built on number multiplied by timespan.

signature: left * right
callAs: operator
arg left: number to multiply timespan
argType left: number
arg right: timespan object
argType right: timespan object
returnType: timespan

yaql> let(2 * timespan(hours => 24)) -> $.hours
48.0

•: Returns timespan object built on timespan multiplied by number.

signature: left * right
callAs: operator
arg left: timespan object
argType left: timespan object
arg right: number to multiply timespan
argType right: number
returnType: timespan

yaql> let(timespan(hours => 24) * 2) -> $.hours
48.0

operator +

•: Returns datetime object with added timespan.

signature: left + right
callAs: operator
arg left: input datetime object
argType left: datetime object
arg right: input timespan object
argType right: timespan object
returnType: datetime object

yaql> let(now() + timespan(days => 100)) -> $.month
10

•: Returns datetime object with added timespan.

signature: left + right
callAs: operator
arg left: input timespan object
argType left: timespan object
arg right: input datetime object
argType right: datetime object
returnType: datetime object

yaql> let(timespan(days => 100) + now()) -> $.month
10

•: Returns sum of two timespan objects.

signature: left + right
callAs: operator
arg left: input timespan object
argType left: timespan object
arg right: input timespan object
argType right: timespan object
returnType: timespan object

yaql> let(timespan(days => 1) + timespan(hours => 12)) -> $.hours
36.0

operator -

•: Returns datetime object dt1 with subtracted dt2.

signature: left - right
callAs: operator
arg left: input datetime object
argType left: datetime object
arg right: datetime object to be subtracted
argType right: datetime object
returnType: timespan object

yaql> let(now() - now()) -> $.microseconds
-325

•: Returns datetime object with subtracted timespan.

signature: left - right
callAs: operator
arg left: input datetime object
argType left: datetime object
arg right: input timespan object
argType right: timespan object
returnType: datetime object

yaql> let(now() - timespan(days => 100)) -> $.month
4

•: Returns timespan object with subtracted another timespan object.

signature: left - right
callAs: operator
arg left: input timespan object
argType left: timespan object
arg right: input timespan object
argType right: timespan object
returnType: timespan object

yaql> let(timespan(days => 1) - timespan(hours => 12)) -> $.hours
12.0

operator /

•: Returns timespan object divided by number.

signature: left / right
callAs: operator
arg left: left timespan object
argType left: timespan object
arg right: number to divide by
argType right: number
returnType: timespan object

yaql> let(timespan(hours => 24) / 2) -> $.hours
12.0

•: Returns result of division of timespan microseconds by another timespan microseconds.

signature: left / right
callAs: operator
arg left: left timespan object
argType left: timespan object
arg right: right timespan object
argType right: timespan object
returnType: float

yaql> timespan(hours => 24) / timespan(hours => 12)
2.0

operator <

•: Returns true if left datetime is strictly less than right datetime, false otherwise.

signature: left < right
callAs: operator
arg left: left datetime object
argType left: datetime object
arg right: right datetime object
argType right: datetime object
returnType: boolean

yaql> datetime(2011, 11, 11) < datetime(2011, 11, 11)
false

•: Returns true if left timespan is strictly less than right timespan, false otherwise.

signature: left < right
callAs: operator
arg left: left timespan object
argType left: timespan object
arg right: right timespan object
argType right: timespan object
returnType: boolean

yaql> timespan(hours => 23) < timespan(days => 1)
true

operator <=

•: Returns true if left datetime is less or equal to right datetime, false otherwise.

signature: left <= right
callAs: operator
arg left: left datetime object
argType left: datetime object
arg right: right datetime object
argType right: datetime object
returnType: boolean

yaql> datetime(2011, 11, 11) <= datetime(2011, 11, 11)
true

•: Returns true if left timespan is less or equal to right timespan, false otherwise.

signature: left <= right
callAs: operator
arg left: left timespan object
argType left: timespan object
arg right: right timespan object
argType right: timespan object
returnType: boolean

yaql> timespan(hours => 23) <= timespan(days => 1)
true

operator >

•: Returns true if left datetime is strictly greater than right datetime, false otherwise.

signature: left > right
callAs: operator
arg left: left datetime object
argType left: datetime object
arg right: right datetime object
argType right: datetime object
returnType: boolean

yaql> datetime(2011, 11, 11) > datetime(2010, 10, 10)
true

•: Returns true if left timespan is strictly greater than right timespan, false otherwise.

signature: left > right
callAs: operator
arg left: left timespan object
argType left: timespan object
arg right: right timespan object
argType right: timespan object
returnType: boolean

yaql> timespan(hours => 2) > timespan(hours => 1)
true

operator >=

•: Returns true if left datetime is greater or equal to right datetime, false otherwise.

signature: left >= right
callAs: operator
arg left: left datetime object
argType left: datetime object
arg right: right datetime object
argType right: datetime object
returnType: boolean

yaql> datetime(2011, 11, 11) >= datetime(2011, 11, 11)
true

•: Returns true if left timespan is greater or equal to right timespan, false otherwise.

signature: left >= right
callAs: operator
arg left: left timespan object
argType left: timespan object
arg right: right timespan object
argType right: timespan object
returnType: boolean

yaql> timespan(hours => 24) >= timespan(days => 1)
true

operator unary +

Returns timespan.

signature: +arg
callAs: operator
arg arg: input timespan object
argType arg: timespan object
returnType: timespan object

yaql> let(+timespan(hours => -24)) -> $.hours
-24.0

operator unary -

Returns negative timespan.

signature: -arg
callAs: operator
arg arg: input timespan object
argType arg: timespan object
returnType: timespan object

yaql> let(-timespan(hours => 24)) -> $.hours
-24.0

replace

Returns datetime object with applied replacements.

signature: dt.replace(year => null, month => null, day => null, hour => null, minute => null, second => null, microsecond => null, offset => null)
callAs: method
receiverArg dt: input datetime object
argType dt: datetime object
arg year: number of years to replace, null by default which means no replacement
argType year: integer between 1 and 9999 inclusive
arg month: number of months to replace, null by default which means no replacement
argType month: integer between 1 and 12 inclusive
arg day: number of days to replace, null by default which means no replacement
argType day: integer between 1 and number of days in given month
arg hour: number of hours to replace, null by default which means no replacement
argType hour: integer between 0 and 23 inclusive
arg minute: number of minutes to replace, null by default which means no replacement
argType minute: integer between 0 and 59 inclusive
arg second: number of seconds to replace, null by default which means no replacement
argType second: integer between 0 and 59 inclusive
arg microsecond: number of microseconds to replace, null by default which means no replacement
argType microsecond: integer between 0 and 1000000-1
arg offset: datetime offset in microsecond resolution to replace, null by default which means no replacement
argType offset: timespan type
returnType: datetime object

yaql> datetime(2015, 9, 29).replace(year => 2014).year
2014

timespan

Returns timespan object with specified args.

signature: timespan(days => 0, hours => 0, minutes => 0, seconds => 0, milliseconds => 0, microseconds => 0)
callAs: function
arg days: number of days in timespan, 0 by default
argType days: integer
arg hours: number of hours in timespan, 0 by default
argType hours: integer
arg minutes: number of minutes in timespan, 0 by default
argType minutes: integer
arg seconds: number of seconds in timespan, 0 by default
argType seconds: integer
arg milliseconds: number of microseconds in timespan, 0 by default
argType milliseconds: integer
arg microsecond: number of microseconds in timespan, 0 by default
argType microsecond: integer
returnType: timespan object

yaql> timespan(days => 1, hours => 2, minutes => 3).hours
26.05

utctz

Returns UTC time zone in timespan object.

signature: utctz()
callAs: function
returnType: timespan object

yaql> utctz().hours
0.0

Intrinsic functions

The module describes main system functions for working with objects.

assert

Evaluates condition against object. If it evaluates to true returns the object, otherwise throws an exception with provided message.

signature: obj.assert(condition, message => "Assertion failed")
callAs: method
arg obj: object to evaluate condition on
argType obj: any
arg condition: lambda function to be evaluated on obj. If result of function evaluates to false then trows exception message
argType condition: lambda
arg message: message to trow if condition returns false
argType message: string
returnType: obj type or message

yaql> 12.assert($ < 2)
Execution exception: Assertion failed
yaql> 12.assert($ < 20)
12
yaql> [].assert($, "Failed assertion")
Execution exception: Failed assertion

call

•: Evaluates function with specified args and kwargs and returns the result. This function is used to transform expressions like '$foo(args, kwargs)' to '#call($foo, args, kwargs)'. Note that to use this functionality 'delegate' mode has to be enabled.

signature: call(callable, args, kwargs)
callAs: function
arg callable: callable function
argType callable: python type
arg args: sequence of items to be used for calling
argType args: sequence
arg kwargs: dictionary with kwargs to be used for calling
argType kwargs: mapping
returnType: any (callable return type)

•: Evaluates function name with specified args and kwargs and returns the result.

signature: call(name, args, kwargs)
callAs: function
arg name: name of callable
argType name: string
arg args: sequence of items to be used for calling
argType args: sequence
arg kwargs: dictionary with kwargs to be used for calling
argType kwargs: mapping
returnType: any (callable return type)

yaql> call(let, [1, 2], {a => 3, b => 4}) -> $1 + $a + $2 + $b
10

def

Returns new context object with function name defined.

signature: def(name, func)
callAs: function
arg name: name of function
argType name: string
arg func: function to be stored under provided name
argType func: lambda
returnType: context object

yaql> def(sq, $*$) -> [1, 2, 3].select(sq($))
[1, 4, 9]

getContextData

Returns the context value by its name. This function is system and can be overridden to change the way of getting context data.

signature: getContextData(name)
callAs: function
arg name: value's key name
argType name: string
returnType: any (value type)

lambda

Constructs a new anonymous function Note that to use this function 'delegate' mode has to be enabled.

signature: lambda(func)
callAs: function
arg func: function to be returned
argType func: lambda
returnType: obj type or message

yaql> let(func => lambda(2 * $)) -> [1, 2, 3].select($func($))
[2, 4, 6]
yaql> [1, 2, 3, 4].where(lambda($ > 3)($ + 1))
[3, 4]

let

Returns context object where args are stored with 1-based indexes and kwargs values are stored with appropriate keys.

signature: let([args], {kwargs})
callAs: function
arg [args]: values to be stored under appropriate numbers $1, $2, ...
argType [args]: chain of any values
arg {kwargs}: values to be stored under appropriate keys
argType {kwargs}: chain of mappings
returnType: context object

yaql> let(1, 2, a => 3, b => 4) -> $1 + $a + $2 + $b
10

operator ->

Evaluates lambda on provided context and returns the result.

signature: left -> right
callAs: operator
arg left: context to be used for function
argType left: context object
arg right: function
argType right: lambda
returnType: any (function return value type)

yaql> let(a => 1) -> $a
1

operator .

•: Returns value of 'name' property.

signature: left.right
callAs: operator
arg left: object
argType left: any
arg right: object property name
argType right: keyword
returnType: any

yaql> now().year
2016

•: Returns expr evaluated on receiver.

signature: receiver.expr
callAs: operator
arg receiver: object to evaluate expression
argType receiver: any
arg expr: expression
argType expr: expression that can be evaluated as a method
returnType: expression result type

yaql> [0, 1].select($+1)
[1, 2]

operator ?.

Evaluates expr on receiver if receiver isn't null and returns the result. If receiver is null returns null.

signature: receiver?.expr
callAs: operator
arg receiver: object to evaluate expression
argType receiver: any
arg expr: expression
argType expr: expression that can be evaluated as a method
returnType: expression result or null

yaql> [0, 1]?.select($+1)
[1, 2]
yaql> null?.select($+1)
null

unpack

Returns context object with sequence values unpacked to args. If args size is equal to sequence size then args get appropriate sequence values. If args size is 0 then args are 1-based indexes. Otherwise ValueError is raised.

signature: sequence.unpack([args])
callAs: method
receiverArg sequence: iterable of items to be passed as context values
argType sequence: iterable
arg [args]: keys to be associated with sequence items. If args size is equal to sequence size then args get appropriate sequence items. If args size is 0 then args are indexed start from 1. Otherwise exception will be thrown
argType [args]: chain of strings
returnType: context object

yaql> [1, 2].unpack(a, b) -> $a + $b
3
yaql> [2, 3].unpack() -> $1 + $2
5

with

Returns new context object where args are stored with 1-based indexes.

signature: with([args])
callAs: function
arg [args]: values to be stored under appropriate numbers $1, $2, ...
argType [args]: chain of any values
returnType: context object

yaql> with("ab", "cd") -> $1 + $2
"abcd"

YAQL`ization of Python classes

Any Python class or object can be yaqlized. It is possible to call methods, access attributes/properties and index of yaqlized objects.

The first way to yaqlize object is using function call:

class A(object):


    foo = 256


    def bar(self):


        print('yaqlization works with methods too')
sample_object = A()
yaqlization.yaqlize(sample_object)

The second way is using decorator:

@yaqlization.yaqlize
class A(object):


    foo = 256


    def bar(self):


        print('yaqlization works with methods too')

Any mentioned operation on yaqlized objects can be disabled with additional parameters for yaqlization. Also it is possible to specify whitelist/blacklist of methods/attributes/keys that are exposed to the yaql.

This module provides implemented operators on Yaqlized objects.

operator .

•: Returns attribute of the object.

signature: obj.attr
callAs: operator
arg obj: yaqlized object
argType obj: yaqlized object, initialized with yaqlize_attributes equal to True
arg attr: attribute name
argType attr: keyword
returnType: any

•: Evaluates expression on receiver and returns its result.

signature: receiver.expr
callAs: operator
arg receiver: yaqlized receiver
argType receiver: yaqlized object, initialized with yaqlize_methods equal to True
arg expr: expression to be evaluated
argType expr: expression
returnType: any (expression return type)

operator indexer

Returns value of attribute/property key of the object.

signature: obj[key]
callAs: function
arg obj: yaqlized object
argType obj: yaqlized object, initialized with yaqlize_indexer equal to True
arg key: index name
argType key: keyword
returnType: any

Legacy YAQL compatibility functions

The module describes functions which are available with backward compatibility mode with YAQL v0.2. Examples are provided with CLI started with legacy mode.

as

Returns context object with mapping functions applied on receiver and passed under corresponding keys.

signature: receiver.as([args])
callAs: method
receiverArg receiver: value to be used for mappings lambdas evaluating
argType receiver: any type
arg [args]: tuples with lambdas and appropriate keys to be passed to context
argType [args]: chain of tuples
returnType: context object

yaql> [1, 2].as(len($) => a, sum($) => b) -> $a + $b
5

dict

Returns dict built from tuples.

signature: dict([args])
callAs: function or method
arg [args]: chain of tuples to be interpreted as (key, value) for dict
argType [args]: chain of tuples
returnType: dictionary

yaql> dict(a => 1, b => 2)
{"a": 1, "b": 2}
yaql> dict(tuple(a, 1), tuple(b, 2))
{"a": 1, "b": 2}

operator .

Returns dict's key value.

signature: left.right
callAs: operator
arg left: input dictionary
argType left: mapping
arg right: key
argType right: keyword
returnType: any (appropriate value type)

yaql> {a => 2, b => 2}.a
2

operator =>

Returns tuple.

signature: left => right
callAs: operator
arg left: left value for tuple
argType left: any
arg right: right value for tuple
argType right: any
returnType: tuple

yaql> a => b
["a", "b"]
yaql> null => 1 => []
[null, 1, []]

range

Returns sequence from [start, stop).

signature: start.range(stop => null)
callAs: function or method
receiverArg start: value to start from
argType start: integer
arg stop: value to end with. null by default, which means returning iterator to sequence
argType stop: integer
returnType: sequence

yaql> 0.range(3)
[0, 1, 2]
yaql> 0.range().take(4)
[0, 1, 2, 3]

switch

Returns the value of the first key-value pair for which condition returned true. If there is no such returns null.

signature: value.switch([args])
callAs: function or method
receiverArg value: value to be used evaluating conditions
argType value: any type
arg [args]: conditions to be checked for the first true
argType [args]: chain of mappings
returnType: any (appropriate value type)

yaql> 15.switch($ < 3 => "a", $ < 7 => "b", $ => "c")
"c"

toList

Returns collection converted to list.

signature: collection.toList()
callAs: method
receiverArg collection: collection to be converted
argType collection: iterable
returnType: list

yaql> range(0, 3).toList()
[0, 1, 2]

tuple

Returns tuple of args.

signature: tuple([args])
callAs: function
arg [args]: chain of values for tuple
argType [args]: chain of any types
returnType: tuple

yaql> tuple(0, [], "a")
[0, [], "a"]

•: If you are a new contributor to Yaql please refer: So You Want to Contribute...

SO YOU WANT TO CONTRIBUTE...

For general information on contributing to OpenStack, please check out the contributor guide to get started. It covers all the basics that are common to all OpenStack projects: the accounts you need, the basics of interacting with our Gerrit review system, how we communicate as a community, etc. Below will cover the more project specific information you need to get started with yaql.

Communication

IRC channel #murano at OFTC
Mailing list (prefix subjects with [murano] for faster responses) http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-discuss

Contacting the Core Team

Please refer the yaql Core Team contacts.

New Feature Planning

yaql features are tracked on Launchpad.

Task Tracking

We track our tasks in Launchpad. If you're looking for some smaller, easier work item to pick up and get started on, search for the 'low-hanging-fruit' tag.

Reporting a Bug

You found an issue and want to make sure we are aware of it? You can do so on Launchpad.

Getting Your Patch Merged

All changes proposed to the yaql project require one or two +2 votes from yaql core reviewers before one of the core reviewers can approve patch by giving Workflow +1 vote.

Project Team Lead Duties

All common PTL duties are enumerated in the PTL guide.

AUTHOR

unknown

COPYRIGHT

2024, OpenStack Foundation

April 16, 2024