icmscript

title: icmscript(7)

summary: The C\-like icmake scripting language

text:
# icmscript(7)

## NAME
icmscript - The C-like icmake scripting language

## DESCRIPTION
Icmake(1) is a generic tool handling program maintenance that can be used as an alternative for make(1). It’s a generic tool in that icmake-scripts, written in a language closely resembling the C programming language, can perform tasks that are traditionally the domain of scripting languages.

Icmake allows programmers to use a programming language (closely resembling the C-programming language) to define the actions that are required for (complex) program maintenance. For this, icmake offers various special operators as well as a set of support functions that have shown their usefulness in program maintenance.

This man-page covers the icmake scripting language in de following sections:

o

DATA TYPES

- int, list, string, and void (for functions);

o

OUTLINE

- outline of icmake scripts: what are their requirements, the structure and organization of their main-functions.

o

PREPROCESSOR DIRECTIVES

- supported preprocessor directives, like #include and #define;

o

PREDEFINED CONSTANTS

- like O_FILE, OFF, and S_IFREG;

o

OPERATORS

- like +, younger, and casts

o

FLOW CONTROL

- if, for, while, etc. (the switch is not available);

o

PREDEFINED FUNCTIONS

- executing programs, changing directories, operations on string and list type variables, etc.. Functions are marked as INT FUNCTIONS, LIST FUNCTIONS, STRING FUNCTIONS

o

USER DEFINED FUNCTIONS

- at least main, with or without its common parameters argc, argv, and envp.

## DATA TYPES
Icmake supports the following five data and value types:

o

ASCII character constants

ASCII character constants are ascii-characters, surrounded by single or double quotes. Single characters (e.g., ’a’) represent the character itself. Standard escape sequences (e.g., ’\n’) are supported and are converted to their well-known values (e.g., ’\n’ represents ascii value 10 (decimal)). Non-standard escape sequences (e.g., ’\x’) are converted to the ascii character following the escape character (so ’\x’ equals ’x’). Escaped sequences consisting of three octal digits represent the ascii character corresponding to the octal value, modulo 256 (e.g., ’\113’ represents ’K’). Escape sequences consisting of an x followed by two hexadecimal digits represent the ascii character corresponding to the hexadecimal value (e.g., ’\x4b’, also representing ’K’);

o

int

Integral values, ranging from -0x8000 through 0x7fff. int constants may be specified as decimal numbers (starting with digits 1 through 9), octal numbers (starting with 0, followed by one or more octal digits), hexadecimal numbers (starting with 0x, followed by one or more hexadecimal digits), or as ASCII character constants;

o

string

Text values: text (or `string’) constants are delimited by double or single quotes. Multiple string constants may be concatenated, but a single string constant may not span multiple lines. Multiple string constants, only separated by white space (i.e., blanks, newlines, comment) are concatenated and are considered one single string constant. To indicate an end-of-line in a string constant use the \n escape sequence;

If arithmetic expressions use at least one int operand then those expressions may also contain single character ASCII constants using double or single quotes. In those cases they represent the ascii-values of their characters.

Conversely, ASCII character constants using single quotes may be used in situations where string operands are expected;

o

list

A list is a data structure containing a series of individually accessible string values. When a list contains elements, its first element has index 0;

Lists may be written to the standard output stream or to file (using printf or fprintf). Lists can also be inserted into string variables using the function strformat. In these cases all (space delimited) elements of the lists are inserted into their destinations;

Lists can also be defined as constants. They consist of an optional series of comma separated string constants surrounded by a pair of square brackets. E.g.,

```
    list words = ["a", "list", "constant"];

```

o

void

The type void is used when defining functions which do not return values. Alternatively, functions may return int, string or list values (cf. section USER DEFINED FUNCTIONS).

Variables can be defined at the global level and inside functions (not only at the top of compound statements but also between statements and in the initialization section of for- and if-statements). When defined inside functions, the standard C scoping and visibility rules apply. Variables are strongly typed, and cannot have type void.

Variables may be initialized when they are defined. Initializations are expressions which may use predefined or user-defined functions, constant values, and values of variables. Functions or variables that are used for initialization must be visible at the initialization point.

## OUTLINE
Icmake scripts require a user-defined function main. The function main has three optional parameters, which may be omitted from the last one (envp) to the first one (argc), like in C. Its full prototype is:

```
    void main(int argc, list argv, list envp)

```

or

```
    int main(int argc, list argv, list envp)

```

When a void main function ends (using a return; statement or when its execution reaches its body’s closing curly) the value 0 is returned to the operating system. When int main functions end using return statements then those statements must be provided with int-expressions. It’s OK when the execution of an int main function reaches its body’s closing curly, in which case 0 is automatically returned to the operating system

In main the parameter

o

argc represents the number of elements in argv;

o

argv contains the arguments, with element 0 being equal to the name of the .bim file, that were passed to the .bim file. The OPTIONS section of the icmake(1) manpage covers how these arguments are forwarded to the icmake script using options -e, -s, and -t.

o

envp contains the `environment’ variables. The function listlen can be used to determine the number of its elements. Elements in envp use the format variable=value. Alternatively, the (predefined) function getenv can be used to retrieve a specific environment variable immediately.

Example (the implementations of the user-defined functions usage, modified, and compile are left as an exercise for the reader):

```
    void main(int argc, list argv)

    {

        if (argc == 1)

            usage(argv[0]);

        if (list toCompile = modified("*.cc"))

        {

            for (int idx = listlen(toCompile); idx--; )

                compile(toCompile[idx]);

        }

    }

```

When executing an icmake script icmake’s run-time support system first initializes all all global variables in the order of their definitions. Following this the function main is called. The script ends when main returns or when the function exit is called by the script.

## PREPROCESSOR DIRECTIVES
Before actually compiling icmake scripts they are first pre-processed by the icmake pre-processor. The pre-processor removes comment, includes files specified by include-directives, and processes #define and comparable directives.

The following preprocessor directives are recognized:

o

comment:

standard C comment (everything from /* through */) as well as comment-to-end-of-line (starting at //, continuing to the end of the line) is ignored;

o

Shell startup: The first line of the icmake-script may start with #!path, where path defines the (usually absolute) location of the icmake program. By making the script executable, it can be called without explicitly calling icmake.

E.g., if the first line of an (executable) icmake script ’icm’ (without extension) contains

```
    #!/usr/bin/icmake -t.

```

then icm can be issued as a command, interpreting the remaining lines of the script as an icmake source which is compiled and then executed by icmake. The icmake(1) man-page’s section EXECUTING ICMAKE SCRIPTS covers the elements of the first line of icmake scripts.

o

\#include "filename"

The file filename is included at the location of the directive;

o

\#include <filename>

The file filename is included at the location of the #include directive; filename is searched in the colon-separated directories specified by the IM environment variable. The first occurrence of filename in the directories specified by the IM environment variable is used;

o

\#define identifier [definition]

The text identifier is replaced by definition. The definition may contain references to already defined identifiers, using the syntax ${identifier}. If the ${identifier} hasn’t been defined (yet), the literal text ${identifier} is used. To prevent infinite recursion at most 100 ${identifier} replacements are accepted;

If the last character on a line is a backslash (\) then definitions continue at the next line. (the backslash is not included in the definition). The preprocessor concatenates double-quoted strings. Double quoted strings themselves may not span multiple lines. Multiple blanks (outside of double quoted strings) in definitions are contracted to a single space character;

Following the #define’s identifier a definition may optional be provided. If omitted, the macro is defined, so it can be used in #if(n)def directives (see below), but in those cases these intentifiers are simply removed from icmake code statements.

o

\#ifdef identifier

If the identifier macro was defined the next block of code (until a matching #else or #endif directive was read) is byte-compiled. Otherwise, the code block is ignored;

o

\#ifndef identifier

If the identifier macro was not defined the next block of code (until a matching #else or #endif directive was detected) is byte-compiled. Otherwise, the code block is ignored;

o

\#else

Terminates #ifdef and #ifndef code blocks, reversing the acceptance decision about the following code block. Only one #else directive can be associated with #if(n)def directives;

o

\#endif

Terminates the code block starting beyond the matching #ifdef, #ifndef or #else directive. Associated #if(n)def, #else, and #endif directives must be specified in the same file;

o

\#undef identifier

Remove identifier from the set of defined symbols. This does not affect the specification of any previously defined symbols using identifier’s definition. If identifier hasn’t been defined a warning is issued.

## PREDEFINED CONSTANTS
The following predefined int constants are available (the functions listed in the intended for column are described in the upcoming sections covering the predefined functions):

symbol

value

intended for

O_ALL

8

makelist

O_DIR

2

makelist

O_FILE

1

makelist

O_SUBDIR

4

makelist

OFF

0

echo

ON

1

echo

P_CHECK

0

system calls

P_NOCHECK

1

system calls

S_IEXEC

32

stat

S_IFCHR

1

stat

S_IFDIR

2

stat

S_IFREG

4

stat

S_IREAD

8

stat

S_IWRITE

16

stat

The following constants are architecture dependent:

symbol

1 when defined on the platform, otherwise 0

unix

Unix, usually with GNU’s gcc compiler

UNIX

may alternatively be available

linux

x86 running Linux (usually with gcc)

LINUX

may alternatively be available

M_SYSV, M_UNIX

x86 running SCO/Unix

_POSIX

_SOURCE Unix with Posix compliant compiler

__hpux

HP-UX, with the native HP compiler

## OPERATORS
Since icmake version 10.00.00 the << operator can be used like the C++ insertion operator. See the description of the functions printf and fprintf below.

int-operators:

All C operators (including the ternary operator) are available (except for pointer operators, as icmake does not support pointers). They operate like their C-programming language’s counterparts. Comparison operators return 1 if the comparison is true, otherwise 0 is returned.

string-operators:

For string variables and/or constants the following operators are available (lhs and rhs are string variables or constants):

o

lhs + rhs: returns a new string value containing the concatenation of strings lhs and rhs. Note that string constants can also directly be concatetated (not using the + operator), e.g., the following two lines both define the string "hello world":

```
    "hello "   "world"

    "hello " + "world"

```

o

lhs += rhs: lhs must be a string variable, to which the string variable or value rhs is appended;

o

string comparisons: operators == != <= >= < > != and == return 1 if the comparison is true, otherwise 0. The ordering operators (like < and >=) use the (case sensitive) character ordering defined by the ASCII character set;

o

!lhs: the boolean ! (not) operator returns 1 if the string lhs is empty, otherwise 0 is returned. Strings merely containing white-space characters are not empty;

o

lhs younger rhs, lhs newer rhs: returns 1 if file lhs is more recent than file rhs. E.g., "source.cc" newer "source.o". The files lhs and rhs do not have to exist:

o

if both don’t exist 0 is returned,

o

if lhs doesn’t exist 0 is returned,

o

if rhs doesn’t exist, 1 is returned,

o

if they are equally old 0 is returned.

The function exists() (see below, section PREDEFINED FUNCTIONS) can be used to test whether a file exists;

o

lhs older rhs: returns 1 if file lhs is older than file rhs. E.g., "libprog.a" older "source.o". The files lhs and rhs do not have to exist:

o

if both don’t exist 0 is returned,

o

if lhs doesn’t exist 1 is returned,

o

if rhs doesn’t exist, 0 is returned,

o

if they are equally old 0 is returned.

o

[]: the index operator returns a character from a string variable or constant. A string is returned as an rvalue. Thus, the following statement compiles OK:

```
    lhs = rhs[3];

```

but the following statement won’t compile (as lhs[3] is an rvalue):

```
    lhs[3] = "a";

```

If an invalid (out of bounds) index value is specified an empty string is returned.

o

The backtick operator (`string cmd`)

A string placed between two backticks is executed as a separate command. Different from the exec and system calls the backtick operator collects the standard output produced by `cmd’ returning this output as a list.

The elements of the list contain the subsequent lines of output (including a final newline, if present) produced by `cmd’. A command that could be executed but that did not produce any output returns a list containing one string element, which is empty.

An empty list indicates that the command could not be executed.

The command’s standard error stream output is ignored by the backtick operator. However, standard shell redirection may be used to collect the standard error stream’s output.

Example:

```
    printf << `"ls"`;   // prints the elements in

                        // the current directory

```

Note that the backtick operator requires a string argument: either a string constant or a string variable.

The function eval(string cmd) behaves exactly like the backtick operator: they are synonyms.

list-operators:

For list variables and/or values the following operators are available:

o

lhs + rhs: returns a new list value containing the concatenation of the values of lists lhs and rhs. This is not a set operation: if an element appears both in lhs and in rhs, then both will appear in the resulting list (set-addition is provided by the function listunion);

o

lhs - rhs: returns a new list value containing the elements in lhs that are not present in rhs. This is a set-difference operation. The ordering of the remaining elements in the returned list is equal to the ordering of those elements in lhs;

o

lhs += rhs: elements in rhs are added to the elements in lhs, which must be a list variable. This is not a set operation;

o

lhs -= rhs: elements in rhs are removed from the elements in lhs. This is a set operation: all elements of lhs that are found in rhs are removed from lhs. The ordering of the remaining elements in lhs is not altered;

o

list equality comparisons: operators != and == may be applied to list values or variables. Operator == returns 1 if both lists have element-by-element identical elements, otherwise 0 is returned. Operator != reverses the result of ==;

o

!lhs: the boolean ! operator returns 1 if the list lhs is empty, otherwise 0 is returned;

o

[]: the index operator retrieves an element from a list variable: it returns a string as an rvalue. Thus, the following statement compiles OK:

```
    // assume lst is a list, str is a string

    str = lst[3];

```

but the following statement won’t compile (as lst[3] is an rvalue):

```
    lst[3] = str;

```

If an invalid (out of bounds) index value is specified an empty string is returned.

casting:

Type-casts using the standard C-style cast-operator can be used to cast:

o

strings to ints and vice versa ((int)"123", (string)55)

If the content of a string does not represent a (decimal) int value 0 the cast returns 0;

o

Strings to lists (list lst = (list)"hello"): this returns a list having one element (hello) (note that casting a string to a list as shown is overkill as list lst = ["hello"] performs the same initialization).

## FLOW CONTROL
Icmake offers a subset of C’s flow control statements. They can be used as in the C programming language.

o

expression ;

The plain expression statement.

Insert-expression statements are defined for the functions fprintf and printf. Expression statements may start with printf << or fprintf << filename <<. The values of all subsequent expressions, separated by << operators (which in this context are called insertion operators) are written to the standard output stream (when using printf <<), or to the file whose name is provided in the string filename (when using fprintf << filename <<). Examples:

```
    printf << "hello" << ’ ’ << "world" << ’\n’;

    fprintf << "out.txt" << "hello" << ’ ’ << "world" << ’\n’;

```

o

The compound statement

Variables may be defined and initialized inside compound statements at locations where expression statements can also be used. The visibility of variables starts at their points of definition;

o

if ([definition;] condition) statement

The [definition;] phrase is optional. If used it defines a type followed by a comma-separated list of variables which may be provided with initialization expressions.

The condition phrase is required, and may define and initialize a variable. E.g,

```
    if (string str = getText())

        process(str);

```

In this example, process is not called if getText() returns an empty string.

Variables defined in the definition and condition phrases do not exist either before or after the if statement.

o

if ([definition;] condition) statement1 else statement2

Acts like the previous statement. If the condition is true statement1 is executed; if the condition is false statement2 is executed;

o

for (init; condition; increment) statement

Variables (of a single type) may be initialized (and optionally defined) in the init section. The condition phrase may define and initialize a variable. The init, condition and increment sections may remain empty. An empty condition section is interpreted as `always true’;

o

while (condition) statement

Inside the condition a variable may be defined and initialized.

A complementary do ... while() statement is not available. Note that when a variable is defined and initialized in the condition section the initialization expression is executed at each iteration of the while statement. Thus the following statement never ends, and displays a never ending stream of values 10:

```
    while (int x = 10)

        printf(x--, "\n");

```

o

return;, and return expression;

Plain return statements can be used in void functions, and return expression statements are used in other type of functions.

o

break

break; statements can only be used in for and while statements, ending those statements;

o

continue

continue; statements can only be used in for and while statements, continuing their next iteration.

## PREDEFINED FUNCTIONS
Icmake provides the following predefined functions, which can be used anywhere in icmake scripts. In the following overview the functions are ordered by categories, and within categories they are ordered alphabetically by function name.

Five categories are distinguished:

o

Functions operating on ints (see INT FUNCTIONS below):

these functions receive int arguments, processing those arguments;

o

Functions operating on strings (see STRING FUNCTIONS below):

these functions operate on the strings which are passed to these functions as arguments;

o

Functions operating on lists (see LIST FUNCTIONS below):

these functions operate on the lists which are passed to these functions as arguments;

o

Functions manipulating file system entries (see FILESYSTEM FUNCTIONS below):

these functions receive the names of file-system entries (files, directories, etc.) as their string arguments.

Note that these functions are not listed in the STRING FUNCTIONS section, as they do not directly operate on their string arguments, but merely use those arguments to identify file system entries.

On the other hand, functions like change_base do not operate on file-system entries and are therefore entries in the STRING FUNCTIONS section;

o

System-related functions (see SYSTEM FUNCTIONS below):

these functions interface to facilities provided by the operating system, like executing programs or changing the script’s environment variables. Some of these functions use specialized support functions, which are also included in this section.

INT FUNCTIONS:

o

string ascii(int value)

returns value as a string: ascii(65) returns the string "A";

o

echo(int opt)

controls echoing of called programs (and their arguments), specify OFF if echoing is not requested. By default echo(ON) is active.

STRING FUNCTIONS:

o

int ascii(string str)

returns the first character of str as an in: ascii("A") returns 65;

o

string change_base(string file, string base)

returns file whose base name is changed into base: change_base("/path/demo.im", "out") returns "/path/out.im";

o

string change_ext(string file, string ext)

returns file whose extension is changed into ext: change_ext("source.cc", "o") returns "source.o". The extension of the returned string is separated from the file’s base name by a single dot (e.g., change_ext("source.", ".cc") returns "source.cc");

o

string change_path(string file, string path)

return file whose path is changed into path: change_path("tmp/binary", "/usr/bin") returns "/usr/bin/binary". To remove the path specify path as an empty string;

o

string element(int index, string var)

acts identically to the index operator: refer to the index ([]) operator in section OPERATORS;

o

string get_base(string file)

returns the base name of file. The base name is the file without its path prefix and without its extension. The extension is all information starting at the final dot in the filename. If no final dot is found, the file name is the base name. E.g., the base name of a.b equals a, the base name of a.b.c equals a.b, the base name of a/b/c equals c;

o

string get_dext(string file)

returns the extension of file, including the separating dot (hence the d in dext). The extension is all information starting at the filename’s final dot. If file does not have a final dot then an empty string is returned;

o

string get_ext(string file)

returns the extension of file, without the separating dot. The extension are all characters in file starting at file’s final dot. If no final dot is found, an empty string is returned;

o

string get_path(string file)

returns file’s path-prefix. The path prefix is all information up to (and including) the final directory separator (which is, depending on the operating system, a forward slash or a backslash). If file does not contain a path-element, then an empty string is returned;

o

string resize(string str, int newlength) returns a copy of string str, resized to newlength characters. If newlength is negative then an empty string is returned, if newlength exceeds str’s length then the newly added characters are initialized to blank spaces;

o

int strchr(string str, string chars)

returns the first index in str where any of the characters in chars is found, or -1 if str does not contain any of the characters in chars;

o

int strfind(string haystack, string needle)

returns index in haystack where needle is found, or -1 if needle is not found in haystack;

o

string strformat(string format, argument(s))

returns a string constructed from the format string containing placeholders %1 .. %2 to refer to arguments following the format string. The specification %1 refers to the first argument following the format string. If fewer arguments than n are provided then additional 0 arguments are provided by icmake. Example:

```
    void main()

    {

        string s2 = = strformat("%1 %2 %1\n", 10, 20);

        printf("s2 = ", s2);        // shows: s2 = 10 20 10

    }

```

o

int strlen(string str)

returns the number of characters in str (not counting the terminating NUL-character);

o

string strlwr(string str)

returns a lower-case duplicate of str;

o

list strtok(string str, string separators)

returns a list containing all substrings of str separated by one or more (consecutive) characters in separators: strtok("hello icmake’s+world", " +") returns a list containing the three strings "hello", "icmake’s", and "world";

o

string strupr(string str)

returns an upper-case duplicate of str.

o

string substr(string text, int offset, int count)

returns a substring of text, starting at offset, consisting of count characters. If offset exceeds (or equals) the string’s length or if count <= 0, then an empty string is returned. If offset is less than 0 then offset = 0 is used. If offset + count exceeds text’s length then the available substring starting at text[offset] is returned (which may be empty);

o

string trim(string str)

returns a copy of str without leading and trailing white spaces;

o

string trimleft(string str)

returns a copy of str without leading white spaces;

o

string trimright(string str)

Returns a copy of str without trailing white spaces.

LIST FUNCTIONS:

o

string element(int index, list var)

acts identically to the index operator: refer to the index ([]) operator in section OPERATORS;

o

int listfind(list lst, string str)

returns the smallest index in lst where the string str is found, or -1 if lst does not contain str;

o

int listlen(list l)

returns the number of elements in list;

o

list listunion(list lhs, list rhs)

returns a list containing the union of the elements in lhs and the elements of rhs. The original order of the elements in lhs is kept. Subsequent elements in rhs that are not available in lhs are added to the end of lhs;

o

list listunion(list lst, string str)

returns a list containing the union of the elements in lst and str. The original order of the elements in lhs is kept. If rhs is not available in lhs then it is added to the end of lhs.

FILESYSTEM FUNCTIONS:

o

string chdir([int check,] string dir)

changes the script’s working directory to dir (which may be specified as absolute or relative to the script’s current working directory). The first argument is optional: if omitted and changing the working directory fails then the icmake-script ends with exit value 1; by specifying P_NOCHECK the function won’t terminate the script but merely returns the script’s current working directory. The script’s working directory after completing the change-dir request is returned as an absolute path, ending in a `/’ directory separator.

Use chdir(".") to merely obtain the current working directory; use chdir("") to change-dir to the script’s startup working directory;

o

int exists(string file)

if file exists, 1 is returned, otherwise 0 is returned;

o

list fgets(string file, list offset)

the next line found at offset value offset[3] is read from file. Pass an empty list to fgets to read file from its beginning.

The returned list has four elements:

o

its first element ([0]) contains the read line (without the line’s \n line terminator);

o

its second element ([1]) contains the line’s \n line terminator (or an empty string if the line was not terminated by a \n);

o

its third element ([2]) contains the string OK if the line was successfully read and FAIL if reading from file failed;

o

its fourth element ([3]) contains the offset beyond the last read byte.

To read multiple lines, pass the returned list as argument to fgets:

```
    list ret;

    while (ret = fgets("filename", ret))

        process(ret);

```

Be careful not to define list ret in while’s condition, as this will reset ret to an empty list at each iteration;

o

int fprintf(string filename, argument(s))

appends all (comma or left-shift (insertion) operator separated) arguments to the file filename. Returns the number of printed arguments.

If the first argument (following filename) contains placeholders (%1, %2, ... %n) then that argument is considered a format string (see also the function strformat in the string functions section for additional information about format strings). Some examples:

```
    fprintf("out", "hello", "world", ’\n’);

    fprintf << "out" << "hello" << "world" << ’\n’;

    fprintf("out", "%1 %2\n", "hello", "world");           // 1

    fprintf << "out" << "hello" << ’ ’ << "world" << ’\n’; // 2

    fprintf << "out" << "%1 %2\n" << "hello" << "world";   // 3

```

When writing statement 1 using insertion operators (cf. the expression statement description in section FLOW CONTROL) statement 2 would normally be encountered, although statement 3, using the format string, would still be accepted;

o

string getch()

returns the next pressed key as a string (pressing the `Enter’-key is not required). The pressed key is not echoed. If the key should be echoed use, e.g., printf(getch());

o

string gets()

returns the next line read from the keyboard as a string. The line contains all entered characters until the `Enter’-key was pressed. The `Enter’-key’s value itself is not stored in the returned string;

o

list makelist([int type = O_FILE], string mask)

the argument type is optional, in which case O_FILE is used. Makelist returns a list of all type file-system entries matching mask. E.g., makelist("*.c") returns a list containing all files ending in .c. For type one of the following set of values can be used to obtain a more specific selection of directory entries: symbol

meaning

O_ALL

obtain all directory entries

O_DIR

obtain all directories, including . and ..

O_FILE

obtain a list of regular files

O_SUBDIR

obtain all directories except for . and ..

In Unix-type operating systems the pattern * does not match entries starting with a dot (hidden entries). To obtain a list of such entries use the pattern .*;

o

list makelist([int type = O_FILE,] string mask, {newer,older,younger}, string comparefile)

the (optional) parameter type may be specified as in the previous variant of makelist. The third parameter must be either newer (or younger) or older. A list of all file-system entries matching mask which are, resp., newer or older than a provided comparefile is returned. Note that newer and younger are operators, not strings;

o

int printf(argument(s))

the function’s (comma or left-shift (insertion) operator separated) arguments are written to the standard output file (cf. the expression statement description in section FLOW CONTROL and this section’s description of the fprintf function). If the first argument contains %1, %2, ... %n specifications then it’s considered a format string (see also the function strformat in the STRING FUNCTIONS section for additional information about format strings). Like fprintf printf returns the number of printed arguments;

o

list stat([int check,] string entry)

Returns stat(2) information of directory entry entry as a list. The first argument is optional: if omitted and calling the system stat function fails then the icmake-script ends with exit value 1; by specifying P_NOCHECK the function won’t terminate the script but returns the return value (-1) of the system stat function.

The returned list has two elements:

its first element ([0]) holds the entry’s attributes. Attributes are returned as the file type and mode of the specified file (cf. stat(2) and inode(7)). E.g.,

```
    S_IRUSR  - owner has read permission

    S_IWUSR  - owner has write permission

    S_IXUSR  - owner has execute permission

    S_IFSOCK - socket

    S_IFLNK  - symbolic link

    S_IFREG  - regular file

    S_IFBLK  - block device

    S_IFDIR  - directory

    S_IFCHR  - character device

    S_IFIFO  - FIFO

```

its second element ([1]) contains the entry’s size in bytes. If P_NOCHECK was specified and ’entry’ doesn’t exists then a list having one element is returned containing -1.

SYSTEM FUNCTIONS:

o

void arghead(string str)

support function of exec() (see also below at exec()): defines the `argument head’ that is used with exec(). By default, the `argument head’ is an empty string. The argument head is text that is prefixed to all exec arguments, like a directory in which provided arguments are found;

o

void argtail (string str)

support function of exec() (see also below at exec()): defines the `argument tail’ that is used with exec(). By default, the `argument tail’ is an empty string. The argument tail is text that is appended to all exec arguments, like the extensions of files that are passed as arguments to exec;

o

cmdhead(string str)

support function of exec() (see also below at exec()). Defines a `command head’ that is used with exec(). By default it is an empty string. It can be used to specify, e.g., compiler options when the arguments themselves are modified by arghead and argtail. The cmdhead argument itself is not modified by arghead or argtail;

o

cmdtail(string str)

support function of exec() (see also below at exec()). Defines a `command tail that is used with exec(). By default it is an empty string. It can be used to specify a final argument (not modified by arghead and argtail);

o

list eval(string str)

this function can be used instead of the backtick operator (cf. section OPERATORS). The example provided with the backtick operator could therefore also have been written like this:

```

    printf << eval("ls");   // prints the elements in the current

                            // directory

```

As mentioned at the backtick operator: the elements of the list contain the subsequent lines of output (including a final newline, if present) produced by `cmd’. A command that could be executed but that did not produce any output returns a list containing one string element, which is empty.

An empty list indicates that the command could not be executed.

o

int exec([int check,] string cmd, argument(s))

Executes the command cmd with (optional) arguments. Each argument is prefixed by arghead and postfixed by argtail. Note that no blanks are inserted between arghead, argument(s), and argtail. The thus modified arguments are concatenated, separated by single blanks. Cmdhead is inserted between cmd and the first argument (delimited by single blanks) and cmdtail is appended to the arguments, separated by a single blank. PATH is searched to locate cmd. 0 is returned.

The first argument is optional: if omitted and the command does not return 0 the icmake script terminates. By specifying P_NOCHECK exec won’t terminate the script but returns the called command’s exit status, or 0x7f00 if the command wasn’t found.

The remaining arguments may be ints, strings or lists. Int and list arguments are cast to strings. Their string representations are then appended to cmd;

o

int execute([int checking,] string cmd, string cmdhead, string arghead, argument(s), string argtail, string cmdtail)

Same functionality as the previous function, but the cmdhead, arghead, argtail, and cmdtail are explicitly specified (and are reset to empty strings after executing cmd);

o

exit(expression)

Ends the execution of an icmake-script. The expression must evaluate to an int value, which is used as the script’s exit value;

o

list getenv(string envvar)

returns the value of environment variable envvar in a list containing two elements:

if the first element ([0]) is "1" then the environment variable was defined;

environment variables are of the form variable=value. If element [0] is "1" then the returned list’s second element [1] holds the value part of the environment variable, which is empty if the environment variable is merely defined;

o

int getpid()

returns the process-id of the icmake byte code interpreter icm-exec;

o

int putenv(string envvar)

adds or modifies envvar to the current icmake-script environment. Use the format: "VAR=value". Use "VAR" to remove "VAR" from the environment. The function returns 0 unless envvar is empty, in which case 1 is returned;

o

int system([int check,] string command)

executes command using the system(3) function. The first argument is optional: if omitted and calling the system(3) function does not return 0 then the icmake-script ends with exit value 1; by specifying P_NOCHECK icmake’s system function won’t terminate the script but returns the return value of the system(3) function (normally the executed command’s exit value). The string command may use redirection and/or piping.

## USER DEFINED FUNCTIONS
In addition to main additional functions can be defined. Once defined, they can be called. Forward referencing of either variables or functions is not supported, but calling functions recursively is. As function declarations are not supported indirect recursion cannot be used.

Function overloading (based on differently typed parameters) is supported.

User-defined functions must have the following elements:

o

The function’s return type, which must be void, int, string or list. There is no default type;

o

The function’s name, e.g., compile;

o

A parameter list, defining zero or more comma-separated parameters. The parameters themselves consist of a type name (int, string, or list) followed by the parameter’s identifier. E.g., (string outfile, string source);

o

A body surrounded by a pair of curly braces ({ and }).

Function bodies may contain variable definitions (optionally initialized at their definitions). Variable definitions start with a type name, followed by one or more comma separated and optionally initialized variable identifiers.

If a variable is not explicitly initialized it is initialized by default: int variables are initialized to 0, string variables are initialized to empty strings ("") and list variables are initialized to empty lists.

Function bodies may also contain zero or more statements (cf. section FLOW CONTROL). Note that variables may be defined (and optionally initialized) anywhere inside functions where expression statements can be used, and also in the condition sections of if, for, and while statements and in the initialization sections of if andd for statements.

## EXAMPLE
In the following example all C++ source files in the current directory are compiled unless their object files are more recent. The main function creates a list of source files and then passes each of them to a function inspect. That function inspects whether the source file is younger than its object file, and if so it calls compile. The function compile uses exec to call the compiler. If a compilation fails the script stops so the error can be repaired.

```
    void compile(string src)

    {

        exec("g++ -c " + src);      // compile ’src’

    }

    void inspect(string src)

    {                               // get the obj-file’s name:

                                    // only compile if necessary

        if (src younger change_ext(src, ".o"))

            compile(src);

    }

    int main()

    {                               // find all .cc source files

        list sources = makelist("*.cc");

        for (                       // visit all source files

            int idx = 0, end = listlen(sources);

                idx != end;

                    ++idx

        )

            inspect(sources[idx]);  // compile if needed

    }
```

## SEE ALSO
icmake(1), icmbuild(1), icmconf(7), icmstart(1), icmstart.rc(7)

## BUGS
Standard comment starting on lines containing preprocessor directives may not extend over multiple lines.

Path names containing blanks are not supported.

## COPYRIGHT
This is free software, distributed under the terms of the GNU General Public License (GPL).

## AUTHOR
Frank B. Brokken (f.b.brokken@rug.nl).


icmstart.rc

package: icmake

title: icmstart.rc(7)

summary: The icmstart(1) resource file

text:
# icmstart.rc(7)

## NAME
icmstart.rc - The icmstart(1) resource file

## DESCRIPTION
The icmstart.rc file specifies files to be installed by icmstart(1).

A default icmstart.rc is found in /etc/icmake/icmstart.rc, but if $HOME/.icmake/icmstart.rc exists then the latter file is used by icmstart(1). $HOME/.icmake/icmstart.rc in turn is overruled by specifying a -c option when invoking icmstart(1).

The default resource file contains the following specifications, preparing for the construction of a C++ program using icmbuild(1):

```
    CLASSES

    icmconf

    P  main.cc

    P  main.ih

    Pb usage.cc

    Pb version.cc

    P ? scanner

    P ? parser

```

The icmstart.rc file may contain: o

empty lines, which are ignored;

o

lines beginning with a hash-character (#), also ignored;

o

an optional installation mode followed by a source-destination specification.

## INSTALLATION MODE
An installation mode consists of a combination of:

o

either a P (don’t install with icmstart xxx library) or an L (don’t install with icmstart xxx program);

o

a D, indicating that the source must also be installed by default, i.e., if neither `program’ or `library’ was specified as second argument;

o

a b, indicating that the file must not be installed if the -b (basic installation) option was specified when calling icmstart;

o

Following the optional P, L, D, or b character a space delimited optional ? may be specified. If specified the installation of the file or directory must be confirmed by the user.

## SOURCE-DESTINATION SPECIFICATIONS
The following source-destination specifications can be used in an icmstart.rc file (using skeletons below to refer to icmstart’s skeleton files directory):

o

source

a file named source must exist in skeletons. It is installed in the destination directory `dest’ that’s specified when calling icmstart(1). Example:

```
    CLASSES

```

skeletons/CLASSES is installed as dest/CLASSES;

o

pathspec

if `pathspec’ does not begin with a slash it must exist in skeletons. It is installed as `pathspec’ in the destination directory specified when calling icmstart(1). Example:

```
    dir/file

```

skeletons/file is installed as dest/dir/file;

o

/pathspec or ~/pathspec

the ~-character is expanded to the user’s home directory. The pathspec’s final element is installed in the destination directory specified when calling icmstart(1). Example:

```
    ~/.icmake/file

```

$HOME/.icmake/file is installed as dest/file

When the above source specifications are followed by a destination specification destspec (a file or non-absolute directory specification) then the source specification is installed as dest/destspec below the destination directory specified when calling icmstart(1). Examples:

```
    CLASSES CLASSES             # skeletons/CLASSES is installed as

                                #   dest/CLASSES

    dir/file dir/file           # skeletons/dir/file is installed as

                                #   dest/dir/file

    ~/.icmake/file .icmake/file # $HOME/.icmake/file is installed as

                                #   dest/.icmake/file

```

Icmstart converts destination specifications to absolute paths. If these absolute paths do not begin with dest’s absolute path then an error message is displayed and icmstart ends.

## FINE-TUNING ICMCONF
The default icmconf file contains #defines which are used when a project uses a parser (by default generated by bisonc++(1)) and/or a scanner (by default generated by flexc++(1)). All those #defines are commented out in the default icmconf file. They are:

```
    #define PARSER_DIR          ""

    #define PARSGEN             "bisonc++"

    #define PARSFLAGS           "-V"

    #define PARSSPEC            "grammar"

    #define PARSFILES           ""

    #define PARSOUT             "parse.cc"

    #define SCANNER_DIR         ""

    #define SCANGEN             "flexc++"

    #define SCANFLAGS           ""

    #define SCANSPEC            "lexer"

    #define SCANFILES            ""

    #define SCANOUT             "lex.cc"

```

However, the icmstart.rc file can be modified so that these #defines are omitted from the project’s icmconf file unless a parser and/or scanner is actually used by the project. For this:

o

copy the files in /usr/share/icmake to ~/.icmake;

o

if the program uses a parser then define the file ~/.icmake/parser/icmconf containing

```
    #define PARSER_DIR          "parser"

    #define PARSGEN             "bisonc++"

    #define PARSFLAGS           "-V"

    #define PARSSPEC            "grammar"

    #define PARSFILES           ""

    #define PARSOUT             "parse.cc"

```

(here `parser’ is the name of the directory containing the parser’s implementation; optionally another directory name can be specified);

Remove the parser-defines from ~/.icmake/icmconf;

In ~/.icmake/icmconf/icmstart.rc replace the parser-directory specification, like

```
    ? parser    parser

```

by

```
    ? ~/.icmake/parser    parser

```

o

if the program uses a scanner then define the file ~/.icmake/scanner/icmconf containing

```
    #define SCANNER_DIR         "scanner"

    #define SCANGEN             "flexc++"

    #define SCANFLAGS           ""

    #define SCANSPEC            "lexer"

    #define SCANFILES            ""

    #define SCANOUT             "lex.cc"

```

(here `scanner’ is the name of the directory containing the scanner’s implementation; optionally another directory name can be specified);n() Remove the scanner-defines from ~/.icmake/icmconf;

In ~/.icmake/icmconf/icmstart.rc replace the scanner-directory specification, like

```
    ? scanner    scanner

```

by

```
    ? ~/.icmake/scanner    scanner

```

In general: icmconf #defines can be redefined by removing them from ~/.icmake/icmconf, specifying their redefinitions in icmconf files in subdirectories of ~/.icmake/icmconf.

## FILES
The mentioned paths are sugestive only and may be installation dependent:

o

/usr/share/icmake/icmconf: example of an icmbuild configuration file;

o

/usr/share/icmake/CLASSES: example of an icmbuild CLASSES file;

o

/usr/share/icmake/icmstart.rc: default skeleton resource file.

## SEE ALSO
bisonc++(1), flexc++(1), icmake(1), icmbuild(1), icmconf(7), icmstart(1)

## BUGS
None reported

## COPYRIGHT
This is free software, distributed under the terms of the GNU General Public License (GPL).

## AUTHOR
Frank B. Brokken (f.b.brokken@rug.nl).

branch: sisyphus


integrity

package: ima-evm-integrity-check

title: integrity(7)

summary: IMA and EVM system integrity control

text:
# integrity(7)

## NAME
ima-evm-integrity - IMA and EVM system integrity control

## SYNOPSIS

```
integrity-applier
```

```
integrity-remover
```

## DESCRIPTION
Basic usage:

1. Run integrity-applier command. The system reloads to so-called Stage II with ima_appraise=fix kernel option.

2. Run integrity-applier command again to sign the files. Use --cert and --key command-line options to sign files with user-defined keys (use -B BASENAME | --basename=BASENAME option to install the given user-defined certificate file under non-default filename). After the sign procedure is done the system reloads to IMA enforced mode (i. e. with ima_appraise=enforce kernel command-line option).

3. In order to disable IMA, run the integrity-remover script.

By default the file signing log is written to /var/log/integrity-sign.log.

The trusted IMA keyring

In order to use the trusted IMA keyring select keyring name ".ima" using the IMA_KEYRING=.ima configuration option in the /etc/integrity/config file. The secondary trusted keyring certificates can be configured with the SECONDARY_SUFFIX configuration option.

Note that in order to make the IMA trusted keyring available the kernel needs to be compiled with CONFIG_INTEGRITY_TRUSTED_KEYRING=y and CONFIG_IMA_KEYRINGS_PERMIT_SIGNED_BY_BUILTIN_OR_SECONDARY=y options set.

Advanced usage of the integrity-applier utility:

* Use --log=FILE option to output file signing log to specified FILE at Stage II. Use --log-stderr option to get it output to the current error stream.

* Use --init and --sign to explicitly select Stage I or Stage II.

* Use -R | --no-reboot to suppress automatic reboot of the system.

* Use -a HASH | --hash=HASH option to select the type of cryptographic hash function to sign the files with. It can also be configured in /etc/sysconfig/integrity (and/or /etc/sysconfig/integrity).

* Use -r | --resign option together with --cert and --key options to overwrite existing IMA and EVM signatures.

* Use -B BASENAME | --basename=BASENAME option to specify the base name of the certificate file that is installed in /etc/keys/ (see also the CERT_BASENAME configuration option).

* Use -A | --auto option to automatically run Stage II after reboot using the special ima-signing.target.

* Visit /etc/integrity/config and /etc/sysconfig/integrity to configure the directories with files to sign and other options, such as EVM. Contents of /etc/sysconfig/integrity override values set in /etc/integrity/config.

* Enable ima-check.service to check IMA policy at system boot. Add services to ima-check-failed.target to run actions when the check fails.

* Touch /etc/integrity/reboot-on-initrd-error in order not to load the system at all if loading of IMA policy fails at initrd stage. Also, use /etc/integrity/on-initrd-error script to run it instead of reboot.

* Override the default integrity policy with /etc/integrity/policy.

## CONFIGURATION FILE
Options to configure in the /etc/integrity/config and /etc/sysconfig/integrity configuration files are as follows:

* HASH_ALGO -- select one of the supported hash algorithms: sha512, streebog256 and streebog512;

* GOST_PARAMSET -- a default GOST param-set ("paramset:A") can be overriden;

* LIBDIRS -- space-separated paths to search for library (.so) files;

* EXECLIBDIRS -- space-separated paths to additional executable files;

* LIB_EXT -- select suffices for additional files to sign (searched in /var/lib/ and LIBDIRS, a space-separated list);

* LOG_FILE -- define path to log file (the default is /var/log/integrity-sign.log);

* IMA_KEYRING -- set to ".ima" in order to use the trusted keyring;

* CERT_BASENAME -- select the base name of the certificate file that is installed in the /etc/keys/ directory (the default is x509, can also be specified on the command-line with the -B BASENAME | --basename=BASENAME option);

* SECONDARY_SUFFIX -- select suffix for intermediate authority certificate files to load into the secondary trusted kernel keyring (the default is "_ca");

* PROTECT_KEYS -- set to nonzero in order to protect the keys form removal (may not work with the trusted keyrings);

* PROTECT_KEYRINGS -- set to nonzero in order to protect the keyring itself from key removal (may not work with the trusted keyrings);

* WITH_EVM -- set to nonempty value in order to enable EVM; Use a hex value (i. e.: 0x...) to specify the EVM mode;

* CERT -- path to a user-defined certificate (it's better to use the --cert command-line option);

* PRIVKEY -- path to a user-defined private key (it's better to use the --key command-line option).

* RESIGN -- set to nonempty value in order to overwrite existing IMA and EVM signatures.

## BUGS
Report bugs to https://bugzilla.altlinux.org/.

## COPYRIGHT
Copyright © BaseALT, Ltd. License GPLv2+: GNU GPL version 2 or later See <http://gnu.org/licenses/gpl.html> for more information.

This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.

## SEE ALSO
integrity-applier(8), integrity-remover(8)


tc-hfsc

title: tc-hfsc(7)

summary: Hierarchical Fair Service Curve

text:
# tc-hfsc(7)

## NAME
tc-hfcs - Hierarchical Fair Service Curve

## HISTORY & INTRODUCTION
HFSC (Hierarchical Fair Service Curve) is a network packet scheduling algorithm that was first presented at SIGCOMM'97. Developed as a part of ALTQ (ALTernative Queuing) on NetBSD, found its way quickly to other BSD systems, and then a few years ago became part of the linux kernel. Still, it's not the most popular scheduling algorithm - especially if compared to HTB - and it's not well documented for the enduser. This introduction aims to explain how HFSC works without using too much math (although some math it will be inevitable).

In short HFSC aims to:

1)

guarantee precise bandwidth and delay allocation for all leaf classes (realtime criterion)

2)

allocate excess bandwidth fairly as specified by class hierarchy (linkshare & upperlimit criterion)

3)

minimize any discrepancy between the service curve and the actual amount of service provided during linksharing

The main "selling" point of HFSC is feature (1), which is achieved by using nonlinear service curves (more about what it actually is later). This is particularly useful in VoIP or games, where not only a guarantee of consistent bandwidth is important, but also limiting the initial delay of a data stream. Note that it matters only for leaf classes (where the actual queues are) - thus class hierarchy is ignored in the realtime case.

Feature (2) is well, obvious - any algorithm featuring class hierarchy (such as HTB) strives to achieve that. HFSC does that well, although you might end with unusual situations, if you define service curves carelessly - see section CORNER CASES for examples.

Feature (3) is mentioned due to the nature of the problem. There may be situations where it's either not possible to guarantee service of all curves at the same time, and/or it's impossible to do so fairly. Both will be explained later. Note that this is mainly related to interior (aka aggregate) classes, as the leafs are already handled by (1). Still, it's perfectly possible to create a leaf class without realtime service, and in such a case the caveats will naturally extend to leaf classes as well.

## ABBREVIATIONS
For the remaining part of the document, we'll use following shortcuts:

```
RT - realtime
LS - linkshare
UL - upperlimit
SC - service curve
```

```

```

## BASICS OF HFSC
To understand how HFSC works, we must first introduce a service curve. Overall, it's a nondecreasing function of some time unit, returning the amount of service (an allowed or allocated amount of bandwidth) at some specific point in time. The purpose of it should be subconsciously obvious: if a class was allowed to transfer not less than the amount specified by its service curve, then the service curve is not violated.

Still, we need more elaborate criterion than just the above (although in the most generic case it can be reduced to it). The criterion has to take two things into account:

- idling periods

- the ability to "look back", so if during current active period the service curve is violated, maybe it isn't if we count excess bandwidth received during earlier active period(s)

Let's define the criterion as follows:

(1)

```
For each t1, there must exist t0 in set B, so S(t1-t0) <= w(t0,t1)
```

Here 'w' denotes the amount of service received during some time period between t0 and t1. B is a set of all times, where a session becomes active after idling period (further denoted as 'becoming backlogged'). For a clearer picture, imagine two situations:

a)

our session was active during two periods, with a small time gap between them

b)

as in (a), but with a larger gap

Consider (a): if the service received during both periods meets (1), then all is well. But what if it doesn't do so during the 2nd period? If the amount of service received during the 1st period is larger than the service curve, then it might compensate for smaller service during the 2nd period and the gap - if the gap is small enough.

If the gap is larger (b) - then it's less likely to happen (unless the excess bandwidth allocated during the 1st part was really large). Still, the larger the gap - the less interesting is what happened in the past (e.g. 10 minutes ago) - what matters is the current traffic that just started.

From HFSC's perspective, more interesting is answering the following question: when should we start transferring packets, so a service curve of a class is not violated. Or rephrasing it: How much X() amount of service should a session receive by time t, so the service curve is not violated. Function X() defined as below is the basic building block of HFSC, used in: eligible, deadline, virtual-time and fit-time curves. Of course, X() is based on equation (1) and is defined recursively:

- At the 1st backlogged period beginning function X is initialized to generic service curve assigned to a class

- At any subsequent backlogged period, X() is:

```
min(X() from previous period ; w(t0)+S(t-t0) for t>=t0),
```

... where t0 denotes the beginning of the current backlogged period.

HFSC uses either linear, or two-piece linear service curves. In case of linear or two-piece linear convex functions (first slope < second slope), min() in X's definition reduces to the 2nd argument. But in case of two-piece concave functions, the 1st argument might quickly become lesser for some t>=t0. Note, that for some backlogged period, X() is defined only from that period's beginning. We also define X^(-1)(w) as smallest t>=t0, for which X(t) = w. We have to define it this way, as X() is usually not an injection.

The above generic X() can be one of the following:

E()

In realtime criterion, selects packets eligible for sending. If none are eligible, HFSC will use linkshare criterion. Eligible time 'et' is calculated with reference to packets' heads ( et = E^(-1)(w) ). It's based on RT service curve, but in case of a convex curve, uses its 2nd slope only.

D()

In realtime criterion, selects the most suitable packet from the ones chosen by E(). Deadline time 'dt' corresponds to packets' tails (dt = D^(-1)(w+l), where 'l' is packet's length). Based on RT service curve.

V()

In linkshare criterion, arbitrates which packet to send next. Note that V() is function of a virtual time - see LINKSHARE CRITERION section for details. Virtual time 'vt' corresponds to packets' heads (vt = V^(-1)(w)). Based on LS service curve.

F()

An extension to linkshare criterion, used to limit at which speed linkshare criterion is allowed to dequeue. Fit-time 'ft' corresponds to packets' heads as well (ft = F^(-1)(w)). Based on UL service curve.

Be sure to make clean distinction between session's RT, LS and UL service curves and the above "utility" functions.

## REALTIME CRITERION
RT criterion ignores class hierarchy and guarantees precise bandwidth and delay allocation. We say that a packet is eligible for sending, when the current real time is later than the eligible time of the packet. From all eligible packets, the one most suited for sending is the one with the shortest deadline time. This sounds simple, but consider the following example:

Interface 10Mbit, two classes, both with two-piece linear service curves:

- 1st class - 2Mbit for 100ms, then 7Mbit (convex - 1st slope < 2nd slope)

- 2nd class - 7Mbit for 100ms, then 2Mbit (concave - 1st slope > 2nd slope)

Assume for a moment, that we only use D() for both finding eligible packets, and choosing the most fitting one, thus eligible time would be computed as D^(-1)(w) and deadline time would be computed as D^(-1)(w+l). If the 2nd class starts sending packets 1 second after the 1st class, it's of course impossible to guarantee 14Mbit, as the interface capability is only 10Mbit. The only workaround in this scenario is to allow the 1st class to send the packets earlier that would normally be allowed. That's where separate E() comes to help. Putting all the math aside (see HFSC paper for details), E() for RT concave service curve is just like D(), but for the RT convex service curve - it's constructed using only RT service curve's 2nd slope (in our example

7Mbit).

The effect of such E() - packets will be sent earlier, and at the same time D() will be updated - so the current deadline time calculated from it will be later. Thus, when the 2nd class starts sending packets later, both the 1st and the 2nd class will be eligible, but the 2nd session's deadline time will be smaller and its packets will be sent first. When the 1st class becomes idle at some later point, the 2nd class will be able to "buffer" up again for later active period of the 1st class.

A short remark - in a situation, where the total amount of bandwidth available on the interface is larger than the allocated total realtime parts (imagine a 10 Mbit interface, but 1Mbit/2Mbit and 2Mbit/1Mbit classes), the sole speed of the interface could suffice to guarantee the times.

Important part of RT criterion is that apart from updating its D() and E(), also V() used by LS criterion is updated. Generally the RT criterion is secondary to LS one, and used only if there's a risk of violating precise realtime requirements. Still, the "participation" in bandwidth distributed by LS criterion is there, so V() has to be updated along the way. LS criterion can than properly compensate for non-ideal fair sharing situation, caused by RT scheduling. If you use UL service curve its F() will be updated as well (UL service curve is an extension to LS one - see UPPERLIMIT CRITERION section).

Anyway - careless specification of LS and RT service curves can lead to potentially undesired situations (see CORNER CASES for examples). This wasn't the case in HFSC paper where LS and RT service curves couldn't be specified separately.

## LINKSHARING CRITERION
LS criterion's task is to distribute bandwidth according to specified class hierarchy. Contrary to RT criterion, there're no comparisons between current real time and virtual time - the decision is based solely on direct comparison of virtual times of all active subclasses - the one with the smallest vt wins and gets scheduled. One immediate conclusion from this fact is that absolute values don't matter - only ratios between them (so for example, two children classes with simple linear 1Mbit service curves will get the same treatment from LS criterion's perspective, as if they were 5Mbit). The other conclusion is, that in perfectly fluid system with linear curves, all virtual times across whole class hierarchy would be equal.

Why is VC defined in term of virtual time (and what is it)?

Imagine an example: class A with two children - A1 and A2, both with let's say 10Mbit SCs. If A2 is idle, A1 receives all the bandwidth of A (and update its V() in the process). When A2 becomes active, A1's virtual time is already far later than A2's one. Considering the type of decision made by LS criterion, A1 would become idle for a long time. We can workaround this situation by adjusting virtual time of the class becoming active - we do that by getting such time "up to date". HFSC uses a mean of the smallest and the biggest virtual time of currently active children fit for sending. As it's not real time anymore (excluding trivial case of situation where all classes become active at the same time, and never become idle), it's called virtual time.

Such approach has its price though. The problem is analogous to what was presented in previous section and is caused by non-linearity of service curves:

1)

either it's impossible to guarantee service curves and satisfy fairness during certain time periods:

Recall the example from RT section, slightly modified (with 3Mbit slopes instead of 2Mbit ones):

- 1st class - 3Mbit for 100ms, then 7Mbit (convex - 1st slope < 2nd slope)

- 2nd class - 7Mbit for 100ms, then 3Mbit (concave - 1st slope > 2nd slope)

They sum up nicely to 10Mbit - the interface's capacity. But if we wanted to only use LS for guarantees and fairness - it simply won't work. In LS context, only V() is used for making decision which class to schedule. If the 2nd class becomes active when the 1st one is in its second slope, the fairness will be preserved - ratio will be 1:1 (7Mbit:7Mbit), but LS itself is of course unable to guarantee the absolute values themselves - as it would have to go beyond of what the interface is capable of.

2)

and/or it's impossible to guarantee service curves of all classes at the same time [fairly or not]:

This is similar to the above case, but a bit more subtle. We will consider two subtrees, arbitrated by their common (root here) parent:

```
R (root) - 10Mbit
A  - 7Mbit, then 3Mbit
A1 - 5Mbit, then 2Mbit
A2 - 2Mbit, then 1Mbit
B  - 3Mbit, then 7Mbit
```

R arbitrates between left subtree (A) and right (B). Assume that A2 and B are constantly backlogged, and at some later point A1 becomes backlogged (when all other classes are in their 2nd linear part).

What happens now? B (choice made by R) will always get 7 Mbit as R is only (obviously) concerned with the ratio between its direct children. Thus A subtree gets 3Mbit, but its children would want (at the point when A1 became backlogged) 5Mbit + 1Mbit. That's of course impossible, as they can only get 3Mbit due to interface limitation.

In the left subtree - we have the same situation as previously (fair split between A1 and A2, but violated guarantees), but in the whole tree - there's no fairness (B got 7Mbit, but A1 and A2 have to fit together in 3Mbit) and there's no guarantees for all classes (only B got what it wanted). Even if we violated fairness in the A subtree and set A2's service curve to 0, A1 would still not get the required bandwidth.

## UPPERLIMIT CRITERION
UL criterion is an extensions to LS one, that permits sending packets only if current real time is later than fit-time ('ft'). So the modified LS criterion becomes: choose the smallest virtual time from all active children, such that fit-time < current real time also holds. Fit-time is calculated from F(), which is based on UL service curve. As you can see, its role is kinda similar to E() used in RT criterion. Also, for obvious reasons - you can't specify UL service curve without LS one.

The main purpose of the UL service curve is to limit HFSC to bandwidth available on the upstream router (think adsl home modem/router, and linux server as NAT/firewall/etc. with 100Mbit+ connection to mentioned modem/router). Typically, it's used to create a single class directly under root, setting a linear UL service curve to available bandwidth - and then creating your class structure from that class downwards. Of course, you're free to add a UL service curve (linear or not) to any class with LS criterion.

An important part about the UL service curve is that whenever at some point in time a class doesn't qualify for linksharing due to its fit-time, the next time it does qualify it will update its virtual time to the smallest virtual time of all active children fit for linksharing. This way, one of the main things the LS criterion tries to achieve - equality of all virtual times across whole hierarchy - is preserved (in perfectly fluid system with only linear curves, all virtual times would be equal).

Without that, 'vt' would lag behind other virtual times, and could cause problems. Consider an interface with a capacity of 10Mbit, and the following leaf classes (just in case you're skipping this text quickly - this example shows behavior that doesn't happen):

```
A - ls 5.0Mbit
B - ls 2.5Mbit
C - ls 2.5Mbit, ul 2.5Mbit
```

If B was idle, while A and C were constantly backlogged, A and C would normally (as far as LS criterion is concerned) divide bandwidth in 2:1 ratio. But due to UL service curve in place, C would get at most 2.5Mbit, and A would get the remaining 7.5Mbit. The longer the backlogged period, the more the virtual times of A and C would drift apart. If B became backlogged at some later point in time, its virtual time would be set to (A's vt + C's vt)/2, thus blocking A from sending any traffic until B's virtual time catches up with A.

## SEPARATE LS / RT SCs
Another difference from the original HFSC paper is that RT and LS SCs can be specified separately. Moreover, leaf classes are allowed to have only either RT SC or LS SC. For interior classes, only LS SCs make sense: any RT SC will be ignored.

## CORNER CASES
Separate service curves for LS and RT criteria can lead to certain traps that come from "fighting" between ideal linksharing and enforced realtime guarantees. Those situations didn't exist in original HFSC paper, where specifying separate LS / RT service curves was not discussed.

Consider an interface with a 10Mbit capacity, with the following leaf classes:

```
A - ls 5.0Mbit, rt 8Mbit
B - ls 2.5Mbit
C - ls 2.5Mbit
```

Imagine A and C are constantly backlogged. As B is idle, A and C would divide bandwidth in 2:1 ratio, considering LS service curve (so in theory - 6.66 and 3.33). Alas RT criterion takes priority, so A will get 8Mbit and LS will be able to compensate class C for only 2 Mbit - this will cause discrepancy between virtual times of A and C.

Assume this situation lasts for a long time with no idle periods, and suddenly B becomes active. B's virtual time will be updated to (A's vt + C's vt)/2, effectively landing in the middle between A's and C's virtual time. The effect - B, having no RT guarantees, will be punished and will not be allowed to transfer until C's virtual time catches up.

If the interface had a higher capacity, for example 100Mbit, this example would behave perfectly fine though.

Let's look a bit closer at the above example - it "cleverly" invalidates one of the basic things LS criterion tries to achieve - equality of all virtual times across class hierarchy. Leaf classes without RT service curves are literally left to their own fate (governed by messed up virtual times).

Also, it doesn't make much sense. Class A will always be guaranteed up to 8Mbit, and this is more than any absolute bandwidth that could happen from its LS criterion (excluding trivial case of only A being active). If the bandwidth taken by A is smaller than absolute value from LS criterion, the unused part will be automatically assigned to other active classes (as A has idling periods in such case). The only "advantage" is, that even in case of low bandwidth on average, bursts would be handled at the speed defined by RT criterion. Still, if extra speed is needed (e.g. due to latency), non linear service curves should be used in such case.

In the other words: the LS criterion is meaningless in the above example.

You can quickly "workaround" it by making sure each leaf class has RT service curve assigned (thus guaranteeing all of them will get some bandwidth), but it doesn't make it any more valid.

Keep in mind - if you use nonlinear curves and irregularities explained above happen only in the first segment, then there's little wrong with "overusing" RT curve a bit:

```
A - ls 5.0Mbit, rt 9Mbit/30ms, then 1Mbit
B - ls 2.5Mbit
C - ls 2.5Mbit
```

Here, the vt of A will "spike" in the initial period, but then A will never get more than 1Mbit until B & C catch up. Then everything will be back to normal.

## LINUX AND TIMER RESOLUTION
In certain situations, the scheduler can throttle itself and setup so called watchdog to wakeup dequeue function at some time later. In case of HFSC it happens when for example no packet is eligible for scheduling, and UL service curve is used to limit the speed at which LS criterion is allowed to dequeue packets. It's called throttling, and accuracy of it is dependent on how the kernel is compiled.

There're 3 important options in modern kernels, as far as timers' resolution goes: 'tickless system', 'high resolution timer support' and 'timer frequency'.

If you have 'tickless system' enabled, then the timer interrupt will trigger as slowly as possible, but each time a scheduler throttles itself (or any other part of the kernel needs better accuracy), the rate will be increased as needed / possible. The ceiling is either 'timer frequency' if 'high resolution timer support' is not available or not compiled in, or it's hardware dependent and can go far beyond the highest 'timer frequency' setting available.

If 'tickless system' is not enabled, the timer will trigger at a fixed rate specified by 'timer frequency' - regardless if high resolution timers are or aren't available.

This is important to keep those settings in mind, as in scenario like: no tickless, no HR timers, frequency set to 100hz - throttling accuracy would be at 10ms. It doesn't automatically mean you would be limited to ~0.8Mbit/s (assuming packets at ~1KB) - as long as your queues are prepared to cover for timer inaccuracy. Of course, in case of e.g. locally generated UDP traffic - appropriate socket size is needed as well. Short example to make it more understandable (assume hardcore anti-schedule settings - HZ=100, no HR timers, no tickless):

```
tc qdisc add dev eth0 root handle 1:0 hfsc default 1
tc class add dev eth0 parent 1:0 classid 1:1 hfsc rt m2 10Mbit
```

Assuming packet of ~1KB size and HZ=100, that averages to ~0.8Mbit - anything beyond it (e.g. the above example with specified rate over 10x larger) will require appropriate queuing and cause bursts every ~10 ms. As you can imagine, any HFSC's RT guarantees will be seriously invalidated by that. Aforementioned example is mainly important if you deal with old hardware - as is particularly popular for home server chores. Even then, you can easily set HZ=1000 and have very accurate scheduling for typical adsl speeds.

Anything modern (apic or even hpet msi based timers + 'tickless system') will provide enough accuracy for superb 1Gbit scheduling. For example, on one of my cheap dual-core AMD boards I have the following settings:

```
tc qdisc add dev eth0 parent root handle 1:0 hfsc default 1
tc class add dev eth0 parent 1:0 classid 1:1 hfsc rt m2 300mbit
```

And a simple:

```
nc -u dst.host.com 54321 </dev/zero
nc -l -p 54321 >/dev/null
```

...will yield the following effects over a period of ~10 seconds (taken from /proc/interrupts):

```
319: 42124229   0  HPET_MSI-edge  hpet2 (before)
319: 42436214   0  HPET_MSI-edge  hpet2 (after 10s.)
```

That's roughly 31000/s. Now compare it with HZ=1000 setting. The obvious drawback of it is that cpu load can be rather high with servicing that many timer interrupts. The example with 300Mbit RT service curve on 1Gbit link is particularly ugly, as it requires a lot of throttling with minuscule delays.

Also note that it's just an example showing the capabilities of current hardware. The above example (essentially a 300Mbit TBF emulator) is pointless on an internal interface to begin with: you will pretty much always want a regular LS service curve there, and in such a scenario HFSC simply doesn't throttle at all.

300Mbit RT service curve (selected columns from mpstat -P ALL 1):

```
10:56:43 PM  CPU  %sys     %irq   %soft   %idle
10:56:44 PM  all  20.10    6.53   34.67   37.19
10:56:44 PM    0  35.00    0.00   63.00    0.00
10:56:44 PM    1   4.95   12.87    6.93   73.27
```

So, in the rare case you need those speeds with only a RT service curve, or with a UL service curve: remember the drawbacks.

## CAVEAT: RANDOM ONLINE EXAMPLES
For reasons unknown (though well guessed), many examples you can google love to overuse UL criterion and stuff it in every node possible. This makes no sense and works against what HFSC tries to do (and does pretty damn well). Use UL where it makes sense: on the uppermost node to match upstream router's uplink capacity. Or in special cases, such as testing (limit certain subtree to some speed), or customers that must never get more than certain speed. In the last case you can usually achieve the same by just using a RT criterion without LS+UL on leaf nodes.

As for the router case - remember it's good to differentiate between "traffic to router" (remote console, web config, etc.) and "outgoing traffic", so for example:

```
tc qdisc add dev eth0 root handle 1:0 hfsc default 0x8002
tc class add dev eth0 parent 1:0 classid 1:999 hfsc rt m2 50Mbit
tc class add dev eth0 parent 1:0 classid 1:1 hfsc ls m2 2Mbit ul m2 2Mbit
```

... so "internet" tree under 1:1 and "router itself" as 1:999

## LAYER2 ADAPTATION
Please refer to tc-stab(8)

## SEE ALSO
tc(8), tc-hfsc(8), tc-stab(8)

Please direct bugreports and patches to: <netdev@vger.kernel.org>

## AUTHOR
Manpage created by Michal Soltys (soltys@ziu.info)


irda

package: irda-utils

title: irda(7)

summary: The Linux-IrDA protocol stack

text:
# irda(7)

## NAME
irda - The Linux-IrDA protocol stack

## SYNOPSIS
\#include <linux/irda.h>

stream_s = socket(PF_INET, SOCK_STREAM, 0);

seqpkt_s = socket(PF_INET, SOCK_SEQPACKET, 0);

dgram_s = socket(PF_INET, SOCK_DGRAM, IRDAPROTO_UNITDATA);

ultra_s = socket(PF_INET, SOCK_DGRAM, IRDAPROTO_ULTRA);

## DESCRIPTION
Linux-IrDA is a complete implementation of the various IrDA protocols and IrDA drivers for Linux.

The configuration of the low level of the IrDA stack (IrDA ports) and the various IrDA drivers are described in irattach(8).

The usage of the higher layers of the IrDA stack (IrCOMM, IrLPT, IrNET, IrLAN, OBEX...) are described in their respective man pages.

The core of the Linux-IrDA protocol stack is composed of the IrLAP, IrLMP, IrTTP and IrIAS protocols and is implemented as a kernel module. The programmer's interface for the IrDA stack is socket based. For more information on sockets, see socket(7).

Note that higher level IrDA protocols may offer a different programmer's interface. For example IrCOMM offers a TTY based interface and OBEX is available through a user space library.

## SOCKETS TYPE
The Linux-IrDA socket interface offers 4 types of sockets:

Stream socket - SOCK_STREAM, 0

Stream sockets provides a reliable, stream oriented, full duplex connection between two sockets on top of IrTTP, similar to TCP sockets. IrTTP guarantees that the data arrives in order and manages flow contol, IrLAP retransmits lost packets.

Stream sockets does not preserve record boundaries.

SeqPacket socket - SOCK_SEQPACKET, 0

SeqPacket sockets provides a reliable, datagram oriented, full duplex connection between two sockets on top of IrTTP. IrTTP guarantees that the data arrives in order and manages flow contol, IrLAP retransmits lost packets.

SeqPacket sockets preserve record boundaries. Large datagrams will be fragmented as needed.

Datagram socket - SOCK_DGRAM, IRDAPROTO_UNITDATA

SeqPacket sockets provides a reliable, datagram oriented, full duplex connection between two sockets on top of IrLMP. There is no guarantees that the data arrives in order and there is no flow contol, however IrLAP retransmits lost packets.

Datagram sockets preserve record boundaries. No fragmentation is provided, datagrams larger than the IrDA link MTU are truncated or discarded.

Ultra socket - SOCK_DGRAM, IRDAPROTO_ULTRA

Ultra sockets provides a connectionless, unreliable datagram packet service on top of IrLAP-Ultra. Ultra datagrams are sent and receive only when no IrLAP connection is established and otherwise discarded. They are always transmitted at 9600 bauds. Datagrams may be lost, reordered, and there is no flow control.

Ultra sockets preserve record boundaries. No fragmentation is provided, datagrams larger than the Ultra MTU (382 bytes) are truncated or discarded.

## ADDRESS FORMAT
An IrDA socket address is defined as a combination of an IrDA interface address and a service name. Service names are automatically resolved to lsap through the IrIAP protocol.

```
    struct sockaddr_irda {

        sa_family_t sir_family;   /* AF_IRDA */

        __u8        sir_lsap_sel; /* LSAP selector */

        __u32       sir_addr;     /* Device address */

        char        sir_name[25]; /* Usually <service>:IrDA:TinyTP */

    };
```

sin_family is always set to AF_IRDA. sir_lsap_sel is usually not used. sir_addr is the address of the peer and optional (and that case the first peer discoverd will be used). sir_name is the service name of the socket.

## SOCKET OPTIONS
IrDA sockets supports some protocol specific socket options that can be set with setsockopt(2) and read with getsockopt(2). The socket option level for IrDA is SOL_IRLMP.

Options that can be set:

IRLMP_IAS_SET

Set an entry in the local IAS database.

IRLMP_IAS_DEL

Delete an entry from the local IAS database.

IRLMP_MAX_SDU_SIZE

Set maximum receive MTU for SeqPacket sockets.

IRLMP_HINTS_SET

Set IrDA hint bits to be advertised to peers.

IRLMP_HINT_MASK_SET

Set IrDA hint bits filter to be used when performing discovery.

Options that can be get:

IRLMP_ENUMDEVICES

Perform link level discovery and return the content of the discovery log.

IRLMP_MAX_SDU_SIZE

Get maximum receive MTU.

IRLMP_IAS_GET

Get an entry from the local IAS database.

IRLMP_IAS_QUERY

Query an entry in IAS database of a peer using IrIAP.

IRLMP_WAITDEVICE

Block until a new device is discovered. Experimental feature, use with caution.

## AUTHOR
Jean Tourrilhes - jt@hpl.hp.com

## SEE ALSO
irattach(8), irdadump(8), irnet(4), irnetd(8).


iwd.debug

package: iwd

title: iwd.debug(7)

text:
# iwd.debug(7)

## Name
iwd.debug - Debugging information for wireless daemon

## SYNOPSIS
Debugging information for wireless daemon

## DESCRIPTION
Common methods of obtaining extra debugging information.

## ENVIRONMENT
$IWD_RTNL_DEBUG set to 1 enables RTNL debugging.

$IWD_GENL_DEBUG set to 1 enables printing Generic Netlink communication with the kernel.

$IWD_DHCP_DEBUG set to 1 enables DHCP debugging.

$IWD_TLS_DEBUG set to 1 enables TLS debugging.

$IWD_WSC_DEBUG_KEYS set to 1 enables printing received WSC keys.

The granularity of the DHCP logging can also be tailored by setting $IWD_DHCP_DEBUG to the maximum verbosity level, "debug", "info", "warn", or "error".

## SEE ALSO
iwd(8), iwmon(1)

## Author
Marcel Holtmann <marcel@holtmann.org>, Denis Kenzior <denkenz@gmail.com>, Andrew Zaborowski <andrew.zaborowski@intel.com>, Tim Kourt <tim.a.kourt@linux.intel.com>, James Prestwood <prestwoj@gmail.com>

## Copyright
2013-2019 Intel Corporation


package: jurand

title: java_remove_annotations(7)

text:
# java_remove_annotations(7)

## NAME
java_remove_annotations - remove imports and annotations from Java source files

## SYNOPSIS
%java_remove_annotations [optional flags] <matcher>... [file path]...

## DESCRIPTION
This macro removes import statements as well as usage of annotations from Java source files. This does the same as java_remove_imports and on top of that finds all uses of annotations and matches the content between the '@' symbol and either ending with whitespace or with an opening parethesis '(' (the annotation arguments).

In case of match, the script also removes the block of paretheses that follows the matched annotation, if it is present.

File path arguments are handled the following way:

•Symlinks are ignored.

•Regular files are handled regardless of the file name.

•Directories are traversed recursively and all .java files are handled.

Arguments can be specified in arbitrary order.

Matcher is one of:

-n <name>

Simple class name to be matched against the simple names of imported symbols. Names are matched for exact equality. Can be specified multiple times.

-p <pattern>

Regex patterns to be matched against imported symbols. Each imported symbol found in the code is matched against each pattern. Can be specified multiple times.

Optional flags:

-s, --strict

Fail if any of the user provided arguments were redundant.

## EXAMPLES
Example of usage in a .spec file:

%java_remove_annotations src -n Nullable

Examples of patterns:

•Annotations present in Java source file:

```
1) @SuppressWarnings
2) @SuppressFBWarnings(value = {"EI_EXPOSE_REP", "EI_EXPOSE_REP2"})
3) @org.junit.Test
4) @org.junit.jupiter.api.Test
```

•Names used to match:

```
Name 'SuppressWarnings' matches 1)
Name 'Test' matches 3), 4).
```

```
Name 'junit' does not match anything.
```

•Patterns used to match:

```
Pattern 'SuppressWarnings' matches 1).
Pattern 'Suppress' matches 1), 2).
Pattern 'org[.]junit[.]Test' matches 3).
Pattern 'junit' matches 3), 4).
```

```
Pattern '@SuppressWarnings' does not match anything.
Pattern 'EI_EXPOSE_REP' does not match anything.
```

## REPORTING BUGS
Bugs should be reported through the issue tracker at GitHub: <https://github.com/fedora-java/jurand/issues>.

## AUTHOR
Written by Marián Konček.

## SEE ALSO
java_remove_imports(7).


title: java_remove_imports(7)

text:
# java_remove_imports(7)

## NAME
java_remove_imports - remove import statements from Java source files

## SYNOPSIS
%java_remove_imports [optional flags] <matcher>... [file path]...

## DESCRIPTION
This macro removes import statements from Java source files. The script matches all non-whitespace content following the 'import [static]' statement against all patterns provided via the -p flag and all simple class names against names provided by the -n flag.

File path arguments are handled the following way:

•Symlinks are ignored.

•Regular files are handled regardless of the file name.

•Directories are traversed recursively and all .java files are handled.

Arguments can be specified in arbitrary order.

Matcher is one of:

-n <name>

Simple class name to be matched against the simple names of imported symbols. Names are matched for exact equality. Can be specified multiple times.

-p <pattern>

Regex patterns to be matched against imported symbols. Each imported symbol found in the code is matched against each pattern. Can be specified multiple times.

Optional flags:

-s, --strict

Fail if any of the user provided arguments were redundant.

## EXAMPLES
Example of usage in a .spec file:

%java_remove_imports src -n Nullable

Examples of patterns:

•Import statements present in Java source file:

```
1) import java.lang.Runnable;
2) import java.util.List;
3) import static java.util.*;
4) import static java.lang.String.valueOf;
5) import com.google.common.util.concurrent.Service;
```

•Names used to match:

```
Name 'Runnable' matches 1)
Name 'String' matches 4)
```

```
Name 'util' does not match anything.
Name '*' does not match anything.
Name 'valueOf' does not match anything.
```

•Patterns used to match:

```
Pattern 'Runnable' matches 1).
Pattern '[*]' matches 3).
Pattern 'java[.]util' matches 2), 3).
Pattern 'util' matches 2), 3), 5).
Patterns 'java', 'java.*' match 1) - 4).
```

```
Pattern 'static' does not match anything.
```

## REPORTING BUGS
Bugs should be reported through the issue tracker at GitHub: <https://github.com/fedora-java/jurand/issues>.

## AUTHOR
Written by Marián Konček.

## SEE ALSO
java_remove_annotations(7).


asymmetric-key

package: keyutils

title: asymmetric-key(7)

text:
# asymmetric-key(7)

## NAME
asymmetric - Kernel key type for holding asymmetric keys

## OVERVIEW
A kernel key of asymmetric type acts as a handle to an asymmetric key as used for public-key cryptography. The key material itself may be held inside the kernel or it may be held in hardware with operations being offloaded. This prevents direct user access to the cryptographic material.

Keys may be any asymmetric type (RSA, ECDSA, ...) and may have both private and public components present or just the public component.

Asymmetric keys can be made use of by both the kernel and userspace. The kernel can make use of them for module signature verification and kexec image verification for example. Userspace is provided with a set of keyctl(KEYCTL_PKEY_*) calls for querying and using the key. These are wrapped by libkeyutils as functions named keyctl_pkey_*().

An asymmetric-type key can be loaded by the keyctl utility using a command line like:

```
openssl x509 -in key.x509 -outform DER |
keyctl padd asymmetric foo @s
```

## DESCRIPTION
The asymmetric-type key can be viewed as a container that comprises of a number of components:

Parsers

The asymmetric key parsers attempt to identify the content of the payload blob and extract useful data from it with which to instantiate the key. The parser is only used when adding, instantiating or updating a key and isn't thereafter associated with the key.

Available parsers include ones that can deal with DER-encoded X.509, DER-encoded PKCS#8 and DER-encoded TPM-wrapped blobs.

Public and private keys

These are the cryptographic components of the key pair. The public half should always be available, but the private half might not be. What operations are available can be queried, as can the size of the key. The key material may or may not actually reside in the kernel.

Identifiers

In addition to the normal key description (which can be generated by the parser), a number of supplementary identifiers may be available that can be searched for. These may be obtained, for example, by hashing the public key material or from the subjectKeyIdentifier in an X.509 certificate.

Identifier-based searches are selected by passing as the description to keyctl_search() a string constructed of hex characters prefixed with either "id:" or "ex:". The "id:" prefix indicates that a partial tail match is permissible whereas "ex:" requires an exact match on the full string. The hex characters indicate the data to match.

Subtype

This is the driver inside the kernel that accesses the key material and performs operations on it. It might be entirely software-based or it may offload the operations to a hardware key store, such as a TPM.

Note that expiry times from the payload are ignored as these patches may be used during boot before the system clock is set.

## PARSERS
The asymmetric key parsers can handle keys in a number of forms:

X.509

DER-encoded X.509 certificates can be accepted. Two identifiers are constructed: one from from the certificate issuer and serial number and the other from the subjectKeyIdentifier, if present. If left blank, the key description will be filled in from the subject field plus either the subjectKeyIdentifier or the serialNumber. Only the public key is filled in and only the encrypt and verify operations are supported.

The signature on the X.509 certificate may be checked by the keyring it is being added to and it may also be rejected if the key is blacklisted.

PKCS#8

Unencrypted DER-encoded PKCS#8 key data containers can be accepted. Currently no identifiers are constructed. The private key and the public key are loaded from the PKCS#8 blobs. Encrypted PKCS#8 is not currently supported.

TPM-Wrapped keys

DER-encoded TPM-wrapped TSS key blobs can be accepted. Currently no identifiers are constructed. The public key is extracted from the blob but the private key is expected to be resident in the TPM. Encryption and signature verification is done in software, but decryption and signing are offloaded to the TPM so as not to expose the private key.

This parser only supports TPM-1.2 wrappings and enc=pkcs1 encoding type. It also uses a hard-coded null SRK password; password-protected SRKs are not yet supported.

## USERSPACE API
In addition to the standard keyutils library functions, such as keyctl_update(), there are five calls specific to the asymmetric key type (though they are open to being used by other key types also):

keyctl_pkey_query()

keyctl_pkey_encrypt()

keyctl_pkey_decrypt()

keyctl_pkey_sign()

keyctl_pkey_verify()

The query function can be used to retrieve information about an asymmetric key, such as the key size, the amount of space required by buffers for the other operations and which operations are actually supported.

The other operations form two pairs: encrypt/decrypt and create/verify signature. Not all of these operations will necessarily be available; typically, encrypt and verify only require the public key to be available whereas decrypt and sign require the private key as well.

All of these operations take an information string parameter that supplies additional information such as encoding type/form and the password(s) needed to unlock/unwrap the key. This takes the form of a comma-separated list of "key[=value]" pairs, the exact set of which depends on the subtype driver used by a particular key.

Available parameters include:

enc=<type>

The encoding type for use in an encrypted blob or a signature. An example might be "enc=pkcs1".

hash=<name>

The name of the hash algorithm that was used to digest the data to be signed. Note that this is only used to construct any encoding that is used in a signature. The data to be signed or verified must have been parsed by the caller and the hash passed to keyctl_pkey_sign() or keyctl_pkey_verify() beforehand. An example might be "hash=sha256".

Note that not all parameters are used by all subtypes.

## RESTRICTED KEYRINGS
An additional keyutils function, keyctl_restrict_keyring(), can be used to gate a keyring so that a new key can only be added to the affected keyring if (a) it's an asymmetric key, (b) it's validly signed by a key in some appropriate keyring and (c) it's not blacklisted.

```
 keyctl_restrict_keyring(keyring, "asymmetric",

                         "key_or_keyring:<signing-key>[:chain]");
```

Where <signing-key> is the ID of a key or a ring of keys that act as the authority to permit a new key to be added to the keyring. The chain flag indicates that keys that have been added to the keyring may also be used to verify new keys. Authorising keys must themselves be asymmetric-type keys that can be used to do a signature verification on the key being added.

Note that there are various system keyrings visible to the root user that may permit additional keys to be added. These are typically gated by keys that already exist, preventing unauthorised keys from being used for such things as module verification.

## BLACKLISTING
When the attempt is made to add a key to the kernel, a hash of the public key is checked against the blacklist. This is a system keyring named .blacklist and contains keys of type blacklist. If the blacklist contains a key whose description matches the hash of the new key, that new key will be rejected with error EKEYREJECTED.

The blacklist keyring may be loaded from multiple sources, including a list compiled into the kernel and the UEFI dbx variable. Further hashes may also be blacklisted by the administrator. Note that blacklisting is not retroactive, so an asymmetric key that is already on the system cannot be blacklisted by adding a matching blacklist entry later.

## VERSIONS
The asymmetric key type first appeared in v3.7 of the Linux kernel, the restriction function in v4.11 and the public key operations in v4.20.

## SEE ALSO
keyctl(1), add_key(2), keyctl(3), keyctl_pkey_encrypt(3), keyctl_pkey_query(3), keyctl_pkey_sign(3), keyrings(7), keyutils(7)


keyutils

title: keyutils(7)

summary: in-kernel key management utilities

text:
# keyutils(7)

## NAME
keyutils - in-kernel key management utilities

## DESCRIPTION
The keyutils package is a library and a set of utilities for accessing the kernel keyrings facility.

A header file is supplied to provide the definitions and declarations required to access the library:

\#include <keyutils.h>

To link with the library, the following:

-lkeyutils

should be specified to the linker.

Three system calls are provided:

add_key(2)

Supply a new key to the kernel.

request_key(2)

Find an existing key for use, or, optionally, create one if one does not exist.

keyctl(2)

Control a key in various ways. The library provides a variety of wrappers around this system call and those should be used rather than calling it directly.

See the add_key(2), request_key(2), and keyctl(2) manual pages for more information.

The keyctl() wrappers are listed on the keyctl(3) manual page.

## UTILITIES
A program is provided to interact with the kernel facility by a number of subcommands, e.g.:

keyctl add user foo bar @s

See the keyctl(1) manual page for information on that.

The kernel has the ability to upcall to userspace to fabricate new keys. This can be triggered by request_key(), but userspace is better off using add_key() instead if it possibly can.

The upcalling mechanism is usually routed via the request-key(8) program. What this does with any particular key is configurable in:

/etc/request-key.conf

/etc/request-key.d/

See the request-key.conf(5) and the request-key(8) manual pages for more information.

## SEE ALSO
keyctl(1), keyctl(3), keyrings(7), persistent-keyring(7), process-keyring(7), session-keyring(7), thread-keyring(7), user-keyring(7), user-session-keyring(7), pam_keyinit(8)


kryoptic

package: kryoptic

title: kryoptic(7)

summary: A PKCS#11 soft token written in Rust

text:
# kryoptic(7)

## NAME
kryoptic - A PKCS#11 soft token written in Rust

## SYNOPSIS
libkryoptic_pkcs11.so

## DESCRIPTION
Kryoptic is a PKCS#11 software token implemented in Rust. It utilizes OpenSSL for cryptographic operations and provides support for multiple storage backends, including SQLite and NSS DB. It is distributed as a dynamic library (libkryoptic_pkcs11.so) that can be loaded by applications expecting a standard PKCS#11 module.

Kryoptic aims to provide a modern, secure, and flexible software token. It can be built to use the system OpenSSL dynamically or can be statically linked. It also includes support for FIPS 140-3 builds (when linked against OpenSSL’s libfips.a), restricting algorithms and enforcing FIPS approved behaviors.

## INITIALIZATION
Applications initialize the Kryoptic token via the standard PKCS#11 C_Initialize() function. Kryoptic can process custom configurations passed through the reserved argument in C_Initialize(), allowing flexible initialization methods:

- Config file path: e.g., kryoptic_conf=/path/to/config.toml

- Legacy SQLite path: e.g., /path/to/database.sql

- NSS config directory: e.g., configDir=/etc/pki/nssdb

If no explicit arguments are provided, Kryoptic falls back to searching for its TOML configuration file in predefined system and user locations.

## ENVIRONMENT VARIABLES
Several environment variables affect the runtime behavior of Kryoptic:

KRYOPTIC_CONF

The path to the Kryoptic configuration file. This has the highest precedence.

XDG_CONFIG_HOME

Used as a fallback if KRYOPTIC_CONF is not set. Kryoptic will look for ${XDG_CONFIG_HOME}/kryoptic/token.conf.

HOME

Used as a fallback if XDG_CONFIG_HOME is not set. Kryoptic will look for ${HOME}/.config/kryoptic/token.conf.

KRYOPTIC_EC_POINT_ENCODING

Can be used to override the default ec_point_encoding specified in the configuration file. Valid values are BYTES or DER.

## SEE ALSO
kryoptic.conf(5), softhsm_migrate(1)