ossl-guide-quic-client-non-block

title: ossl-guide-quic-client-non-block(7ossl)

summary: guide\-quic\-client\-non\-block

text:
# ossl-guide-quic-client-non-block(7ossl)

## NAME
ossl-guide-quic-client-non-block - OpenSSL Guide: Writing a simple nonblocking QUIC client

## SIMPLE NONBLOCKING QUIC CLIENT EXAMPLE
This page will build on the example developed on the ossl-guide-quic-client-block(7) page which demonstrates how to write a simple blocking QUIC client. On this page we will amend that demo code so that it supports nonblocking functionality.

The complete source code for this example nonblocking QUIC client is available in the demos/guide directory of the OpenSSL source distribution in the file quic-client-non-block.c. It is also available online at <https://github.com/openssl/openssl/blob/master/demos/guide/quic-client-non-block.c>.

As we saw in the previous example an OpenSSL QUIC application always uses a nonblocking socket. However, despite this, the SSL object still has blocking behaviour. When the SSL object has blocking behaviour then this means that it waits (blocks) until data is available to read if you attempt to read from it when there is no data yet. Similarly it waits when writing if the SSL object is currently unable to write at the moment. This can simplify the development of code because you do not have to worry about what to do in these cases. The execution of the code will simply stop until it is able to continue. However in many cases you do not want this behaviour. Rather than stopping and waiting your application may need to go and do other tasks whilst the SSL object is unable to read/write, for example updating a GUI or performing operations on some other connection or stream.

We will see later in this tutorial how to change the SSL object so that it has nonblocking behaviour. With a nonblocking SSL object, functions such as SSL_read_ex(3) or SSL_write_ex(3) will return immediately with a non-fatal error if they are currently unable to read or write respectively.

Since this page is building on the example developed on the ossl-guide-quic-client-block(7) page we assume that you are familiar with it and we only explain how this example differs.

### Performing work while waiting for the socket
In a nonblocking application you will need work to perform in the event that we want to read or write to the SSL object but we are currently unable to. In fact this is the whole point of using a nonblocking SSL object, i.e. to give the application the opportunity to do something else. Whatever it is that the application has to do, it must also be prepared to come back and retry the operation that it previously attempted periodically to see if it can now complete. Ideally it would only do this in the event that something has changed such that it might succeed on the retry attempt, but this does not have to be the case. It can retry at any time.

Note that it is important that you retry exactly the same operation that you tried last time. You cannot start something new. For example if you were attempting to write the text "Hello World" and the operation failed because the SSL object is currently unable to write, then you cannot then attempt to write some other text when you retry the operation.

In this demo application we will create a helper function which simulates doing other work. In fact, for the sake of simplicity, it will do nothing except wait for the state of the underlying socket to change or until a timeout expires after which the state of the SSL object might have changed. We will call our function wait_for_activity().

```
    static void wait_for_activity(SSL *ssl)
    {
        fd_set wfds, rfds;
        int width, sock, isinfinite;
        struct timeval tv;
        struct timeval *tvp = NULL;
        /* Get hold of the underlying file descriptor for the socket */
        sock = SSL_get_fd(ssl);
        FD_ZERO(&wfds);
        FD_ZERO(&rfds);
        /*
         * Find out if we would like to write to the socket, or read from it (or
         * both)
         */
        if (SSL_net_write_desired(ssl))
            FD_SET(sock, &wfds);
        if (SSL_net_read_desired(ssl))
            FD_SET(sock, &rfds);
        width = sock + 1;
        /*
         * Find out when OpenSSL would next like to be called, regardless of
         * whether the state of the underlying socket has changed or not.
         */
        if (SSL_get_event_timeout(ssl, &tv, &isinfinite) && !isinfinite)
            tvp = &tv;
        /*
         * Wait until the socket is writeable or readable. We use select here
         * for the sake of simplicity and portability, but you could equally use
         * poll/epoll or similar functions
         *
         * NOTE: For the purposes of this demonstration code this effectively
         * makes this demo block until it has something more useful to do. In a
         * real application you probably want to go and do other work here (e.g.
         * update a GUI, or service other connections).
         *
         * Let's say for example that you want to update the progress counter on
         * a GUI every 100ms. One way to do that would be to use the timeout in
         * the last parameter to "select" below. If the tvp value is greater
         * than 100ms then use 100ms instead. Then, when select returns, you
         * check if it did so because of activity on the file descriptors or
         * because of the timeout. If the 100ms GUI timeout has expired but the
         * tvp timeout has not then go and update the GUI and then restart the
         * "select" (with updated timeouts).
         */
        select(width, &rfds, &wfds, NULL, tvp);
}
```

If you are familiar with how to write nonblocking applications in OpenSSL for TLS (see ossl-guide-tls-client-non-block(7)) then you should note that there is an important difference here between the way a QUIC application and a TLS application works. With a TLS application if we try to read or write something to the SSL object and we get a "retry" response (SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE) then we can assume that is because OpenSSL attempted to read or write to the underlying socket and the socket signalled the "retry". With QUIC that is not the case. OpenSSL may signal retry as a result of an SSL_read_ex(3) or SSL_write_ex(3) (or similar) call which indicates the state of the stream. This is entirely independent of whether the underlying socket needs to retry or not.

To determine whether OpenSSL currently wants to read or write to the underlying socket for a QUIC application we must call the SSL_net_read_desired(3) and SSL_net_write_desired(3) functions.

It is also important with QUIC that we periodically call an I/O function (or otherwise call the SSL_handle_events(3) function) to ensure that the QUIC connection remains healthy. This is particularly important with a nonblocking application because you are likely to leave the SSL object idle for a while while the application goes off to do other work. The SSL_get_event_timeout(3) function can be used to determine what the deadline is for the next time we need to call an I/O function (or call SSL_handle_events(3)).

An alternative to using SSL_get_event_timeout(3) to find the next deadline that OpenSSL must be called again by is to use "thread assisted" mode. In "thread assisted" mode OpenSSL spawns an additional thread which will periodically call SSL_handle_events(3) automatically, meaning that the application can leave the connection idle safe in the knowledge that the connection will still be maintained in a healthy state. See "Creating the SSL_CTX and SSL objects" below for further details about this.

In this example we are using the "select" function to check the readability/writeability of the socket because it is very simple to use and is available on most Operating Systems. However you could use any other similar function to do the same thing. "select" waits for the state of the underlying socket(s) to become readable/writeable or until the timeout has expired before returning.

### Handling errors from OpenSSL I/O functions
A QUIC application that has been configured for nonblocking behaviour will need to be prepared to handle errors returned from OpenSSL I/O functions such as SSL_read_ex(3) or SSL_write_ex(3). Errors may be fatal for the stream (for example because the stream has been reset or because the underlying connection has failed), or non-fatal (for example because we are trying to read from the stream but no data has not yet arrived from the peer for that stream).

SSL_read_ex(3) and SSL_write_ex(3) will return 0 to indicate an error and SSL_read(3) and SSL_write(3) will return 0 or a negative value to indicate an error. SSL_shutdown(3) will return a negative value to incidate an error.

In the event of an error an application should call SSL_get_error(3) to find out what type of error has occurred. If the error is non-fatal and can be retried then SSL_get_error(3) will return SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE depending on whether OpenSSL wanted to read to or write from the stream but was unable to. Note that a call to SSL_read_ex(3) or SSL_read(3) can still generate SSL_ERROR_WANT_WRITE. Similarly calls to SSL_write_ex(3) or SSL_write(3) might generate SSL_ERROR_WANT_READ.

Another type of non-fatal error that may occur is SSL_ERROR_ZERO_RETURN. This indicates an EOF (End-Of-File) which can occur if you attempt to read data from an SSL object but the peer has indicated that it will not send any more data on the stream. In this case you may still want to write data to the stream but you will not receive any more data.

Fatal errors that may occur are SSL_ERROR_SYSCALL and SSL_ERROR_SSL. These indicate that the stream is no longer usable. For example, this could be because the stream has been reset by the peer, or because the underlying connection has failed. You can consult the OpenSSL error stack for further details (for example by calling ERR_print_errors(3) to print out details of errors that have occurred). You can also consult the return value of SSL_get_stream_read_state(3) to determine whether the error is local to the stream, or whether the underlying connection has also failed. A return value of SSL_STREAM_STATE_RESET_REMOTE tells you that the stream has been reset by the peer and SSL_STREAM_STATE_CONN_CLOSED tells you that the underlying connection has closed.

In our demo application we will write a function to handle these errors from OpenSSL I/O functions:

```
    static int handle_io_failure(SSL *ssl, int res)
    {
        switch (SSL_get_error(ssl, res)) {
        case SSL_ERROR_WANT_READ:
        case SSL_ERROR_WANT_WRITE:
            /* Temporary failure. Wait until we can read/write and try again */
            wait_for_activity(ssl);
            return 1;
        case SSL_ERROR_ZERO_RETURN:
            /* EOF */
            return 0;
        case SSL_ERROR_SYSCALL:
            return -1;
        case SSL_ERROR_SSL:
            /*
             * Some stream fatal error occurred. This could be because of a
             * stream reset - or some failure occurred on the underlying
             * connection.
             */
            switch (SSL_get_stream_read_state(ssl)) {
            case SSL_STREAM_STATE_RESET_REMOTE:
                printf("Stream reset occurred\n");
                /*
                 * The stream has been reset but the connection is still
                 * healthy.
                 */
                break;
            case SSL_STREAM_STATE_CONN_CLOSED:
                printf("Connection closed\n");
                /* Connection is already closed. */
                break;
            default:
                printf("Unknown stream failure\n");
                break;
            }
            /*
             * If the failure is due to a verification error we can get more
             * information about it from SSL_get_verify_result().
             */
            if (SSL_get_verify_result(ssl) != X509_V_OK)
                printf("Verify error: %s\n",
                    X509_verify_cert_error_string(SSL_get_verify_result(ssl)));
            return -1;
        default:
            return -1;
        }
    }
```

This function takes as arguments the SSL object that represents the connection, as well as the return code from the I/O function that failed. In the event of a non-fatal failure, it waits until a retry of the I/O operation might succeed (by using the wait_for_activity() function that we developed in the previous section). It returns 1 in the event of a non-fatal error (except EOF), 0 in the event of EOF, or -1 if a fatal error occurred.

### Creating the SSL_CTX and SSL objects
In order to connect to a server we must create SSL_CTX and SSL objects for this. Most of the steps to do this are the same as for a blocking client and are explained on the ossl-guide-quic-client-block(7) page. We won't repeat that information here.

One key difference is that we must put the SSL object into nonblocking mode (the default is blocking mode). To do that we use the SSL_set_blocking_mode(3) function:

```
    /*
     * The underlying socket is always nonblocking with QUIC, but the default
     * behaviour of the SSL object is still to block. We set it for nonblocking
     * mode in this demo.
     */
    if (!SSL_set_blocking_mode(ssl, 0)) {
        printf("Failed to turn off blocking mode\n");
        goto end;
    }
```

Although the demo application that we are developing here does not use it, it is possible to use "thread assisted mode" when developing QUIC applications. Normally, when writing an OpenSSL QUIC application, it is important that SSL_handle_events(3) (or alternatively any I/O function) is called on the connection SSL object periodically to maintain the connection in a healthy state. See "Performing work while waiting for the socket" for more discussion on this. This is particularly important to keep in mind when writing a nonblocking QUIC application because it is common to leave the SSL connection object idle for some time when using nonblocking mode. By using "thread assisted mode" a separate thread is created by OpenSSL to do this automatically which means that the application developer does not need to handle this aspect. To do this we must use OSSL_QUIC_client_thread_method(3) when we construct the SSL_CTX as shown below:

```
    ctx = SSL_CTX_new(OSSL_QUIC_client_thread_method());
    if (ctx == NULL) {
        printf("Failed to create the SSL_CTX\n");
        goto end;
    }
```

### Performing the handshake
As in the demo for a blocking QUIC client we use the SSL_connect(3) function to perform the handshake with the server. Since we are using a nonblocking SSL object it is very likely that calls to this function will fail with a non-fatal error while we are waiting for the server to respond to our handshake messages. In such a case we must retry the same SSL_connect(3) call at a later time. In this demo we do this in a loop:

```
    /* Do the handshake with the server */
    while ((ret = SSL_connect(ssl)) != 1) {
        if (handle_io_failure(ssl, ret) == 1)
            continue; /* Retry */
        printf("Failed to connect to server\n");
        goto end; /* Cannot retry: error */
    }
```

We continually call SSL_connect(3) until it gives us a success response. Otherwise we use the handle_io_failure() function that we created earlier to work out what we should do next. Note that we do not expect an EOF to occur at this stage, so such a response is treated in the same way as a fatal error.

### Sending and receiving data
As with the blocking QUIC client demo we use the SSL_write_ex(3) function to send data to the server. As with SSL_connect(3) above, because we are using a nonblocking SSL object, this call could fail with a non-fatal error. In that case we should retry exactly the same SSL_write_ex(3) call again. Note that the parameters must be exactly the same, i.e. the same pointer to the buffer to write with the same length. You must not attempt to send different data on a retry. An optional mode does exist (SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER) which will configure OpenSSL to allow the buffer being written to change from one retry to the next. However, in this case, you must still retry exactly the same data - even though the buffer that contains that data may change location. See SSL_CTX_set_mode(3) for further details. As in the TLS tutorials (ossl-guide-tls-client-block(7)) we write the request in three chunks.

```
    /* Write an HTTP GET request to the peer */
    while (!SSL_write_ex(ssl, request_start, strlen(request_start), &written)) {
        if (handle_io_failure(ssl, 0) == 1)
            continue; /* Retry */
        printf("Failed to write start of HTTP request\n");
        goto end; /* Cannot retry: error */
    }
    while (!SSL_write_ex(ssl, hostname, strlen(hostname), &written)) {
        if (handle_io_failure(ssl, 0) == 1)
            continue; /* Retry */
        printf("Failed to write hostname in HTTP request\n");
        goto end; /* Cannot retry: error */
    }
    while (!SSL_write_ex(ssl, request_end, strlen(request_end), &written)) {
        if (handle_io_failure(ssl, 0) == 1)
            continue; /* Retry */
        printf("Failed to write end of HTTP request\n");
        goto end; /* Cannot retry: error */
    }
```

On a write we do not expect to see an EOF response so we treat that case in the same way as a fatal error.

Reading a response back from the server is similar:

```
    do {
        /*
         * Get up to sizeof(buf) bytes of the response. We keep reading until
         * the server closes the connection.
         */
        while (!eof && !SSL_read_ex(ssl, buf, sizeof(buf), &readbytes)) {
            switch (handle_io_failure(ssl, 0)) {
            case 1:
                continue; /* Retry */
            case 0:
                eof = 1;
                continue;
            case -1:
            default:
                printf("Failed reading remaining data\n");
                goto end; /* Cannot retry: error */
            }
        }
        /*
         * OpenSSL does not guarantee that the returned data is a string or
         * that it is NUL terminated so we use fwrite() to write the exact
         * number of bytes that we read. The data could be non-printable or
         * have NUL characters in the middle of it. For this simple example
         * we're going to print it to stdout anyway.
         */
        if (!eof)
            fwrite(buf, 1, readbytes, stdout);
    } while (!eof);
    /* In case the response didn't finish with a newline we add one now */
    printf("\n");
```

The main difference this time is that it is valid for us to receive an EOF response when trying to read data from the server. This will occur when the server closes down the connection after sending all the data in its response.

In this demo we just print out all the data we've received back in the response from the server. We continue going around the loop until we either encounter a fatal error, or we receive an EOF (indicating a graceful finish).

### Shutting down the connection
As in the QUIC blocking example we must shutdown the connection when we are finished with it.

Even though we have received EOF on the stream that we were reading from above, this tell us nothing about the state of the underlying connection. Our demo application will initiate the connection shutdown process via SSL_shutdown(3).

Since our application is initiating the shutdown then we might expect to see SSL_shutdown(3) give a return value of 0, and then we should continue to call it until we receive a return value of 1 (meaning we have successfully completed the shutdown). Since we are using a nonblocking SSL object we might expect to have to retry this operation several times. If SSL_shutdown(3) returns a negative result then we must call SSL_get_error(3) to work out what to do next. We use our handle_io_failure() function that we developed earlier for this:

```
    /*
     * Repeatedly call SSL_shutdown() until the connection is fully
     * closed.
     */
    while ((ret = SSL_shutdown(ssl)) != 1) {
        if (ret < 0 && handle_io_failure(ssl, ret) == 1)
            continue; /* Retry */
    }
```

### Final clean up
As with the blocking QUIC client example, once our connection is finished with we must free it. The steps to do this for this example are the same as for the blocking example, so we won't repeat it here.

## FURTHER READING
See ossl-guide-quic-client-block(7) to read a tutorial on how to write a blocking QUIC client. See ossl-guide-quic-multi-stream(7) to see how to write a multi-stream QUIC client.

## SEE ALSO
ossl-guide-introduction(7), ossl-guide-libraries-introduction(7), ossl-guide-libssl-introduction(7), ossl-guide-quic-introduction(7), ossl-guide-quic-client-block(7), ossl-guide-quic-multi-stream(7)

## COPYRIGHT
Copyright 2023 The OpenSSL Project Authors. All Rights Reserved.

Licensed under the Apache License 2.0 (the "License"). You may not use this file except in compliance with the License. You can obtain a copy in the file LICENSE in the source distribution or at <https://www.openssl.org/source/license.html>.


ossl-guide-quic-introduction

package: openssl

title: ossl-guide-quic-introduction(7ossl)

summary: guide\-quic\-introduction

text:
# ossl-guide-quic-introduction(7ossl)

## NAME
ossl-guide-quic-introduction - OpenSSL Guide: An introduction to QUIC in OpenSSL

## INTRODUCTION
This page will provide an introduction to some basic QUIC concepts and background and how it is used within OpenSSL. It assumes that you have a basic understanding of UDP/IP and sockets. It also assumes that you are familiar with some OpenSSL and TLS fundamentals (see ossl-guide-libraries-introduction(7) and ossl-guide-tls-introduction(7)).

## WHAT IS QUIC?
QUIC is a general purpose protocol for enabling applications to securely communicate over a network. It is defined in RFC9000 (see <https://datatracker.ietf.org/doc/rfc9000/>). QUIC integrates parts of the TLS protocol for connection establishment but independently protects packets. It provides similar security guarantees to TLS such as confidentiality, integrity and authentication (see ossl-guide-tls-introduction(7)).

QUIC delivers a number of advantages:

Multiple streams

It supports multiple streams of communication (see "QUIC STREAMS" below), allowing application protocols built on QUIC to create arbitrarily many bytestreams for communication between a client and server. This allows an application protocol to avoid problems where one packet of data is held up waiting on another packet being delivered (commonly referred to as "head-of-line blocking"). It also enables an application to open additional logical streams without requiring a round-trip exchange of packets between the client and server as is required when opening an additional TLS/TCP connection.

HTTP/3

Since QUIC is the basis of HTTP/3, support for QUIC also enables applications to use HTTP/3 using a suitable third-party library.

Fast connection initiation

Future versions of OpenSSL will offer support for 0-RTT connection initiation, allowing a connection to be initiated to a server and application data to be transmitted without any waiting time. This is similar to TLS 1.3's 0-RTT functionality but also avoids the round trip needed to open a TCP socket; thus, it is similar to a combination of TLS 1.3 0-RTT and TCP Fast Open.

Connection migration

Future versions of OpenSSL will offer support for connection migration, allowing connections to seamlessly survive IP address changes.

Datagram based use cases

Future versions of OpenSSL will offer support for the QUIC datagram extension, allowing support for both TLS and DTLS-style use cases on a single connection.

Implemented as application library

Because most QUIC implementations, including OpenSSL's implementation, are implemented as an application library rather than by an operating system, an application can gain the benefit of QUIC without needing to wait for an OS update to be deployed. Future evolutions and enhancements to the QUIC protocol can be delivered as quickly as an application can be updated without dependency on an OS update cadence.

Multiplexing over a single UDP socket

Because QUIC is UDP-based, it is possible to multiplex a QUIC connection on the same UDP socket as some other UDP-based protocols, such as RTP.

## QUIC TIME BASED EVENTS
A key difference between the TLS implementation and the QUIC implementation in OpenSSL is how time is handled. The QUIC protocol requires various actions to be performed on a regular basis regardless of whether application data is being transmitted or received.

OpenSSL introduces a new function SSL_handle_events(3) that will automatically process any outstanding time based events that must be handled. Alternatively calling any I/O function such as SSL_read_ex(3) or SSL_write_ex(3) will also process these events. There is also SSL_get_event_timeout(3) which tells an application the amount of time that remains until SSL_handle_events(3) (or any I/O function) must be called.

Fortunately a blocking application that does not leave the QUIC connection idle, and is regularly calling I/O functions does not typically need to worry about this. However if you are developing a nonblocking application or one that may leave the QUIC connection idle for a period of time then you will need to arrange to call these functions.

OpenSSL provides an optional "thread assisted mode" that will automatically create a background thread and will regularly call SSL_handle_events(3) in a thread safe manner. This provides a simple way for an application to satisfy the QUIC requirements for time based events without having to implement special logic to accomplish it.

## QUIC AND TLS
QUIC reuses parts of the TLS protocol in its implementation. Specifically the TLS handshake also exists in QUIC. The TLS handshake messages are wrapped up in QUIC protocol messages in order to send them to the peer. Once the TLS handshake is complete all application data is sent entirely using QUIC protocol messages without using TLS - although some TLS handshake messages may still be sent in some circumstances.

This relationship between QUIC and TLS means that many of the API functions in OpenSSL that apply to TLS connections also apply to QUIC connections and applications can use them in exactly the same way. Some functions do not apply to QUIC at all, and others have altered semantics. You should refer to the documentation pages for each function for information on how it applies to QUIC. Typically if QUIC is not mentioned in the manual pages then the functions apply to both TLS and QUIC.

## QUIC STREAMS
QUIC introduces the concept of "streams". A stream provides a reliable mechanism for sending and receiving application data between the endpoints. The bytes transmitted are guaranteed to be received in the same order they were sent without any loss of data or reordering of the bytes. A TLS application effectively has one bi-directional stream available to it per TLS connection. A QUIC application can have multiple uni-directional or bi-directional streams available to it for each connection.

In OpenSSL an SSL object is used to represent both connections and streams. A QUIC application creates an initial SSL object to represent the connection (known as the connection SSL object). Once the connection is complete additional SSL objects can be created to represent streams (known as stream SSL objects). Unless configured otherwise, a "default" stream is also associated with the connection SSL object so you can still write data and read data to/from it. Some OpenSSL API functions can only be used with connection SSL objects, and some can only be used with stream SSL objects. Check the documentation for each function to confirm what type of SSL object can be used in any particular context. A connection SSL object that has a default stream attached to it can be used in contexts that require a connection SSL object or in contexts that require a stream SSL object.

## SOCKETS AND BLOCKING
TLS assumes "stream" type semantics for its underlying transport layer protocol (usually achieved by using TCP). However QUIC assumes "datagram" type semantics by using UDP. An OpenSSL application using QUIC is responsible for creating a BIO to represent the underlying transport layer. This BIO must support datagrams and is typically BIO_s_datagram(3), but other BIO choices are available. See bio(7) for an introduction to OpenSSL's BIO concept.

A significant difference between OpenSSL TLS applications and OpenSSL QUIC applications is the way that blocking is implemented. In TLS if your application expects blocking behaviour then you configure the underlying socket for blocking. Conversely if your application wants nonblocking behaviour then the underlying socket is configured to be nonblocking.

With an OpenSSL QUIC application the underlying socket must always be configured to be nonblocking. Howevever the SSL object will, by default, still operate in blocking mode. So, from an application's perspective, calls to functions such as SSL_read_ex(3), SSL_write_ex(3) and other I/O functions will still block. OpenSSL itself provides that blocking capability for QUIC instead of the socket. If nonblocking behaviour is desired then the application must call SSL_set_blocking_mode(3).

## FURTHER READING
See ossl-guide-quic-client-block(7) to see an example of applying these concepts in order to write a simple blocking QUIC client.

## SEE ALSO
ossl-guide-introduction(7), ossl-guide-libraries-introduction(7), ossl-guide-libssl-introduction(7), ossl-guide-tls-introduction(7), ossl-guide-tls-client-block(7), ossl-guide-quic-client-block(7), bio(7)

## COPYRIGHT
Copyright 2023 The OpenSSL Project Authors. All Rights Reserved.

Licensed under the Apache License 2.0 (the "License"). You may not use this file except in compliance with the License. You can obtain a copy in the file LICENSE in the source distribution or at <https://www.openssl.org/source/license.html>.


ossl-guide-quic-multi-stream

title: ossl-guide-quic-multi-stream(7ossl)

summary: guide\-quic\-multi\-stream

text:
# ossl-guide-quic-multi-stream(7ossl)

## NAME
ossl-guide-quic-multi-stream - OpenSSL Guide: Writing a simple multi-stream QUIC client

## INTRODUCTION
This page will introduce some important concepts required to write a simple QUIC multi-stream application. It assumes a basic understanding of QUIC and how it is used in OpenSSL. See ossl-guide-quic-introduction(7) and ossl-guide-quic-client-block(7).

## QUIC STREAMS
In a QUIC multi-stream application we separate out the concepts of a QUIC "connection" and a QUIC "stream". A connection object represents the overarching details of the connection between a client and a server including all its negotiated and configured parameters. We use the SSL object for that in an OpenSSL application (known as the connection SSL object). It is created by an application calling SSL_new(3).

Separately a connection can have zero or more streams associated with it (although a connection with zero streams is probably not very useful, so normally you would have at least one). A stream is used to send and receive data between the two peers. Each stream is also represented by an SSL object. A stream is logically independent of all the other streams associated with the same connection. Data sent on a stream is guaranteed to be delivered in the order that it was sent within that stream. The same is not true across streams, e.g. if an application sends data on stream 1 first and then sends some more data on stream 2 second, then the remote peer may receive the data sent on stream 2 before it receives the data sent on stream 1.

Once the connection SSL object has completed its handshake (i.e. SSL_connect(3) has returned 1), stream SSL objects are created by the application calling SSL_new_stream(3) or SSL_accept_stream(3) (see "CREATING NEW STREAMS" below).

The same threading rules apply to SSL objects as for most OpenSSL objects (see ossl-guide-libraries-introduction(7)). In particular most OpenSSL functions are thread safe, but the SSL object is not. This means that you can use an SSL object representing one stream at the same time as another thread is using a different SSL object for a different stream on the same connection. But you cannot use the same SSL object on two different threads at the same time (without additional application level locking).

## THE DEFAULT STREAM
A connection SSL object may also (optionally) be associated with a stream. This stream is known as the default stream. The default stream is automatically created and associated with the SSL object when the application calls SSL_read_ex(3), SSL_read(3), SSL_write_ex(3) or SSL_write(3) and passes the connection SSL object as a parameter.

If a client application calls SSL_write_ex(3) or SSL_write(3) first then (by default) the default stream will be a client-initiated bi-directional stream. If a client application calls SSL_read_ex(3) or SSL_read(3) first then the first stream initiated by the server will be used as the default stream (whether it is bi-directional or uni-directional).

This behaviour can be controlled via the default stream mode. See SSL_set_default_stream_mode(3) for further details.

It is recommended that new multi-stream applications should not use a default stream at all and instead should use a separate stream SSL object for each stream that is used. This requires calling SSL_set_default_stream_mode(3) and setting the mode to SSL_DEFAULT_STREAM_MODE_NONE.

## CREATING NEW STREAMS
An endpoint can create a new stream by calling SSL_new_stream(3). This creates a locally initiated stream. In order to do so you must pass the QUIC connection SSL object as a parameter. You can also specify whether you want a bi-directional or a uni-directional stream.

The function returns a new QUIC stream SSL object for sending and receiving data on that stream.

The peer may also initiate streams. An application can use the function SSL_get_accept_stream_queue_len(3) to determine the number of streams that the peer has initiated that are waiting for the application to handle. An application can call SSL_accept_stream(3) to create a new SSL object for a remotely initiated stream. If the peer has not initiated any then this call will block until one is available if the connection object is in blocking mode (see SSL_set_blocking_mode(3)).

When using a default stream OpenSSL will prevent new streams from being accepted. To override this behaviour you must call SSL_set_incoming_stream_policy(3) to set the policy to SSL_INCOMING_STREAM_POLICY_ACCEPT. See the man page for further details. This is not relevant if the default stream has been disabled as described in "THE DEFAULT STREAM" above.

Any stream may be bi-directional or uni-directional. If it is uni-directional then the initiator can write to it but not read from it, and vice-versa for the peer. You can determine what type of stream an SSL object represents by calling SSL_get_stream_type(3). See the man page for further details.

## USING A STREAM TO SEND AND RECEIVE DATA
Once you have a stream SSL object (which includes the connection SSL object if a default stream is in use) then you can send and receive data over it using the SSL_write_ex(3), SSL_write(3), SSL_read_ex(3) or SSL_read(3) functions. See the man pages for further details.

In the event of one of these functions not returning a success code then you should call SSL_get_error(3) to find out further details about the error. In blocking mode this will either be a fatal error (e.g. SSL_ERROR_SYSCALL or SSL_ERROR_SSL), or it will be SSL_ERROR_ZERO_RETURN which can occur when attempting to read data from a stream and the peer has indicated that the stream is concluded (i.e. "FIN" has been signalled on the stream). This means that the peer will send no more data on that stream. Note that the interpretation of SSL_ERROR_ZERO_RETURN is slightly different for a QUIC application compared to a TLS application. In TLS it occurs when the connection has been shutdown by the peer. In QUIC this only tells you that the current stream has been concluded by the peer. It tells you nothing about the underlying connection. If the peer has concluded the stream then no more data will be received on it, however an application can still send data to the peer until the send side of the stream has also been concluded. This can happen by the application calling SSL_stream_conclude(3). It is an error to attempt to send more data on a stream after SSL_stream_conclude(3) has been called.

It is also possible to abandon a stream abnormally by calling SSL_stream_reset(3).

Once a stream object is no longer needed it should be freed via a call to SSL_free(3). An application should not call SSL_shutdown(3) on it since this is only meaningful for connection level SSL objects. Freeing the stream will automatically signal STOP_SENDING to the peer.

## STREAMS AND CONNECTIONS
Given a stream object it is possible to get the SSL object corresponding to the connection via a call to SSL_get0_connection(3). Multi-threaded restrictions apply so care should be taken when using the returned connection object. Specifically, if you are handling each of your stream objects in a different thread and call SSL_get0_connection(3) from within that thread then you must be careful to not to call any function that uses the connection object at the same time as one of the other threads is also using that connection object (with the exception of SSL_accept_stream(3) and SSL_get_accept_stream_queue_len(3) which are thread-safe).

A stream object does not inherit all its settings and values from its parent SSL connection object. Therefore certain function calls that are relevant to the connection as a whole will not work on a stream. For example the function SSL_get_certificate(3) can be used to obtain a handle on the peer certificate when called with a connection SSL object. When called with a stream SSL object it will return NULL.

## SIMPLE MULTI-STREAM QUIC CLIENT EXAMPLE
This section will present various source code samples demonstrating how to write a simple multi-stream QUIC client application which connects to a server, send some HTTP/1.0 requests to it, and read back the responses. Note that HTTP/1.0 over QUIC is non-standard and will not be supported by real world servers. This is for demonstration purposes only.

We will build on the example code for the simple blocking QUIC client that is covered on the ossl-guide-quic-client-block(7) page and we assume that you are familiar with it. We will only describe the differences between the simple blocking QUIC client and the multi-stream QUIC client. Although the example code uses blocking SSL objects, you can equally use nonblocking SSL objects. See ossl-guide-quic-client-non-block(7) for more information about writing a nonblocking QUIC client.

The complete source code for this example multi-stream QUIC client is available in the "demos/guide" directory of the OpenSSL source distribution in the file "quic-multi-stream.c". It is also available online at <https://github.com/openssl/openssl/blob/master/demos/guide/quic-multi-stream.c>.

### Disabling the default stream
As discussed above in "THE DEFAULT STREAM" we will follow the recommendation to disable the default stream for our multi-stream client. To do this we call the SSL_set_default_stream_mode(3) function and pass in our connection SSL object and the value SSL_DEFAULT_STREAM_MODE_NONE.

```
    /*
     * We will use multiple streams so we will disable the default stream mode.
     * This is not a requirement for using multiple streams but is recommended.
     */
    if (!SSL_set_default_stream_mode(ssl, SSL_DEFAULT_STREAM_MODE_NONE)) {
        printf("Failed to disable the default stream mode\n");
        goto end;
    }
```

### Creating the request streams
For the purposes of this example we will create two different streams to send two different HTTP requests to the server. For the purposes of demonstration the first of these will be a bi-directional stream and the second one will be a uni-directional one:

```
    /*
     * We create two new client initiated streams. The first will be
     * bi-directional, and the second will be uni-directional.
     */
    stream1 = SSL_new_stream(ssl, 0);
    stream2 = SSL_new_stream(ssl, SSL_STREAM_FLAG_UNI);
    if (stream1 == NULL || stream2 == NULL) {
        printf("Failed to create streams\n");
        goto end;
    }
```

### Writing data to the streams
Once the streams are successfully created we can start writing data to them. In this example we will be sending a different HTTP request on each stream. To avoid repeating too much code we write a simple helper function to send an HTTP request to a stream:

```
    int write_a_request(SSL *stream, const char *request_start,
                        const char *hostname)
    {
        const char *request_end = "\r\n\r\n";
        size_t written;
        if (!SSL_write_ex(stream, request_start, strlen(request_start), &written))
            return 0;
        if (!SSL_write_ex(stream, hostname, strlen(hostname), &written))
            return 0;
        if (!SSL_write_ex(stream, request_end, strlen(request_end), &written))
            return 0;
        return 1;
    }
```

We assume the strings request1_start and request2_start hold the appropriate HTTP requests. We can then call our helper function above to send the requests on the two streams. For the sake of simplicity this example does this sequentially, writing to stream1 first and, when this is successful, writing to stream2 second. Remember that our client is blocking so these calls will only return once they have been successfully completed. A real application would not need to do these writes sequentially or in any particular order. For example we could start two threads (one for each stream) and write the requests to each stream simultaneously.

```
    /* Write an HTTP GET request on each of our streams to the peer */
    if (!write_a_request(stream1, request1_start, hostname)) {
        printf("Failed to write HTTP request on stream 1\n");
        goto end;
    }
    if (!write_a_request(stream2, request2_start, hostname)) {
        printf("Failed to write HTTP request on stream 2\n");
        goto end;
    }
```

### Reading data from a stream
In this example stream1 is a bi-directional stream so, once we have sent the request on it, we can attempt to read the response from the server back. Here we just repeatedly call SSL_read_ex(3) until that function fails (indicating either that there has been a problem, or that the peer has signalled the stream as concluded).

```
    printf("Stream 1 data:\n");
    /*
     * Get up to sizeof(buf) bytes of the response from stream 1 (which is a
     * bidirectional stream). We keep reading until the server closes the
     * connection.
     */
    while (SSL_read_ex(stream1, buf, sizeof(buf), &readbytes)) {
        /*
        * OpenSSL does not guarantee that the returned data is a string or
        * that it is NUL terminated so we use fwrite() to write the exact
        * number of bytes that we read. The data could be non-printable or
        * have NUL characters in the middle of it. For this simple example
        * we're going to print it to stdout anyway.
        */
        fwrite(buf, 1, readbytes, stdout);
    }
    /* In case the response didn't finish with a newline we add one now */
    printf("\n");
```

In a blocking application like this one calls to SSL_read_ex(3) will either succeed immediately returning data that is already available, or they will block waiting for more data to become available and return it when it is, or they will fail with a 0 response code.

Once we exit the while loop above we know that the last call to SSL_read_ex(3) gave a 0 response code so we call the SSL_get_error(3) function to find out more details. Since this is a blocking application this will either return SSL_ERROR_SYSCALL or SSL_ERROR_SSL indicating a fundamental problem, or it will return SSL_ERROR_ZERO_RETURN indicating that the stream is concluded and there will be no more data available to read from it. Care must be taken to distinguish between an error at the stream level (i.e. a stream reset) and an error at the connection level (i.e. a connection closed). The SSL_get_stream_read_state(3) function can be used to distinguish between these different cases.

```
    /*
     * Check whether we finished the while loop above normally or as the
     * result of an error. The 0 argument to SSL_get_error() is the return
     * code we received from the SSL_read_ex() call. It must be 0 in order
     * to get here. Normal completion is indicated by SSL_ERROR_ZERO_RETURN. In
     * QUIC terms this means that the peer has sent FIN on the stream to
     * indicate that no further data will be sent.
     */
    switch (SSL_get_error(stream1, 0)) {
    case SSL_ERROR_ZERO_RETURN:
        /* Normal completion of the stream */
        break;
    case SSL_ERROR_SSL:
        /*
         * Some stream fatal error occurred. This could be because of a stream
         * reset - or some failure occurred on the underlying connection.
         */
        switch (SSL_get_stream_read_state(stream1)) {
        case SSL_STREAM_STATE_RESET_REMOTE:
            printf("Stream reset occurred\n");
            /* The stream has been reset but the connection is still healthy. */
            break;
        case SSL_STREAM_STATE_CONN_CLOSED:
            printf("Connection closed\n");
            /* Connection is already closed. Skip SSL_shutdown() */
            goto end;
        default:
            printf("Unknown stream failure\n");
            break;
        }
        break;
    default:
        /* Some other unexpected error occurred */
        printf ("Failed reading remaining data\n");
        break;
    }
```

### Accepting an incoming stream
Our stream2 object that we created above was a uni-directional stream so it cannot be used to receive data from the server. In this hypothetical example we assume that the server initiates a new stream to send us back the data that we requested. To do that we call SSL_accept_stream(3). Since this is a blocking application this will wait indefinitely until the new stream has arrived and is available for us to accept. In the event of an error it will return NULL.

```
    /*
     * In our hypothetical HTTP/1.0 over QUIC protocol that we are using we
     * assume that the server will respond with a server initiated stream
     * containing the data requested in our uni-directional stream. This doesn't
     * really make sense to do in a real protocol, but its just for
     * demonstration purposes.
     *
     * We're using blocking mode so this will block until a stream becomes
     * available. We could override this behaviour if we wanted to by setting
     * the SSL_ACCEPT_STREAM_NO_BLOCK flag in the second argument below.
     */
    stream3 = SSL_accept_stream(ssl, 0);
    if (stream3 == NULL) {
        printf("Failed to accept a new stream\n");
        goto end;
    }
```

We can now read data from the stream in the same way that we did for stream1 above. We won't repeat that here.

### Cleaning up the streams
Once we have finished using our streams we can simply free them by calling SSL_free(3). Optionally we could call SSL_stream_conclude(3) on them if we want to indicate to the peer that we won't be sending them any more data, but we don't do that in this example because we assume that the HTTP application protocol supplies sufficient information for the peer to know when we have finished sending request data.

We should not call SSL_shutdown(3) or SSL_shutdown_ex(3) on the stream objects since those calls should not be used for streams.

```
    SSL_free(stream1);
    SSL_free(stream2);
    SSL_free(stream3);
```

## SEE ALSO
ossl-guide-introduction(7), ossl-guide-libraries-introduction(7), ossl-guide-libssl-introduction(7) ossl-guide-quic-introduction(7), ossl-guide-quic-client-block(7)

## COPYRIGHT
Copyright 2023 The OpenSSL Project Authors. All Rights Reserved.

Licensed under the Apache License 2.0 (the "License"). You may not use this file except in compliance with the License. You can obtain a copy in the file LICENSE in the source distribution or at <https://www.openssl.org/source/license.html>.


ossl-guide-tls-client-block

title: ossl-guide-tls-client-block(7ossl)

summary: guide\-tls\-client\-block

text:
# ossl-guide-tls-client-block(7ossl)

## NAME
ossl-guide-tls-client-block - OpenSSL Guide: Writing a simple blocking TLS client

## SIMPLE BLOCKING TLS CLIENT EXAMPLE
This page will present various source code samples demonstrating how to write a simple TLS client application which connects to a server, sends an HTTP/1.0 request to it, and reads back the response.

We use a blocking socket for the purposes of this example. This means that attempting to read data from a socket that has no data available on it to read will block (and the function will not return), until data becomes available. For example, this can happen if we have sent our request, but we are still waiting for the server's response. Similarly any attempts to write to a socket that is not able to write at the moment will block until writing is possible.

This blocking behaviour simplifies the implementation of a client because you do not have to worry about what happens if data is not yet available. The application will simply wait until it is available.

The complete source code for this example blocking TLS client is available in the demos/guide directory of the OpenSSL source distribution in the file tls-client-block.c. It is also available online at <https://github.com/openssl/openssl/blob/master/demos/guide/tls-client-block.c>.

We assume that you already have OpenSSL installed on your system; that you already have some fundamental understanding of OpenSSL concepts and TLS (see ossl-guide-libraries-introduction(7) and ossl-guide-tls-introduction(7)); and that you know how to write and build C code and link it against the libcrypto and libssl libraries that are provided by OpenSSL. It also assumes that you have a basic understanding of TCP/IP and sockets.

### Creating the SSL_CTX and SSL objects
The first step is to create an SSL_CTX object for our client. We use the SSL_CTX_new(3) function for this purpose. We could alternatively use SSL_CTX_new_ex(3) if we want to associate the SSL_CTX with a particular OSSL_LIB_CTX (see ossl-guide-libraries-introduction(7) to learn about OSSL_LIB_CTX). We pass as an argument the return value of the function TLS_client_method(3). You should use this method whenever you are writing a TLS client. This method will automatically use TLS version negotiation to select the highest version of the protocol that is mutually supported by both the client and the server.

```
    /*
     * Create an SSL_CTX which we can use to create SSL objects from. We
     * want an SSL_CTX for creating clients so we use TLS_client_method()
     * here.
     */
    ctx = SSL_CTX_new(TLS_client_method());
    if (ctx == NULL) {
        printf("Failed to create the SSL_CTX\n");
        goto end;
    }
```

Since we are writing a client we must ensure that we verify the server's certificate. We do this by calling the SSL_CTX_set_verify(3) function and pass the SSL_VERIFY_PEER value to it. The final argument to this function is a callback that you can optionally supply to override the default handling for certificate verification. Most applications do not need to do this so this can safely be set to NULL to get the default handling.

```
    /*
     * Configure the client to abort the handshake if certificate
     * verification fails. Virtually all clients should do this unless you
     * really know what you are doing.
     */
    SSL_CTX_set_verify(ctx, SSL_VERIFY_PEER, NULL);
```

In order for certificate verification to be successful you must have configured where the trusted certificate store to be used is located (see ossl-guide-tls-introduction(7)). In most cases you just want to use the default store so we call SSL_CTX_set_default_verify_paths(3).

```
    /* Use the default trusted certificate store */
    if (!SSL_CTX_set_default_verify_paths(ctx)) {
        printf("Failed to set the default trusted certificate store\n");
        goto end;
    }
```

We would also like to restrict the TLS versions that we are willing to accept to TLSv1.2 or above. TLS protocol versions earlier than that are generally to be avoided where possible. We can do that using SSL_CTX_set_min_proto_version(3):

```
    /*
     * TLSv1.1 or earlier are deprecated by IETF and are generally to be
     * avoided if possible. We require a minimum TLS version of TLSv1.2.
     */
    if (!SSL_CTX_set_min_proto_version(ctx, TLS1_2_VERSION)) {
        printf("Failed to set the minimum TLS protocol version\n");
        goto end;
    }
```

That is all the setup that we need to do for the SSL_CTX, so next we need to create an SSL object to represent the TLS connection. In a real application we might expect to be creating more than one TLS connection over time. In that case we would expect to reuse the SSL_CTX that we already created each time. There is no need to repeat those steps. In fact it is best not to since certain internal resources are cached in the SSL_CTX. You will get better performance by reusing an existing SSL_CTX instead of creating a new one each time.

Creating the SSL object is a simple matter of calling the SSL_new(3) function and passing the SSL_CTX we created as an argument.

```
    /* Create an SSL object to represent the TLS connection */
    ssl = SSL_new(ctx);
    if (ssl == NULL) {
        printf("Failed to create the SSL object\n");
        goto end;
    }
```

### Creating the socket and BIO
TLS data is transmitted over an underlying transport layer. Normally a TCP socket. It is the application's responsibility for ensuring that the socket is created and associated with an SSL object (via a BIO).

Socket creation for use by a client is typically a 2 step process, i.e. constructing the socket; and connecting the socket.

How to construct a socket is platform specific - but most platforms (including Windows) provide a POSIX compatible interface via the socket function, e.g. to create an IPv4 TCP socket:

```
    int sock;
    sock = socket(AF_INET, SOCK_STREAM, 0);
    if (sock == -1)
        return NULL;
```

Once the socket is constructed it must be connected to the remote server. Again the details are platform specific but most platforms (including Windows) provide the POSIX compatible connect function. For example:

```
    struct sockaddr_in serveraddr;
    struct hostent *server;
    server = gethostbyname("www.openssl.org");
    if (server == NULL) {
        close(sock);
        return NULL;
    }
    memset(&serveraddr, 0, sizeof(serveraddr));
    serveraddr.sin_family = server->h_addrtype;
    serveraddr.sin_port = htons(443);
    memcpy(&serveraddr.sin_addr.s_addr, server->h_addr, server->h_length);
    if (connect(sock, (struct sockaddr *)&serveraddr,
                sizeof(serveraddr)) == -1) {
        close(sock);
        return NULL;
    }
```

OpenSSL provides portable helper functions to do these tasks which also integrate into the OpenSSL error system to log error data, e.g.

```
    int sock = -1;
    BIO_ADDRINFO *res;
    const BIO_ADDRINFO *ai = NULL;
    /*
     * Lookup IP address info for the server.
     */
    if (!BIO_lookup_ex(hostname, port, BIO_LOOKUP_CLIENT, family, SOCK_STREAM, 0,
                       &res))
        return NULL;
    /*
     * Loop through all the possible addresses for the server and find one
     * we can connect to.
     */
    for (ai = res; ai != NULL; ai = BIO_ADDRINFO_next(ai)) {
        /*
         * Create a TCP socket. We could equally use non-OpenSSL calls such
         * as "socket" here for this and the subsequent connect and close
         * functions. But for portability reasons and also so that we get
         * errors on the OpenSSL stack in the event of a failure we use
         * OpenSSL's versions of these functions.
         */
        sock = BIO_socket(BIO_ADDRINFO_family(ai), SOCK_STREAM, 0, 0);
        if (sock == -1)
            continue;
        /* Connect the socket to the server's address */
        if (!BIO_connect(sock, BIO_ADDRINFO_address(ai), BIO_SOCK_NODELAY)) {
            BIO_closesocket(sock);
            sock = -1;
            continue;
        }
        /* We have a connected socket so break out of the loop */
        break;
    }
    /* Free the address information resources we allocated earlier */
    BIO_ADDRINFO_free(res);
```

See BIO_lookup_ex(3), BIO_socket(3), BIO_connect(3), BIO_closesocket(3), BIO_ADDRINFO_next(3), BIO_ADDRINFO_address(3) and BIO_ADDRINFO_free(3) for further information on the functions used here. In the above example code the hostname and port variables are strings, e.g. "www.example.com" and "443". Note also the use of the family variable, which can take the values of AF_INET or AF_INET6 based on the command line -6 option, to allow specific connections to an ipv4 or ipv6 enabled host.

Sockets created using the methods described above will automatically be blocking sockets - which is exactly what we want for this example.

Once the socket has been created and connected we need to associate it with a BIO object:

```
    BIO *bio;
    /* Create a BIO to wrap the socket */
    bio = BIO_new(BIO_s_socket());
    if (bio == NULL) {
        BIO_closesocket(sock);
        return NULL;
    }
    /*
     * Associate the newly created BIO with the underlying socket. By
     * passing BIO_CLOSE here the socket will be automatically closed when
     * the BIO is freed. Alternatively you can use BIO_NOCLOSE, in which
     * case you must close the socket explicitly when it is no longer
     * needed.
     */
    BIO_set_fd(bio, sock, BIO_CLOSE);
```

See BIO_new(3), BIO_s_socket(3) and BIO_set_fd(3) for further information on these functions.

Finally we associate the SSL object we created earlier with the BIO using the SSL_set_bio(3) function. Note that this passes ownership of the BIO object to the SSL object. Once ownership is passed the SSL object is responsible for its management and will free it automatically when the SSL is freed. So, once SSL_set_bio(3) has been been called, you should not call BIO_free(3) on the BIO.

```
    SSL_set_bio(ssl, bio, bio);
```

### Setting the server's hostname
We have already connected our underlying socket to the server, but the client still needs to know the server's hostname. It uses this information for 2 key purposes and we need to set the hostname for each one.

Firstly, the server's hostname is included in the initial ClientHello message sent by the client. This is known as the Server Name Indication (SNI). This is important because it is common for multiple hostnames to be fronted by a single server that handles requests for all of them. In other words a single server may have multiple hostnames associated with it and it is important to indicate which one we want to connect to. Without this information we may get a handshake failure, or we may get connected to the "default" server which may not be the one we were expecting.

To set the SNI hostname data we call the SSL_set_tlsext_host_name(3) function like this:

```
    /*
     * Tell the server during the handshake which hostname we are attempting
     * to connect to in case the server supports multiple hosts.
     */
    if (!SSL_set_tlsext_host_name(ssl, hostname)) {
        printf("Failed to set the SNI hostname\n");
        goto end;
    }
```

Here the "hostname" argument is a string representing the hostname of the server, e.g. "www.example.com".

Secondly, we need to tell OpenSSL what hostname we expect to see in the certificate coming back from the server. This is almost always the same one that we asked for in the original request. This is important because, without this, we do not verify that the hostname in the certificate is what we expect it to be and any certificate is acceptable unless your application explicitly checks this itself. We do this via the SSL_set1_host(3) function:

```
    /*
     * Ensure we check during certificate verification that the server has
     * supplied a certificate for the hostname that we were expecting.
     * Virtually all clients should do this unless you really know what you
     * are doing.
     */
    if (!SSL_set1_host(ssl, hostname)) {
        printf("Failed to set the certificate verification hostname");
        goto end;
    }
```

All of the above steps must happen before we attempt to perform the handshake otherwise they will have no effect.

### Performing the handshake
Before we can start sending or receiving application data over a TLS connection the TLS handshake must be performed. We can do this explicitly via the SSL_connect(3) function.

```
    /* Do the handshake with the server */
    if (SSL_connect(ssl) < 1) {
        printf("Failed to connect to the server\n");
        /*
         * If the failure is due to a verification error we can get more
         * information about it from SSL_get_verify_result().
         */
        if (SSL_get_verify_result(ssl) != X509_V_OK)
            printf("Verify error: %s\n",
                X509_verify_cert_error_string(SSL_get_verify_result(ssl)));
        goto end;
    }
```

The SSL_connect(3) function can return 1, 0 or less than 0. Only a return value of 1 is considered a success. For a simple blocking client we only need to concern ourselves with whether the call was successful or not. Anything else indicates that we have failed to connect to the server.

A common cause of failures at this stage is due to a problem verifying the server's certificate. For example if the certificate has expired, or it is not signed by a CA in our trusted certificate store. We can use the SSL_get_verify_result(3) function to find out more information about the verification failure. A return value of X509_V_OK indicates that the verification was successful (so the connection error must be due to some other cause). Otherwise we use the X509_verify_cert_error_string(3) function to get a human readable error message.

### Sending and receiving data
Once the handshake is complete we are able to send and receive application data. Exactly what data is sent and in what order is usually controlled by some application level protocol. In this example we are using HTTP 1.0 which is a very simple request and response protocol. The client sends a request to the server. The server sends the response data and then immediately closes down the connection.

To send data to the server we use the SSL_write_ex(3) function and to receive data from the server we use the SSL_read_ex(3) function. In HTTP 1.0 the client always writes data first. Our HTTP request will include the hostname that we are connecting to. For simplicity, we write the HTTP request in three chunks. First we write the start of the request. Secondly we write the hostname we are sending the request to. Finally we send the end of the request.

```
    size_t written;
    const char *request_start = "GET / HTTP/1.0\r\nConnection: close\r\nHost: ";
    const char *request_end = "\r\n\r\n";
    /* Write an HTTP GET request to the peer */
    if (!SSL_write_ex(ssl, request_start, strlen(request_start), &written)) {
        printf("Failed to write start of HTTP request\n");
        goto end;
    }
    if (!SSL_write_ex(ssl, hostname, strlen(hostname), &written)) {
        printf("Failed to write hostname in HTTP request\n");
        goto end;
    }
    if (!SSL_write_ex(ssl, request_end, strlen(request_end), &written)) {
        printf("Failed to write end of HTTP request\n");
        goto end;
    }
```

The SSL_write_ex(3) function returns 0 if it fails and 1 if it is successful. If it is successful then we can proceed to waiting for a response from the server.

```
    size_t readbytes;
    char buf[160];
    /*
     * Get up to sizeof(buf) bytes of the response. We keep reading until the
     * server closes the connection.
     */
    while (SSL_read_ex(ssl, buf, sizeof(buf), &readbytes)) {
        /*
        * OpenSSL does not guarantee that the returned data is a string or
        * that it is NUL terminated so we use fwrite() to write the exact
        * number of bytes that we read. The data could be non-printable or
        * have NUL characters in the middle of it. For this simple example
        * we're going to print it to stdout anyway.
        */
        fwrite(buf, 1, readbytes, stdout);
    }
    /* In case the response didn't finish with a newline we add one now */
    printf("\n");
```

We use the SSL_read_ex(3) function to read the response. We don't know exactly how much data we are going to receive back so we enter a loop reading blocks of data from the server and printing each block that we receive to the screen. The loop ends as soon as SSL_read_ex(3) returns 0 - meaning that it failed to read any data.

A failure to read data could mean that there has been some error, or it could simply mean that server has sent all the data that it wants to send and has indicated that it has finished by sending a "close_notify" alert. This alert is a TLS protocol level message indicating that the endpoint has finished sending all of its data and it will not send any more. Both of these conditions result in a 0 return value from SSL_read_ex(3) and we need to use the function SSL_get_error(3) to determine the cause of the 0 return value.

```
    /*
     * Check whether we finished the while loop above normally or as the
     * result of an error. The 0 argument to SSL_get_error() is the return
     * code we received from the SSL_read_ex() call. It must be 0 in order
     * to get here. Normal completion is indicated by SSL_ERROR_ZERO_RETURN.
     */
    if (SSL_get_error(ssl, 0) != SSL_ERROR_ZERO_RETURN) {
        /*
         * Some error occurred other than a graceful close down by the
         * peer
         */
        printf ("Failed reading remaining data\n");
        goto end;
    }
```

If SSL_get_error(3) returns SSL_ERROR_ZERO_RETURN then we know that the server has finished sending its data. Otherwise an error has occurred.

### Shutting down the connection
Once we have finished reading data from the server then we are ready to close the connection down. We do this via the SSL_shutdown(3) function which has the effect of sending a TLS protocol level message (a "close_notify" alert) to the server saying that we have finished writing data:

```
    /*
     * The peer already shutdown gracefully (we know this because of the
     * SSL_ERROR_ZERO_RETURN above). We should do the same back.
     */
    ret = SSL_shutdown(ssl);
    if (ret < 1) {
        /*
         * ret < 0 indicates an error. ret == 0 would be unexpected here
         * because that means "we've sent a close_notify and we're waiting
         * for one back". But we already know we got one from the peer
         * because of the SSL_ERROR_ZERO_RETURN above.
         */
        printf("Error shutting down\n");
        goto end;
    }
```

The SSL_shutdown(3) function will either return 1, 0, or less than 0. A return value of 1 is a success, and a return value less than 0 is an error. More precisely a return value of 1 means that we have sent a "close_notify" alert to the server, and that we have also received one back. A return value of 0 means that we have sent a "close_notify" alert to the server, but we have not yet received one back. Usually in this scenario you would call SSL_shutdown(3) again which (with a blocking socket) would block until the "close_notify" is received. However in this case we already know that the server has sent us a "close_notify" because of the SSL_ERROR_ZERO_RETURN that we received from the call to SSL_read_ex(3). So this scenario should never happen in practice. We just treat it as an error in this example.

### Final clean up
Before the application exits we have to clean up some memory that we allocated. If we are exiting due to an error we might also want to display further information about that error if it is available to the user:

```
    /* Success! */
    res = EXIT_SUCCESS;
 end:
    /*
     * If something bad happened then we will dump the contents of the
     * OpenSSL error stack to stderr. There might be some useful diagnostic
     * information there.
     */
    if (res == EXIT_FAILURE)
        ERR_print_errors_fp(stderr);
    /*
     * Free the resources we allocated. We do not free the BIO object here
     * because ownership of it was immediately transferred to the SSL object
     * via SSL_set_bio(). The BIO will be freed when we free the SSL object.
     */
    SSL_free(ssl);
    SSL_CTX_free(ctx);
    return res;
```

To display errors we make use of the ERR_print_errors_fp(3) function which simply dumps out the contents of any errors on the OpenSSL error stack to the specified location (in this case stderr).

We need to free up the SSL object that we created for the connection via the SSL_free(3) function. Also, since we are not going to be creating any more TLS connections we must also free up the SSL_CTX via a call to SSL_CTX_free(3).

## TROUBLESHOOTING
There are a number of things that might go wrong when running the demo application. This section describes some common things you might encounter.

### Failure to connect the underlying socket
This could occur for numerous reasons. For example if there is a problem in the network route between the client and the server; or a firewall is blocking the communication; or the server is not in DNS. Check the network configuration.

### Verification failure of the server certificate
A verification failure of the server certificate would result in a failure when running the SSL_connect(3) function. ERR_print_errors_fp(3) would display an error which would look something like this:

```
 Verify error: unable to get local issuer certificate
 40E74AF1F47F0000:error:0A000086:SSL routines:tls_post_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:2069:
```

A server certificate verification failure could be caused for a number of reasons. For example

Failure to correctly setup the trusted certificate store

See the page ossl-guide-tls-introduction(7) and check that your trusted certificate store is correctly configured

Unrecognised CA

If the CA used by the server's certificate is not in the trusted certificate store for the client then this will cause a verification failure during connection. Often this can occur if the server is using a self-signed certificate (i.e. a test certificate that has not been signed by a CA at all).

Missing intermediate CAs

This is a server misconfiguration where the client has the relevant root CA in its trust store, but the server has not supplied all of the intermediate CA certificates between that root CA and the server's own certificate. Therefore a trust chain cannot be established.

Mismatched hostname

If for some reason the hostname of the server that the client is expecting does not match the hostname in the certificate then this will cause verification to fail.

Expired certificate

The date that the server's certificate is valid to has passed.

The "unable to get local issuer certificate" we saw in the example above means that we have been unable to find the issuer of the server's certificate (or one of its intermediate CA certificates) in our trusted certificate store (e.g. because the trusted certificate store is misconfigured, or there are missing intermediate CAs, or the issuer is simply unrecognised).

## FURTHER READING
See ossl-guide-tls-client-non-block(7) to read a tutorial on how to modify the client developed on this page to support a nonblocking socket.

See ossl-guide-quic-client-block(7) to read a tutorial on how to modify the client developed on this page to support QUIC instead of TLS.

## SEE ALSO
ossl-guide-introduction(7), ossl-guide-libraries-introduction(7), ossl-guide-libssl-introduction(7), ossl-guide-tls-introduction(7), ossl-guide-tls-client-non-block(7), ossl-guide-quic-client-block(7)

## COPYRIGHT
Copyright 2023 The OpenSSL Project Authors. All Rights Reserved.

Licensed under the Apache License 2.0 (the "License"). You may not use this file except in compliance with the License. You can obtain a copy in the file LICENSE in the source distribution or at <https://www.openssl.org/source/license.html>.


ossl-guide-tls-client-non-block

title: ossl-guide-tls-client-non-block(7ossl)

summary: guide\-tls\-client\-non\-block

text:
# ossl-guide-tls-client-non-block(7ossl)

## NAME
ossl-guide-tls-client-non-block - OpenSSL Guide: Writing a simple nonblocking TLS client

## SIMPLE NONBLOCKING TLS CLIENT EXAMPLE
This page will build on the example developed on the ossl-guide-tls-client-block(7) page which demonstrates how to write a simple blocking TLS client. On this page we will amend that demo code so that it supports a nonblocking socket.

The complete source code for this example nonblocking TLS client is available in the demos/guide directory of the OpenSSL source distribution in the file tls-client-non-block.c. It is also available online at <https://github.com/openssl/openssl/blob/master/demos/guide/tls-client-non-block.c>.

As we saw in the previous example a blocking socket is one which waits (blocks) until data is available to read if you attempt to read from it when there is no data yet. Similarly it waits when writing if the socket is currently unable to write at the moment. This can simplify the development of code because you do not have to worry about what to do in these cases. The execution of the code will simply stop until it is able to continue. However in many cases you do not want this behaviour. Rather than stopping and waiting your application may need to go and do other tasks whilst the socket is unable to read/write, for example updating a GUI or performing operations on some other socket.

With a nonblocking socket attempting to read or write to a socket that is currently unable to read or write will return immediately with a non-fatal error. Although OpenSSL does the reading/writing to the socket this nonblocking behaviour is propagated up to the application so that OpenSSL I/O functions such as SSL_read_ex(3) or SSL_write_ex(3) will not block.

Since this page is building on the example developed on the ossl-guide-tls-client-block(7) page we assume that you are familiar with it and we only explain how this example differs.

### Setting the socket to be nonblocking
The first step in writing an application that supports nonblocking is to set the socket into nonblocking mode. A socket will be default be blocking. The exact details on how to do this can differ from one platform to another. Fortunately OpenSSL offers a portable function that will do this for you:

```
    /* Set to nonblocking mode */
    if (!BIO_socket_nbio(sock, 1)) {
        sock = -1;
        continue;
    }
```

You do not have to use OpenSSL's function for this. You can of course directly call whatever functions that your Operating System provides for this purpose on your platform.

### Performing work while waiting for the socket
In a nonblocking application you will need work to perform in the event that we want to read or write to the socket, but we are currently unable to. In fact this is the whole point of using a nonblocking socket, i.e. to give the application the opportunity to do something else. Whatever it is that the application has to do, it must also be prepared to come back and retry the operation that it previously attempted periodically to see if it can now complete. Ideally it would only do this in the event that the state of the underlying socket has actually changed (e.g. become readable where it wasn't before), but this does not have to be the case. It can retry at any time.

Note that it is important that you retry exactly the same operation that you tried last time. You cannot start something new. For example if you were attempting to write the text "Hello World" and the operation failed because the socket is currently unable to write, then you cannot then attempt to write some other text when you retry the operation.

In this demo application we will create a helper function which simulates doing other work. In fact, for the sake of simplicity, it will do nothing except wait for the state of the socket to change.

We call our function wait_for_activity() because all it does is wait until the underlying socket has become readable or writeable when it wasn't before.

```
    static void wait_for_activity(SSL *ssl, int write)
    {
        fd_set fds;
        int width, sock;
        /* Get hold of the underlying file descriptor for the socket */
        sock = SSL_get_fd(ssl);
        FD_ZERO(&fds);
        FD_SET(sock, &fds);
        width = sock + 1;
        /*
         * Wait until the socket is writeable or readable. We use select here
         * for the sake of simplicity and portability, but you could equally use
         * poll/epoll or similar functions
         *
         * NOTE: For the purposes of this demonstration code this effectively
         * makes this demo block until it has something more useful to do. In a
         * real application you probably want to go and do other work here (e.g.
         * update a GUI, or service other connections).
         *
         * Let's say for example that you want to update the progress counter on
         * a GUI every 100ms. One way to do that would be to add a 100ms timeout
         * in the last parameter to "select" below. Then, when select returns,
         * you check if it did so because of activity on the file descriptors or
         * because of the timeout. If it is due to the timeout then update the
         * GUI and then restart the "select".
         */
        if (write)
            select(width, NULL, &fds, NULL, NULL);
        else
            select(width, &fds, NULL, NULL, NULL);
    }
```

In this example we are using the "select" function because it is very simple to use and is available on most Operating Systems. However you could use any other similar function to do the same thing. "select" waits for the state of the underlying socket(s) to become readable/writeable before returning. It also supports a "timeout" (as do most other similar functions) so in your own applications you can make use of this to periodically wake up and perform work while waiting for the socket state to change. But we don't use that timeout capability in this example for the sake of simplicity.

### Handling errors from OpenSSL I/O functions
An application that uses a nonblocking socket will need to be prepared to handle errors returned from OpenSSL I/O functions such as SSL_read_ex(3) or SSL_write_ex(3). Errors may be fatal (for example because the underlying connection has failed), or non-fatal (for example because we are trying to read from the underlying socket but the data has not yet arrived from the peer).

SSL_read_ex(3) and SSL_write_ex(3) will return 0 to indicate an error and SSL_read(3) and SSL_write(3) will return 0 or a negative value to indicate an error. SSL_shutdown(3) will return a negative value to incidate an error.

In the event of an error an application should call SSL_get_error(3) to find out what type of error has occurred. If the error is non-fatal and can be retried then SSL_get_error(3) will return SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE depending on whether OpenSSL wanted to read to or write from the socket but was unable to. Note that a call to SSL_read_ex(3) or SSL_read(3) can still generate SSL_ERROR_WANT_WRITE because OpenSSL may need to write protocol messages (such as to update cryptographic keys) even if the application is only trying to read data. Similarly calls to SSL_write_ex(3) or SSL_write(3) might generate SSL_ERROR_WANT_READ.

Another type of non-fatal error that may occur is SSL_ERROR_ZERO_RETURN. This indicates an EOF (End-Of-File) which can occur if you attempt to read data from an SSL object but the peer has indicated that it will not send any more data on it. In this case you may still want to write data to the connection but you will not receive any more data.

Fatal errors that may occur are SSL_ERROR_SYSCALL and SSL_ERROR_SSL. These indicate that the underlying connection has failed. You should not attempt to shut it down with SSL_shutdown(3). SSL_ERROR_SYSCALL indicates that OpenSSL attempted to make a syscall that failed. You can consult errno for further details. SSL_ERROR_SSL indicates that some OpenSSL error occurred. You can consult the OpenSSL error stack for further details (for example by calling ERR_print_errors(3) to print out details of errors that have occurred).

In our demo application we will write a function to handle these errors from OpenSSL I/O functions:

```
    static int handle_io_failure(SSL *ssl, int res)
    {
        switch (SSL_get_error(ssl, res)) {
        case SSL_ERROR_WANT_READ:
            /* Temporary failure. Wait until we can read and try again */
            wait_for_activity(ssl, 0);
            return 1;
        case SSL_ERROR_WANT_WRITE:
            /* Temporary failure. Wait until we can write and try again */
            wait_for_activity(ssl, 1);
            return 1;
        case SSL_ERROR_ZERO_RETURN:
            /* EOF */
            return 0;
        case SSL_ERROR_SYSCALL:
            return -1;
        case SSL_ERROR_SSL:
            /*
            * If the failure is due to a verification error we can get more
            * information about it from SSL_get_verify_result().
            */
            if (SSL_get_verify_result(ssl) != X509_V_OK)
                printf("Verify error: %s\n",
                    X509_verify_cert_error_string(SSL_get_verify_result(ssl)));
            return -1;
        default:
            return -1;
        }
    }
```

This function takes as arguments the SSL object that represents the connection, as well as the return code from the I/O function that failed. In the event of a non-fatal failure, it waits until a retry of the I/O operation might succeed (by using the wait_for_activity() function that we developed in the previous section). It returns 1 in the event of a non-fatal error (except EOF), 0 in the event of EOF, or -1 if a fatal error occurred.

### Creating the SSL_CTX and SSL objects
In order to connect to a server we must create SSL_CTX and SSL objects for this. The steps do this are the same as for a blocking client and are explained on the ossl-guide-tls-client-block(7) page. We won't repeat that information here.

### Performing the handshake
As in the demo for a blocking TLS client we use the SSL_connect(3) function to perform the TLS handshake with the server. Since we are using a nonblocking socket it is very likely that calls to this function will fail with a non-fatal error while we are waiting for the server to respond to our handshake messages. In such a case we must retry the same SSL_connect(3) call at a later time. In this demo we this in a loop:

```
    /* Do the handshake with the server */
    while ((ret = SSL_connect(ssl)) != 1) {
        if (handle_io_failure(ssl, ret) == 1)
            continue; /* Retry */
        printf("Failed to connect to server\n");
        goto end; /* Cannot retry: error */
    }
```

We continually call SSL_connect(3) until it gives us a success response. Otherwise we use the handle_io_failure() function that we created earlier to work out what we should do next. Note that we do not expect an EOF to occur at this stage, so such a response is treated in the same way as a fatal error.

### Sending and receiving data
As with the blocking TLS client demo we use the SSL_write_ex(3) function to send data to the server. As with SSL_connect(3) above, because we are using a nonblocking socket, this call could fail with a non-fatal error. In that case we should retry exactly the same SSL_write_ex(3) call again. Note that the parameters must be exactly the same, i.e. the same pointer to the buffer to write with the same length. You must not attempt to send different data on a retry. An optional mode does exist (SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER) which will configure OpenSSL to allow the buffer being written to change from one retry to the next. However, in this case, you must still retry exactly the same data - even though the buffer that contains that data may change location. See SSL_CTX_set_mode(3) for further details. As in the TLS client blocking tutorial (ossl-guide-tls-client-block(7)) we write the request in three chunks.

```
    /* Write an HTTP GET request to the peer */
    while (!SSL_write_ex(ssl, request_start, strlen(request_start), &written)) {
        if (handle_io_failure(ssl, 0) == 1)
            continue; /* Retry */
        printf("Failed to write start of HTTP request\n");
        goto end; /* Cannot retry: error */
    }
    while (!SSL_write_ex(ssl, hostname, strlen(hostname), &written)) {
        if (handle_io_failure(ssl, 0) == 1)
            continue; /* Retry */
        printf("Failed to write hostname in HTTP request\n");
        goto end; /* Cannot retry: error */
    }
    while (!SSL_write_ex(ssl, request_end, strlen(request_end), &written)) {
        if (handle_io_failure(ssl, 0) == 1)
            continue; /* Retry */
        printf("Failed to write end of HTTP request\n");
        goto end; /* Cannot retry: error */
    }
```

On a write we do not expect to see an EOF response so we treat that case in the same way as a fatal error.

Reading a response back from the server is similar:

```
    do {
        /*
         * Get up to sizeof(buf) bytes of the response. We keep reading until
         * the server closes the connection.
         */
        while (!eof && !SSL_read_ex(ssl, buf, sizeof(buf), &readbytes)) {
            switch (handle_io_failure(ssl, 0)) {
            case 1:
                continue; /* Retry */
            case 0:
                eof = 1;
                continue;
            case -1:
            default:
                printf("Failed reading remaining data\n");
                goto end; /* Cannot retry: error */
            }
        }
        /*
         * OpenSSL does not guarantee that the returned data is a string or
         * that it is NUL terminated so we use fwrite() to write the exact
         * number of bytes that we read. The data could be non-printable or
         * have NUL characters in the middle of it. For this simple example
         * we're going to print it to stdout anyway.
         */
        if (!eof)
            fwrite(buf, 1, readbytes, stdout);
    } while (!eof);
    /* In case the response didn't finish with a newline we add one now */
    printf("\n");
```

The main difference this time is that it is valid for us to receive an EOF response when trying to read data from the server. This will occur when the server closes down the connection after sending all the data in its response.

In this demo we just print out all the data we've received back in the response from the server. We continue going around the loop until we either encounter a fatal error, or we receive an EOF (indicating a graceful finish).

### Shutting down the connection
As in the TLS blocking example we must shutdown the connection when we are finished with it.

If our application was initiating the shutdown then we would expect to see SSL_shutdown(3) give a return value of 0, and then we would continue to call it until we received a return value of 1 (meaning we have successfully completed the shutdown). In this particular example we don't expect SSL_shutdown() to return 0 because we have already received EOF from the server indicating that it has shutdown already. So we just keep calling it until SSL_shutdown() returns 1. Since we are using a nonblocking socket we might expect to have to retry this operation several times. If SSL_shutdown(3) returns a negative result then we must call SSL_get_error(3) to work out what to do next. We use our handle_io_failure() function that we developed earlier for this:

```
    /*
     * The peer already shutdown gracefully (we know this because of the
     * SSL_ERROR_ZERO_RETURN (i.e. EOF) above). We should do the same back.
     */
    while ((ret = SSL_shutdown(ssl)) != 1) {
        if (ret < 0 && handle_io_failure(ssl, ret) == 1)
            continue; /* Retry */
        /*
         * ret == 0 is unexpected here because that means "we've sent a
         * close_notify and we're waiting for one back". But we already know
         * we got one from the peer because of the SSL_ERROR_ZERO_RETURN
         * (i.e. EOF) above.
         */
        printf("Error shutting down\n");
        goto end; /* Cannot retry: error */
    }
```

### Final clean up
As with the blocking TLS client example, once our connection is finished with we must free it. The steps to do this for this example are the same as for the blocking example, so we won't repeat it here.

## FURTHER READING
See ossl-guide-tls-client-block(7) to read a tutorial on how to write a blocking TLS client. See ossl-guide-quic-client-block(7) to see how to do the same thing for a QUIC client.

## SEE ALSO
ossl-guide-introduction(7), ossl-guide-libraries-introduction(7), ossl-guide-libssl-introduction(7), ossl-guide-tls-introduction(7), ossl-guide-tls-client-block(7), ossl-guide-quic-client-block(7)

## COPYRIGHT
Copyright 2023 The OpenSSL Project Authors. All Rights Reserved.

Licensed under the Apache License 2.0 (the "License"). You may not use this file except in compliance with the License. You can obtain a copy in the file LICENSE in the source distribution or at <https://www.openssl.org/source/license.html>.