OpenSSL will not release file handles

I am writing a service (in C for CentOS) that must make lots of outbound SSL connections to a third party REST API via SSL.

OpenSSL is used to create establishment of the secure connections with the remote server. After initialization of the ssl_connection I register the returned file descriptor with our epoll queue.

I can connect and perform the API transaction OK. After that the remote server terminates the connection and we get EPOLLRDHUP on the fd and so call our cleanup routine.

    signal(SIGPIPE,SIG_IGN);

    if(PS(endpoint)->ssl != NULL)
    {
        if(SSL_shutdown(PS(endpoint)->ssl)==0)
        {
            SSL_shutdown(PS(endpoint)->ssl);
        }
    }

   if(PS(endpoint)->web != NULL)
   {
        BIO_free(PS(endpoint)->web);    // This can cause a SIGPIPE, especialy when debugging!
        PS(endpoint)->web = NULL;
   }

Everything seems OK through this shutdown procedure, no errors are thrown.

HOWEVER – every subsequent connection takes the next system fd and eventualy all file descriptors are exhausted hitting the OS soft limit of 1028.

So the problem is the fd is not being closed and released back to the kernel?

This is confirmed via

ls /proc/$PID/fd/ | wc -l

Can any one please help with the correct OpenSSL session shutdown procedure?

Answer

SSL_shutdown documentation mentions, it just sends close notifications for proper shutdown of TLS connection at both ends

It does not actually close the socket, your application should call close() on fd explicitly so that the socket is closed properly. As your current sockets are not closed properly, FDs are not getting reused and each new socket getting new FD, eventually running out of limit as you mentioned