Skip to content
Advertisement

popen (“tar xvf tarball.tar”) works in debug but not release builds

I’m working on a C++ program for Ubuntu that downloads a tar archive using curl_easy_perform, and after the archive is downloaded into /tmp I use popen to execute the appropriate tar command line.

When I run my program’s debug build then popen("tar xvf /tmp/example.tar -C /tmp/existingdir") works, but when I run this command in release builds the popen call always fails.

Here is my code, with most error checking and unrelated stuff removed:

//tl;dr version:
// first I download a tar archive from url using Curl and save it to filelocation,
// then I untar it using pOpen.  
// pOpen always works in debug, never in release builds
////
Status ExpandTarBall(const MyString& FileName)
{
    //extract the tar ball into a previously created temporary directory, tempDirPath
    MyString args = "tar xvf /tmp/ + FileName + " -C " + tempDirPath;
    cout << "running:" << args << endl;
    // args example:
    // tar xvf /tmp/UserIdXXxCtnAl/examplepackage -C /tmp/UserIdXXxCtnAl
    //
    Status result = ER_OPEN_FAILED;
    FILE* fp = popen(args.c_str(), "re");  //<==========  always works in debug builds, fails with 0 returned in release builds! :(
    if (fp)
    {
        result = pclose(fp) == 0 ? ER_OK : ER_INVALID_DATA;
    }

    return result;
}



    //Note: MyString is an std::string class with some local extensions
Status SslDownloader::DownloadFile(MyString url, MyString fileLocation, bool sslVerify) {

    CURL* curl = NULL;
    CurlInitHelper helper(curl);

    cout << "downloading from " << url.c_str() << " to " << fileLocation.c_str() << endl;

    if (!curl) {
        return ER_SSL_INIT;
    }
    FILE* fp = fopen(fileLocation.c_str(), "wb");
    if(NULL == fp) {
        return ER_OPEN_FAILED;
    }
    curl_easy_setopt(curl, CURLOPT_URL, url.c_str());
    curl_easy_setopt(curl, CURLOPT_USERAGENT, AJPM_USER_AGENT);
    curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, WriteCallback);
    curl_easy_setopt(curl, CURLOPT_FAILONERROR, true);
    curl_easy_setopt(curl, CURLOPT_WRITEDATA, fp);

    if (sslVerify) {
        curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, 2L);
        curl_easy_setopt(curl, CURLOPT_CAINFO, AJPM_CERT_STORE_LOCATION );
        curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 1L);
    } else {
         curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 0L);
         curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, 0L);
    }

    CURLcode res = curl_easy_perform(curl);

    if (0 != fclose(fp)) {
        return ER_WRITE_ERROR;
    }

    if (res != CURLE_OK) {  
            return res == ER_SSL_CONNECT;
        }

    cout << "SSL download of " << fileLocation.c_str() << " succeededn";  // works every time
    return ExpandTarBall(const MyString& FileName);
}

What simple thing am I missing?

Advertisement

Answer

After you use popen to start the child process, you are immediately calling pclose(), without reading from the file returned by popen().

tar’s xvf option will dump a file listing on its standard output, which is a pipe, with popen returning the read side of the pipe to you.

pclose() closes the pipe first, then waits for the child process to terminate.

With both the parent and the child process running concurrently, if the parent process wins the race and closes the pipe before the child process gets in gear, when the child process attempts to write to its standard output, the write side of the pipe, it is going to get a SIGPIPE signal, killing the child process.

It’s very likely that the differences in the runtime profile, between your application’s “debug” and “release” builds, are sufficient to tip the scale towards that outcome with the release builds, while the additional overhead in whatever you actually mean by “debug build” will slow things down enough so that the child process has the time to spill its standard output before the parent process gets around to closing the pipe.

Keep in mind that once your tarball contains a sufficient amount of files, even if tar gets ahead in this race, it’s output is going to fill up the pipe buffer and block, and once the parent process gets around to closing the pipe, it will SIGPIPE the child process, and tar will always fail.

Moral of the story: when using popen to read from the started child process, always read from the pipe and consume the child process’s output, until you get an EOF, before you pclose() it.

Advertisement