Is there any command line API change in Debian 9 curl?
Recently I started to use Debian 9 (9.4, from Debian 8.x) and a script involving curl stopped working. I connect to internet through a squid proxy on localhost connected to a parent proxy.
My environment variables are configured like this
root@server:~# printenv | grep -i proxy HTTP_PROXY=http://127.0.0.1:3128 FTP_PROXY=http://127.0.0.1:3128 https_proxy=https://127.0.0.1:3128 http_proxy=http://127.0.0.1:3128 HTTPS_PROXY=https://127.0.0.1:3128 ftp_proxy=http://127.0.0.1:3128
When I use wget, it works:
root@server:~# wget https://www.google.com.cu --2018-03-14 09:08:53-- https://www.google.com.cu/ Connecting to 127.0.0.1:3128... connected. Proxy request sent, awaiting response... 200 OK Length: unspecified [text/html] Saving to: ‘index.html’ index.html [ <=> ] 11.12K --.-KB/s in 0.001s 2018-03-14 09:08:54 (14.9 MB/s) - ‘index.html’ saved [11389]
when I use curl, this is what I get
root@server:~# curl -v https://www.google.com.cu * Rebuilt URL to: https://www.google.com.cu/ * Trying 127.0.0.1... * TCP_NODELAY set * Connected to (nil) (127.0.0.1) port 3128 (#0) * ALPN, offering h2 * ALPN, offering http/1.1 * Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH * successfully set certificate verify locations: * CAfile: none CApath: /etc/ssl/certs * TLSv1.2 (OUT), TLS header, Certificate Status (22): * TLSv1.2 (OUT), TLS handshake, Client hello (1): * error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol * Curl_http_done: called premature == 0 * Closing connection 0 curl: (35) error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol
I know these two commands are not equivalent, this is just to illustrate the HTTPS transfer problem.
I need to use curl because the script uses a web API, so it needs to use POST instead of GET request, and to set some headers and data to the POST request. (api.dropboxapi.com is the target site)
This all used to work on Debian 8 without a hitch, and besides wget WORKS, only curl is failing with the debian version change. All the other HTTPS clients seem unaffected (FF, Chrome, Edge, wget all seems to work as always)
Is there any workaround, fix, command line option change or whatever for making debian 9’s version of curl work?
There must be a way, I can’t conceive curl can’t make a HTTPS connection to google. There must be a command line or something that allows the connection.
Output of “curl -V”
root@server:~# curl -V curl 7.52.1 (x86_64-pc-linux-gnu) libcurl/7.52.1 OpenSSL/1.0.2l zlib/1.2.8 libidn2/0.16 libpsl/0.17.0 (+libidn2/0.16) libssh2/1.7.0 nghttp2/1.18.1 librtmp/2.3 Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP HTTP2 UnixSockets HTTPS-proxy PSL
Output of “curl –insecure” as suggested
root@server:~# curl --insecure -v https://www.google.com.cu * Rebuilt URL to: https://www.google.com.cu/ * Trying 127.0.0.1... * TCP_NODELAY set * Connected to (nil) (127.0.0.1) port 3128 (#0) * ALPN, offering h2 * ALPN, offering http/1.1 * Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH * successfully set certificate verify locations: * CAfile: none CApath: /etc/ssl/certs * TLSv1.2 (OUT), TLS header, Certificate Status (22): * TLSv1.2 (OUT), TLS handshake, Client hello (1): * error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol * Curl_http_done: called premature == 0 * Closing connection 0 curl: (35) error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol
“curl -v https://www.google.com.cu –sslv2″ output
root@server:/etc/squid# curl -v https://www.google.com.cu --sslv2 * Rebuilt URL to: https://www.google.com.cu/ * Trying 192.168.4.65... * TCP_NODELAY set * Connected to (nil) (192.168.4.65) port 81 (#0) * ALPN, offering h2 * ALPN, offering http/1.1 * Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH * successfully set certificate verify locations: * CAfile: none CApath: /etc/ssl/certs * TLSv1.2 (OUT), TLS header, Certificate Status (22): * TLSv1.2 (OUT), TLS handshake, Client hello (1): * error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol * Curl_http_done: called premature == 0 * Closing connection 0 curl: (35) error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol
Advertisement
Answer
Many, many thanks to Michael Hampton. It turns out the problem was in the proxy configuration. It should say
https_proxy=http://127.0.0.1:3128 HTTPS_PROXY=http://127.0.0.1:3128
So curl was trying to connect to squid using TLS and failing of course.
Original answer in https://serverfault.com/questions/901626/debian-version-change-affecting-scripts-using-curl-and-https