How to Use cURL With Proxy [Complete Guide]

When you deal with proxies on a daily basis it is useful to have a range of tools at your disposal that will help you in testing proxies and running simple missions. One of such tools is cURL. It is an easy-to-use and intuitive command-line tool meant to help you in using proxies. With cURL you can send requests, debug and set up proxies – all from the command line.

In this guide (and sometimes tutorial) we will provide you with in-depth information on how to operate cURL and on ways to set it up for scraping using your proxies. In order to benefit from the contents, you need to have at least basic knowledge of proxies, network operating principles, and the basics of web scraping.

What is cURL?

The name cURL stands for client URL. It is an open-source common line tool coupled with a cross-platform library (libcurl). It is used to transfer data between servers operating virtually on any operating system. cURL code can be seen almost ubiquitously, where sending or receiving data is done using the Internet protocols (cURL supports practically all popular Internet protocols including but not limited to FTP, HTTP, DICT, IMAP, etc.).

Historically cURL dates back to the mid-90s when Swedish developer Daniel Stenberg decided to create a script for currency exchange inside a relay chat room. This experimental code led to creating httpget 0.1 tool for data transfer, which later was renamed cURL after FTP came around. Since the 2000s the tool gained global popularity and was available almost on all types of software connected to the Internet.

Why do developers use cURL?

The tool has gained popularity among developers because it can handle rather tricky tasks easily. cURL is scriptable and features a library, allowing for easy scraping without writing special parsing code. The main benefits that you can find in cURL include:

Easy to use for testing and debugging;
Details on transferred data;
Great error logging;
Support of multiple protocols;
Ability to specify URLs, etc.

How to use cURL command with proxy?

Installing cURL

The best thing about cURL is that it is available for use on all contemporary operating systems. In case you are using something older than Windows 10, you may need to download cURL for installation. If you are a Linux user and your distribution does not have cURL, you will be able to install it by entering a simple command:

sudo apt install curl

Once cURL is installed you can run cURL by typing “curl” in the terminal. You should see a prompt for using “curl -help”. The “-help’ command will give you a list of commands you will be able to use.

Also, depending on your operating systems, curl may require the use of double hyphens instead of a single hyphen for flags or commands. You can verify it by running -help or –help. cURL will prompt you on the use of the correct version of syntax.

Connecting a proxy

Regardless of the proxy service (residential or datacenter proxy) you choose for cURL to use proxy with, you will need the following information for setting up cURL using a proxy server:

proxy server address
port
protocol
username (if cURL with proxy authentication is required)
password (if cURL with proxy authentication is required)

Using cURL with HTTP/HTTPS proxy

You can run the following test line to get back your current IP:

curl https://httpbin.org/ip

If the command returns an IP different from your machine’s IP, your proxy is working properly.

If you still need to set up cURL with proxy, you need to run a proper command-line argument of cURL.

For this, you need to open the terminal and run the command line:

curl --help

You will have a huge list of options that will include the following:

-x, --proxy [protocol://]host[:port]

Note that all of the commands for cURL are case-sensitive. And in this case, the cURL proxy settings will be the same as for proxy setup.

So, you can run either one of them to get a cURL proxy initiated. Here is a cURL proxy example:

curl -x "http://user:pwd@127.0.0.1:1234" "http://httpbin.org/ip"

curl --proxy "http://user:pwd@127.0.0.1:1234" "http://httpbin.org/ip"

What you should also notice to set cURL proxy properly is that both the proxy URL (with authorization info) and the target URL are surrounded with double-quotes. This is a good practice of handling special characters in the URLs.

Also, HTTP will act as the default proxy protocol, making the following two commands work for exactly the same cURL proxy configuration:

curl --proxy "http://user:pwd@127.0.0.1:1234" "http://httpbin.org/ip"

curl --proxy "user:pwd@127.0.0.1:1234" "http://httpbin.org/ip"

Using cURL with SOCKS proxy

If you need to use a proxy with SOCKS protocol, the syntax will remain the same as with HTTP proxies:

curl -x "socks5://user:pwd@127.0.0.1:1234" "http://httpbin.org/ip"

If you need to specify the SOCKS version, you can use socks4://, socks4a://, socks5:// or socks5h:// depending on the version.

Another way to set a socks5 proxy with cURL will be by using –socks5 argument instead of -x. If you have a username and password, you can specify them by using –proxy-user switch. Here is an example:

curl --socks5 "127.0.0.1:1234" "http://httpbin.org/ip" --proxy-user user:pwd

Using a .curlrc file

You have already noticed the number of options that are available to use with cURL. The best thing is that you can configure the ones that you want into a configuration file to save you some time. You can specify the file using -K argument but cURL will always look into its home directory ~/.curlrc (_curlrc on Windows) as its default location. If it does not exist already, you can create it and use it every time you need cURL.

Ignoring or overriding proxy for one request

Even if you proxy is set globally or in the .curlrc file, you will still be able to override it and set another proxy by using this simple method.
If you need to override a cURL request with proxy, just set a new proxy with -x or –proxy switch as we described above:

curl --proxy "http://user:pwd@1.0.0.1:8090" "http://httpbin.org/ip"

If you need to bypass all proxies for a request, you should use switch –noproxy followed by “”. This will instruct cURL not to use proxies for any of the connections. Example:

curl --noproxy "" "http://httpbin.org/ip"

Following redirects with cURL

Yet another useful tip for using cURL is to remember that it does not automatically follow redirects. So, for example, we run the following line:
curl http://google.com
expecting that cURL will automatically redirect its request to www.google.com but it will not happen. To make cURL follow redirects we need to use -L switch (all arguments of cURL are case-sensitive).
So, our new line
curl -L http://google.com
should redirect us to www.google.com from google.com.

How to Choose the Right Proxy for cURL

If you need to engage in a lengthy scraping project and need some reliable proxy setup to last undetected for a long time, you can use backconnect (or rotating) proxies provided by PrivateProxy for cURL.

The best choice for scraping will be rotating residential proxies since they will appear as physical addresses to the target sites. Follow this link to find out more information on how to use proxies for web scraping.

If you decide to use datacenter proxies for cURL scraping (and you can certainly do it considering the speed of such proxies), make sure you study the robots.txt file of the target site to make sure you will not raise a lot of red flags and get banned very soon.

Consider PrivateProxy Your Trusted Partner

We at Privateproxy extremely value the customer relations that we establish with our users. So, say if you buy private http proxy from us, we will be happy to provide you with continuous tech support. No matter what proxies will be recommended for your online mission (datacenter or residential proxies), you will always enjoy the ability to approach your account manager with any concerns that you might have. We position ourselves as a one-stop shop for all proxy-related solutions, and we take your say in the matter seriously.

Final Thoughts

As you could see from the guide above, cURL can be a rather useful and flexible tool for a range of proxy-related tasks, from setting up proxies and data export to modifying environment variables and using proxies for scraping and parsing.
Once you are ready to use it with our proxies, make sure you approach our account managers to provide you with the best set of proxies for the mission.

Rate this article, if you like it:

Frequently Asked Questions

Please read our Documentation if you have questions that are not listed below.

How to use cURL with proxy?

When you want to connect cURL with proxy, you will need the following information on your proxy to establish connection: proxy server address, port, protocol, username (if authentication is required), password (if authentication is required).
How to use cURL request with SOCKS proxy?

If you need to use a SOCKS proxy with cURL, the syntax will remain the same as with HTTP proxies:
curl -x "socks5://user:pwd@127.0.0.1:1234" "http://httpbin.org/ip"
If you need to specify the SOCKS version, you can use socks4://, socks4a://, socks5:// or socks5h:// depending on the version.
Is cURL Free?

Yes, cURL is free. It is an open-source software package that is available to all developers who want to use it.
Can I Use cURL without SSL?

This is possible by using ‘-k’ or ‘--insecure’ options in the cURL command line. In this case the SSL validation will be bypassed and all possible warnings regarding invalid SSL certificates will be simply ignored by cURL.

How to enter cURL proxy settings?

You will be able to set up your proxy using cURL with the following arguments:
-x, --proxy [protocol://]host[:port]
They are all case sensitive and are the same for the proxy setup. You can run either one of them to get the proxy initiated. Here is an example of how to do it:
curl -x "http://user:pwd@127.0.0.1:1234" "http://httpbin.org/ip"
or
curl --proxy "http://user:pwd@127.0.0.1:1234" "http://httpbin.org/ip"
Does cURL Use TCP or UDP?

It would be TCP (Transmission Control Protocol) in most cases.
Can I Use cURL with Username and Password Authentication?

Yes, it is possible. In order to use cURL with a username and a password you should apply this command: --user "username:password". It will make it possible to add authentication credentials to the user.
How Can I Check My IP with cURL?

The best way to learn your IP will be through typing “what is my IP address” in Google. However, on a machine running Linux you can view your current IP address by using this cURL command: ‘curl ifconfig.me’.

Previous article Next article