Using a Proxy Server

[rsherry8]

Please consider the following python program.

import requests

url = 'https://washingtonpost.com/'

proxy = {
        "https": "https://202.29.237.212:3128",
        "http": "http://202.29.237.212:3128"
}

text = requests.get(url, proxies=proxy)
# text = requests.get(url)

output = open("out.html", "w")

output.write(text.text)

output.close()

When I run the program, it dies on the call to get with the following error messages:
ProxyError: HTTPSConnectionPool(host='www.washingtonpost.com', port=443): Max retries exceeded with url: / (Caused by ProxyError('Cannot connect to proxy.', NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x000001BC70C2FE48>: Failed to establish a new connection: [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond',)))

However, if I do not use the proxy server it works. What am I doing wrong?
Thanks,
Bob

[snippsat]

However, if I do not use the proxy server it works. What am I doing wrong?

It's most likely the proxy that's has gone down,i get same error.
Here a test with another proxy,i just used list here
No error and it save.

import requests

url = 'https://washingtonpost.com/'
proxy = {"http": "167.71.91.204:80"}
text = requests.get(url, proxies=proxy)
# text = requests.get(url)

output = open("out.html", "w")
output.write(text.text)
output.close()

[rsherry8]

snippsat,

Thanks for the response and your code does work for me. What I am wondering is, how long do these proxy servers stay up? Do I need to change the proxy server every day?

Bob

[snippsat]

Free proxies can be unstable and not stay up so long.
Can look at this list

[EthanPayne]

Are proxies better to use than VPN?

[snippsat]

Are proxies better to use than VPN?

They are similar,but VPN is more secure as all traffic is encrypted.
Now can proxies be more geared against web scraping operations.
VPN services are mostly oriented towards a single user,
who looking to not only mask their IP address,but also to make sure that their internet requests are encrypted.

Personally so do i use Astrill for VPN,used it for 7-years now.
Has also own setting to us proxy severs.

If doing a bigger Businesses like scrape project,then can rotating proxy service help eg Crawlera.

[EthanPayne]

Hey thanks for the response, I wanted to get Nord VPN, there are always codes that lower the price a lot. I will check out Astrill for sure!

[EthanPayne]

Hey thanks for the response, I wanted to get Nord VPN, there are always codes that lower the price a lot. I will check out Astrill for sure!

Hey thanks for the response, I wanted to get NordVPN, there are always codes that lower the price a lot. I will check out Astrill for sure! Most of the people are saying that VPN is better, but they also slow down your PC a little bit and that is not good if you have an old one. Thankfully my PC is not slow and I think I will be ok. You are right that proxies are for more users and that is why TPB is using them to be available for us. You can find those proxies easily on [click-bite link removed] and use them any time you want.

[mikekane]

No way proxies are just meant for hiding the IP address and only on browser means the rest of the applications running on your system will expose your IP address. VPN provides encryption and secures your whole system. Another one is TOR that provides a max level of privacy. For more details read here
https://www.purevpn.com/blog/tor-vpn-proxy/

[lucasbrown]

Actually yes I am using I recommend Web Scraping, Actually, I have read a guide about Web Scraping and proxies so I have changed the way to work, I recommend you guys to read that guide: https://www.noustra.com/