How to send GET requests via python over different IPs?
up vote
-1
down vote
favorite
I am currently trying to gather data from an exchange for deep learning. The issue is I need the data with a high resolution (second intervals), therefore I am creating a lot of GET requests for each currency. But I reach fast the limits of the API really.
I would like to know if there is a possibility to route the GET request over a public proxy server to claim to have a different IP-address to make as many requests as possible with a Raspberry PI.
python https proxy python-requests
edited Nov 11 at 3:14
Santosh Kumar
10.2k134392
asked Nov 11 at 3:09
Synaps
375
add a comment |
up vote
-1
down vote
favorite
I am currently trying to gather data from an exchange for deep learning. The issue is I need the data with a high resolution (second intervals), therefore I am creating a lot of GET requests for each currency. But I reach fast the limits of the API really.
I would like to know if there is a possibility to route the GET request over a public proxy server to claim to have a different IP-address to make as many requests as possible with a Raspberry PI.
python https proxy python-requests
edited Nov 11 at 3:14
Santosh Kumar
10.2k134392
asked Nov 11 at 3:09
Synaps
375
1
You want us to help you circumvent the rate limit of an API?
– Charles Landau
Nov 11 at 3:19
I need to, else I won't be able to get the needed data
– Synaps
Nov 11 at 10:05
add a comment |
up vote
-1
down vote
favorite
up vote
-1
down vote
favorite
I am currently trying to gather data from an exchange for deep learning. The issue is I need the data with a high resolution (second intervals), therefore I am creating a lot of GET requests for each currency. But I reach fast the limits of the API really.
I would like to know if there is a possibility to route the GET request over a public proxy server to claim to have a different IP-address to make as many requests as possible with a Raspberry PI.
python https proxy python-requests
edited Nov 11 at 3:14
Santosh Kumar
10.2k134392
asked Nov 11 at 3:09
Synaps
375
I am currently trying to gather data from an exchange for deep learning. The issue is I need the data with a high resolution (second intervals), therefore I am creating a lot of GET requests for each currency. But I reach fast the limits of the API really.
I would like to know if there is a possibility to route the GET request over a public proxy server to claim to have a different IP-address to make as many requests as possible with a Raspberry PI.
python https proxy python-requests
python https proxy python-requests
edited Nov 11 at 3:14
Santosh Kumar
10.2k134392
asked Nov 11 at 3:09
Synaps
375
edited Nov 11 at 3:14
Santosh Kumar
10.2k134392
asked Nov 11 at 3:09
Synaps
375
edited Nov 11 at 3:14
Santosh Kumar
10.2k134392
edited Nov 11 at 3:14
Santosh Kumar
10.2k134392
edited Nov 11 at 3:14
Santosh Kumar
10.2k134392
10.2k134392
asked Nov 11 at 3:09
Synaps
375
asked Nov 11 at 3:09
Synaps
375
asked Nov 11 at 3:09
Synaps
375
375
1
You want us to help you circumvent the rate limit of an API?
– Charles Landau
Nov 11 at 3:19
I need to, else I won't be able to get the needed data
– Synaps
Nov 11 at 10:05
add a comment |
1
You want us to help you circumvent the rate limit of an API?
– Charles Landau
Nov 11 at 3:19
I need to, else I won't be able to get the needed data
– Synaps
Nov 11 at 10:05
1
1
You want us to help you circumvent the rate limit of an API?
– Charles Landau
Nov 11 at 3:19
You want us to help you circumvent the rate limit of an API?
– Charles Landau
Nov 11 at 3:19
I need to, else I won't be able to get the needed data
– Synaps
Nov 11 at 10:05
I need to, else I won't be able to get the needed data
– Synaps
Nov 11 at 10:05
add a comment |
1 Answer
1
active
oldest
votes
up vote
1
down vote
accepted
If you need to make a GET with different proxies/IP addresses...
Try proxy-requests. I am not sure if that's what you are asking, but it seems like maybe you reached your request limit to a your API endpoint. Some organizations limit the number of requests allowed from certain IP addresses.
Here's an example GET using python proxy requests so that your IP will be different:
pip3 install proxy-requests
from proxy_requests.proxy_requests import ProxyRequests
r = ProxyRequests("https://api.ipify.org")
r.get()
pypi
github
The module seems to use scraped proxies to make the request. I just tried quickly and it worked
answered Nov 11 at 4:06
sit_on_a_pan_otis
977
add a comment |
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
1
down vote
accepted
If you need to make a GET with different proxies/IP addresses...
Try proxy-requests. I am not sure if that's what you are asking, but it seems like maybe you reached your request limit to a your API endpoint. Some organizations limit the number of requests allowed from certain IP addresses.
Here's an example GET using python proxy requests so that your IP will be different:
pip3 install proxy-requests
from proxy_requests.proxy_requests import ProxyRequests
r = ProxyRequests("https://api.ipify.org")
r.get()
pypi
github
The module seems to use scraped proxies to make the request. I just tried quickly and it worked
answered Nov 11 at 4:06
sit_on_a_pan_otis
977
add a comment |
up vote
1
down vote
accepted
If you need to make a GET with different proxies/IP addresses...
Try proxy-requests. I am not sure if that's what you are asking, but it seems like maybe you reached your request limit to a your API endpoint. Some organizations limit the number of requests allowed from certain IP addresses.
Here's an example GET using python proxy requests so that your IP will be different:
pip3 install proxy-requests
from proxy_requests.proxy_requests import ProxyRequests
r = ProxyRequests("https://api.ipify.org")
r.get()
pypi
github
The module seems to use scraped proxies to make the request. I just tried quickly and it worked
answered Nov 11 at 4:06
sit_on_a_pan_otis
977
add a comment |
up vote
1
down vote
accepted
up vote
1
down vote
accepted
If you need to make a GET with different proxies/IP addresses...
Try proxy-requests. I am not sure if that's what you are asking, but it seems like maybe you reached your request limit to a your API endpoint. Some organizations limit the number of requests allowed from certain IP addresses.
Here's an example GET using python proxy requests so that your IP will be different:
pip3 install proxy-requests
from proxy_requests.proxy_requests import ProxyRequests
r = ProxyRequests("https://api.ipify.org")
r.get()
pypi
github
The module seems to use scraped proxies to make the request. I just tried quickly and it worked
answered Nov 11 at 4:06
sit_on_a_pan_otis
977
If you need to make a GET with different proxies/IP addresses...
Try proxy-requests. I am not sure if that's what you are asking, but it seems like maybe you reached your request limit to a your API endpoint. Some organizations limit the number of requests allowed from certain IP addresses.
Here's an example GET using python proxy requests so that your IP will be different:
pip3 install proxy-requests
from proxy_requests.proxy_requests import ProxyRequests
r = ProxyRequests("https://api.ipify.org")
r.get()
pypi
github
The module seems to use scraped proxies to make the request. I just tried quickly and it worked
answered Nov 11 at 4:06
sit_on_a_pan_otis
977
edited Nov 11 at 4:18
answered Nov 11 at 4:06
sit_on_a_pan_otis
977
answered Nov 11 at 4:06
sit_on_a_pan_otis
977
answered Nov 11 at 4:06
sit_on_a_pan_otis
977
977
add a comment |
add a comment |
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53245519%2fhow-to-send-get-requests-via-python-over-different-ips%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
You want us to help you circumvent the rate limit of an API?
– Charles Landau
Nov 11 at 3:19
I need to, else I won't be able to get the needed data
– Synaps
Nov 11 at 10:05