Getting Socket timeout when trying to get a json from api

Hallo there,

I am trying to get a json file as a response to a url call and I get this error.

The url is following:

https://dsob.opap.gr/sb/sport/soccer/coupon?locale=el&onlyLive=false&marketIds=1&fromDate=2023-08-15&toDate=2023-08-15

What I see, is that the data are perfect loaded when trying from their page

https://pamestoixima.opap.gr/

OkHttpClient client = new OkHttpClient.Builder()    .connectTimeout(50, TimeUnit.SECONDS)    .writeTimeout(50, TimeUnit.SECONDS)    .readTimeout(50, TimeUnit.SECONDS)    .build(); Request request = new Request.Builder().url(url)                                        .addHeader("User-Agent","Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko)                                                       Chrome/115.0.0.0 Safari/537.36 Edg/115.0.1901.203") Response response = client.newCall(request).execute();

Am I missing something?

Please show us how you initialize the url variable.

Stephan van Hulst wrote:Please show us how you initialize the url variable.

I don't get any errors other than Socket timeout.

The url that i am using is

https://dsob.opap.gr/sb/sport/soccer/coupon?locale=el&onlyLive=false&marketIds=1%2C2%2C31%2C21%2C18%2C18%2C18%2C14&fromDate=2023-08-15&toDate=2023-08-15

where 1%2C2%2C31%2C21%2C18%2C18%2C18%2C14 are the markets shown in the page.

Question: Can a page somehow protect its data and forbid other applications except browsers, to make calls on its api?

Sot Nikele wrote:Question: Can a page somehow protect its data and forbid other applications except browsers, to make calls on its api?

Yes, they absolutely can do that. And football betting sites will be very diligent about blocking access by people who haven't logged in.

Paul Clapham wrote:

Sot Nikele wrote:Question: Can a page somehow protect its data and forbid other applications except browsers, to make calls on its api?

Yes, they absolutely can do that. And football betting sites will be very diligent about blocking access by people who haven't logged in.

So they can bypass the header that I am adding about the User Agent at the Response object also?

Sot Nikele wrote:So they can bypass the header that I am adding about the User Agent at the Response object also?

Apparently so.

I don't know what the site policy is for scraping information, but you can make this work by adding an Accept header specifying the type of content you are wanting.
final String url = "https://dsob.opap.gr/sb/sport/soccer ...";  // truncated value to conserve space OkHttpClient client = new OkHttpClient.Builder()            .connectTimeout(50, TimeUnit.SECONDS)            .writeTimeout(50, TimeUnit.SECONDS)            .readTimeout(50, TimeUnit.SECONDS)            .build();         Request request = new Request.Builder().url(url)                        .addHeader("User-Agent","Mozilla/5.0 ...")  // truncated value to conserve space                        .addHeader("Accept", "application/json")                        .build(); Response response = client.newCall(request).execute(); System.out.println("Response code: " + response.code()); System.out.println("Response content: " + response.body().string().substring(0,80) + "..."); Response code: 200 Response content: {"locMap":{"ARGE":{"n":"Αργεντινή"},"ASIA":{"n":"Ασία"},"AUST":{"n":"Αυστρία"},"...