The correct POST request

[abhie_lp]

Hi everyone. I have been trying to scrape a website to get the data of the next day. https://www.powerexindia.com/code/fronte...rice.html/. I am sending the correct data in POST request but still i am getting the webpage of current date. There is only error if I do not send the csrftoken otherwise no matter what date i send i am getting the webpage of current date data. Please help in knowing where i am wrong.

        with requests.Session() as session:
            try:
                res = session.get(self.BASE_URL, headers=headers)
            except requests.exceptions.Timeout as e:
                print(standard_datetime(), FILENAME, date, e)
            else:
                csrf = res.cookies.get("csrftoken")
                if csrf is None:
                    return None
            headers.update({"X-CSRFToken": csrf})
            cookies = {"csrftoken": csrf}
            payload = {
                "dataview12": "Daily",
                "fromhour": "0",
                "tohour": "95",
                "DeliveryDate": "2020-06-07",
                "csrfmiddlewaretoken": csrf,
                "markall": "%",
                "N1": "N1", "N2": "N2", "N3": "N3",
                "E1": "E1", "E2": "E2",
                "W1": "W1", "W2": "W2", "W3": "W3",
                "S1": "S1", "S2": "S2", "S3": "S3",
                "A1": "A1", "A2": "A2",
                "DeliveryfromDate": "", "DeliverytoDate": "",
                "year": "2020", "month": "1",
                "submit": "submit"
            }
            print(payload)
            try:
                res = session.post(
                    self.BASE_URL, headers=headers, cookies=cookies
                )
            except requests.exceptions.Timeout as e:
                print(standard_datetime(), FILENAME, date, e)
            except requests.exceptions.SSLError as e:
                print(standard_datetime(), FILENAME, date, e)
            except requests.exceptions.ConnectionError as e:
                print(standard_datetime(), FILENAME, date, e)
            else:
                return res

[buran]

I am sending the correct data in POST request

you never post the payload (assuming it's a correct one)

[lmolter54]

This won't answer the original question -- it's just FYI. I'm no expert, but I have come across commercial web services that require a POST request with parameters in order to retrieve the data. I don't think you can send a JSON payload with a GET request, right? Again, I only ran into this once with a service we had to call. Most other times, I use query strings. <Disclaimer: I'm retired and old and may not know what I'm talking about>.

[buran]

Their POST request is on line 30. The get request is used to get CSRF token

[abhie_lp]

Their POST request is on line 30. The get request is used to get CSRF token

I updated it. Actually i was trying whether i will get any error if i send the POST request without data. Thanks.

[buran]

So do you still have a problem? Post your actual code