asyncio: WebSocketClient:connection closed (ERR): sent 1011

[Michel777]

Hi,

I'm using Python asyncio, everything is working fine. The only issue is, that the (*) writing data from queue into the database: df.to_sql causes the error:

"WebSocketClient:connection closed (ERR): sent 1011 (unexpected error) keepalive ping timeout; no close frame received."

1. The WebSocket puts every minute up to 10.000 items,
   
2. within max 10 seconds ( e.g. between 09:08:00 - 09:08:10 )
   
3. the methode writes every items separatelly into the database (*) , because I could not find a solution for collecting data for a minute and write them once pro minute.
   
4. There is enough time for writing ( e.g. between 09:08:11 - 09:08:59 ).
   

Question: How can I write the dataframe ( df.to_sql ...) every xx:15 (e.g. 09:08:15) ?

Thanks for your suggestions.

Regards, Michel

    async def start_processing_api_calls(self):
            while True:
                ticker = await self.api_call_queue.get()
                try:
                    data = (
                        ticker.symbol,
                        ticker.volume
                    )
                    self.current_batch.append(data)
                    df = pd.DataFrame(self.current_batch)
                    df.columns = self.COLUMN_NAMES
                    df.to_sql(self.TableName, con=self.Engine, schema=self.SchemaName, if_exists='append', index=False)        
                    self.current_batch = []               
                except Exception as e:
                    logging.error(f"Error processing API call for {ticker}: {e}")
                finally:
                    self.api_call_queue.task_done()