Loops and Child Attributes

[antweeman82]

Hi,

I'm a bit of a novice with Python in terms of creating scripts (I tend to edit / read-only) and am really struggling to understand how to use loops for accessing the Fantasy Premier League API, if this is at all the way to do it. It may be FPL, but it also helps me upskill my Python skills for work!

From the FPL API endpoint https://fantasy.premierleague.com/api/event/36/live/, my objective is to extract to CSV all the "stats" attributes, with each one as an individual header column associated to the related ID. I can only extract the "stats" (delimited) and "ID" based on the parent-child structure, but need to have the following:

Output:
ID Minutes Goals Scored Assists Clean Sheets etc.
5      90         0        0        0           1

Code so far:

import requests
import json
import numpy as np
import pandas as pd
import datetime

# Make a get request to get the latest player data from the FPL API
link = "https://fantasy.premierleague.com/api/event/36/live/"
response = requests.get(link)
# Convert JSON data to a python object
data = json.loads(response.text)
# Initialize array to hold ALL player data
# This will be a 2D array where each row is a different player
all_players = []
# Loop through each player in the data 
for i in data["elements"]:
    id = i['id']
    assists = i['assists']
    goals_scored = i['goals_scored']

# Create a 1D array of the current players stats
    individual_stats = [id, assists, goals_scored]

# Append the player array to a 2D array of all players
    all_players.append(individual_stats)
# Convert the 2D array to a numpy array
all_players = np.array(all_players)
# Convert the numpy array to a pandas dataframe (table)
dataset = pd.DataFrame({'id': all_players[:, 0], 'goals': all_players[:, 1], 'assists': all_players[:, 2]})


# Generate a unique filename based on date
filename = str(datetime.datetime.today().date()) + '_fpl_players_weekly)'

# Save the table of data as a CSV
dataset.to_csv(index=False, path_or_buf=filename)

JSON Response eg:

Output:
"elements": [
        {
            "id": 1,
            "stats": {
                "minutes": 0,
                "goals_scored": 0,
                "assists": 0,
                "clean_sheets": 0,
                "goals_conceded": 0,
                "own_goals": 0,
                "penalties_saved": 0,
                "penalties_missed": 0,
                "yellow_cards": 0,
                "red_cards": 0,
                "saves": 0,
                "bonus": 0,
                "bps": 0,
                "influence": "0.0",
                "creativity": "0.0",
                "threat": "0.0",
                "ict_index": "0.0",
                "starts": 0,
                "expected_goals": "0.00",
                "expected_assists": "0.00",
                "expected_goal_involvements": "0.00",
                "expected_goals_conceded": "0.00",
                "total_points": 0,
                "in_dreamteam": false
            },
            "explain": [
                {
                    "fixture": 351,
                    "stats": [
                        {
                            "identifier": "minutes",
                            "points": 0,
                            "value": 0
                        }
                    ]
                }
            ]
        },
        {
            "id": 2,
            "stats": {
                "minutes": 0,
                "goals_scored": 0,
                "assists": 0,
                "clean_sheets": 0,
                "goals_conceded": 0,
                "own_goals": 0,
                "penalties_saved": 0,
                "penalties_missed": 0,
                "yellow_cards": 0,
                "red_cards": 0,
                "saves": 0,
                "bonus": 0,
                "bps": 0,
                "influence": "0.0",
                "creativity": "0.0",
                "threat": "0.0",
                "ict_index": "0.0",
                "starts": 0,
                "expected_goals": "0.00",
                "expected_assists": "0.00",
                "expected_goal_involvements": "0.00",
                "expected_goals_conceded": "0.00",
                "total_points": 0,
                "in_dreamteam": false
            },
            "explain": [
                {
                    "fixture": 351,
                    "stats": [
                        {
                            "identifier": "minutes",
                            "points": 0,
                            "value": 0
                        }
                    ]
                }
            ]
        }

Thanks
Anthony

buran write May-10-2024, 04:04 PM:
Please, use proper tags when post code, traceback, output, etc. This time I have added tags for you.
See BBcode help for more info.

[deanhystad]

You can simplify the code that makes the dataframe:

# make a list of players
players = []
for element in elements:
    player = {"id", element["id"]}  # Makes a dictionary for the player
    player.update{element.stats}   # Adds all stats to the player.
    players.append(player)

# Make dataframe containing all players and stats.
dataset = pd.DataFrame(players)

# Can make dataframe that has particular stats
offense = dataset[["id", "goals_scored", "assists"]

[menator01]

I took a little different approach

# Do imports
import requests, json
import pandas as pd
from datetime import datetime
import os 

# Get the path of the working script
path = os.path.realpath(os.path.dirname(__file__))

# The link
link = 'https://fantasy.premierleague.com/api/event/36/live/'

# Get response
response = requests.get(link)

# Covert to json format
data = json.loads(response.text)

# Create the column headers
columns = [element for element in data['elements'][0]['stats']]

# # Empty list
stats = []

# Get the data
for element in data['elements']:
    stats.append([element['stats'][key] for key in element['stats']])

# # Create the dataframe
dataset = pd.DataFrame(stats, columns=columns)

# # Create the filename
filename = f'{path}/{datetime.today().date()}_fpl_players_weekly'

# # Convert to csv
dataset.to_csv(index=False, path_or_buf=filename)

[antweeman82]

Great thanks @menator01 - this works perfectly!

[antweeman82]

Hi @menator01

Just coming back to this one as the FPL is starting - the code you suggested pulls out the stats in delimited columns as requested perfectly; however, I've just noticed that it doesn't extract / append element['id'] to the data.

Looks like @deanhystad started to do so by creating a dictionary and then appending, but I got a bit lost.

I'm a bit stumped...

If you can help, be great.

Thanks

[menator01]

The link doesn't have any data

[Pedroski55]

Well, this won't overload your hd:

Output:
response
<Response [200]>
response.text
'{"elements":[]}'

The emphasis is on Fantasy one presumes: Fantasy Premier League: Imagine what you will!

[antweeman82]

@menator01 - ah yep, new season so they must reset the data, will have to wait for the updated live endpoint for week 1 - https://fantasy.premierleague.com/api/event/1/live/.

I was creating my offline tables using the data pulled in May, and noticed the IDs missing.

Will ping when it's live - appreciate your help