Skip to content

Updated self-hosted guidelines for Linux and Mac #90

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
konradpabjan merged 4 commits into from
May 11, 2020
Merged

Updated self-hosted guidelines for Linux and Mac #90

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
e54237e
Updated self-hosted guidelines
konradpabjan May 8, 2020
9f57c0f
Apply suggestions from code review
konradpabjan May 11, 2020
9dee9e9
PR Feedback
konradpabjan May 11, 2020
287a2e4
Apply suggestions from code review
konradpabjan May 11, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Updated self-hosted guidelines
  • Loading branch information
@konradpabjan
konradpabjan committed May 8, 2020
commit e54237e30cad9c25a160f285dad1be4ad67e691a
30 changes: 25 additions & 5 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -136,14 +136,34 @@ You should specify only a major and minor version if you are okay with the most

# Using `setup-python` with a self hosted runner

If you would like to use `setup-python` and a self-hosted runner, there isn't much that you need to do. When `setup-python` is run for the first time with a version of Python that it doesn't have, it will download the appropriate version, and set up the tools cache on your machine. Any subsequent runs will use the Python versions that were previously downloaded.
If you would like to use `setup-python` and a self-hosted runner, there are a few extra things you need to make sure are set up so that new versions of Python can be downloaded and configured on your runner.

@brcrista brcrista May 8, 2020 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make sure are set up

+1 👍


A few things to look out for when `setup-python` is first setting up the tools cache
- If using Windows, your runner needs to be running as an administrator so that the appropriate directories and files can be setup. On Linux and Mac, you also need to be running with elevated permissions
- On Windows, you need `7zip` installed and added to your `PATH` so that files can be extracted properly during setup
- MSI installers are used when setting up Python on Windows. A word of caution as MSI installers update registry settings
### Windows

- Your runner needs to be running with administrator privileges so that the appropriate directories and files can be setup when downloading and installing a new version of Python for the first time.
Comment thread
konradpabjan marked this conversation as resolved.
Outdated
- You need `7zip` installed and added to your `PATH` so that the downloaded versions of Python files can be extracted properly during first-time setup.
- MSI installers are used when setting up Python on Windows. A word of caution as MSI installers update registry settings.
- The 3.8 MSI installer for Windows will not let you install another 3.8 version of Python. If `setup-python` fails for a 3.8 version of Python, make sure any previously installed versions are removed by going to "Apps & Features" in the Settings app.

### Linux

- The Python packages that are downloaded from `actions/python-versions` are originally compiled from source in `/opt/hostedtoolcache/` with the [--enable-shared](https://github.com/actions/python-versions/blob/94f04ae6806c6633c82db94c6406a16e17decd5c/builders/ubuntu-python-builder.psm1#L35) flag which makes them non portable.
Comment thread
konradpabjan marked this conversation as resolved.
Outdated
- Create an environment variable called `AGENT_TOOLSDIRECTORY` and set it to `/opt/hostedtoolcache`. This is used to control where Python gets set up and installed.

@brcrista brcrista May 8, 2020

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be better if we could use RUNNER_TOOLSDIRECTORY. I think it would be easiest just to shim it in the action -- have setup-python set AGENT_TOOLSDIRECTORY to the value of RUNNER_TOOLSDIRECTORY.

@konradpabjan konradpabjan May 11, 2020 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We got rid of the RUNNER_TOOLSDIRECTORY env variable, see the comment here: #85 (comment)

If you search through this repo, there are no more references to it.

Also, nix-template-setup.sh doesn't have any references to RUNNER_TOOLSDIRECTORY so we should just stick to AGENT_TOOLSDIRECTORY

Comment thread
konradpabjan marked this conversation as resolved.
Outdated
- In the same shell that your runner is using, type `AGENT_TOOLSDIRECTORY=/opt/hostedtoolcache`

@brcrista brcrista May 8, 2020

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't they have to export?

@konradpabjan konradpabjan May 11, 2020

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's in the same shell I had the runner pick it up without export. Regardless, I added export to the example

- A more permanent way of setting the environment variable is to create a `.env` file in the same directory as your runner and to add `AGENT_TOOLSDIRECTORY=/opt/hostedtoolcache`. This ensures the variable is always set if your runner is configured as a service.
- It is not possible to start the Linux runner with `sudo` and the `/opt` directory usually requires root privileges to write to. To get around this you can change certain permissions.

@brcrista brcrista May 8, 2020 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More generally, "the user starting the runner must have write permission to the /opt directory." This can be accomplished in different ways:

  • Runner user is the owner, and owner has write permission
  • Runner user is in the owning group, and owning group has write permission
  • All users have write permission

@brcrista brcrista May 8, 2020 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, I'm not sure what the implications are for configuring the runner as a service. I know Windows services have a different set of users instead of running as the logged-on user (by definition, this is what makes them services).

@konradpabjan konradpabjan May 11, 2020

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So on Linux and Mac we should be fine. Doing sudo ./svc.sh install shows the user the service will run on (at least for me it's showing the same thing, konradpabjan): https://help.github.com/en/actions/hosting-your-own-runners/configuring-the-self-hosted-runner-application-as-a-service

On Windows, you need to be running as admin to configure the runner as a service:
image

Even when running as admin, it asks for the user account that you would like the service to run as, I'll add a note to make sure users watch out for this:
image

- Create a directory called `hostedtoolcache` inside `/opt`.
- Change Permissions using `chown`
- Change the user and group to be the same as the runners. You can get this information by using the `ls -l` command inside the runners root directory.
- `sudo chown runner-user:runner-group hostedtoolcache/`

### Mac

- The same setup that applies to `Linux` also applies to `Mac`, just with a different tools cache directory.
- Set the `AGENT_TOOLSDIRECTORY` environment variable to `/Users/runner/hostedtoolcache`.
- Change the permissions of `/Users/runner/hostedtoolcache` so that the runner has write access.


# Using Python without `setup-python`

`setup-python` helps keep your dependencies explicit and ensures consistent behavior between different runners. If you use `python` in a shell on a GitHub hosted runner without `setup-python` it will default to whatever is in PATH. The default version of Python in PATH vary between runners and can change unexpectedly so we recommend you always use `setup-python`.
Expand Down