Skip to content

Swith to Master/Detail Publications #242

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
asmacdo merged 3 commits into from
Apr 30, 2019
Merged

Swith to Master/Detail Publications #242

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
66b8e8b
Swith to Master/Detail Publications
asmacdo Apr 26, 2019
4c72c63
Add testable scripts to replace docs snippets.
asmacdo Apr 29, 2019
4a25515
Begin creating Publication CRUD tests
asmacdo Apr 29, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions docs/_scripts/base.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
export BASE_ADDR=http://localhost:24817
export CONTENT_ADDR=http://localhost:24816

wait_for_pulp() {
unset CREATED_RESOURCE
local task_url=$1
while [ -z "$CREATED_RESOURCE" ]

@asmacdo asmacdo Apr 29, 2019

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems to fail sometimes, CREATED_RESOURCE gets set to null.


do
sleep 1
export CREATED_RESOURCE=$(http $BASE_ADDR$task_url | jq -r '.created_resources | first')
done
}
7 changes: 7 additions & 0 deletions docs/_scripts/clean.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
pstop
pclean
prestart
pip uninstall -y shelf-reader

echo "is shelf reader installed?"
pip freeze | grep shelf-reader
14 changes: 14 additions & 0 deletions docs/_scripts/distribution.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Distributions are created asynchronously. Create one, and specify the publication that will
# be served at the base path specified.
export TASK_URL=$(http POST $BASE_ADDR/pulp/api/v3/distributions/ \
name='baz' \
base_path='foo' \
publication=$PUBLICATION_HREF | jq -r '.task')

# Poll the task (here we use a function defined in docs/_scripts/base.sh)
# When the task is complete, it gives us the href for our new Distribution
wait_for_pulp $TASK_URL
export DIST_PATH=${CREATED_RESOURCE[0]}

# Lets inspect the Distribution
http $BASE_ADDR$DIST_PATH
5 changes: 5 additions & 0 deletions docs/_scripts/pip.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
echo 'pip install --trusted-host localhost -i $CONTENT_ADDR/pulp/content/foo/simple/ shelf-reader'
pip install --trusted-host localhost -i $CONTENT_ADDR/pulp/content/foo/simple/ shelf-reader

echo "is shelf reader installed?"
pip freeze | grep shelf-reader
13 changes: 13 additions & 0 deletions docs/_scripts/publication.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Create a new publication specifying the repository_version.
# Alternatively, you can specify the repository, and Pulp will assume the latest version.
export TASK_URL=$(http POST $BASE_ADDR/pulp/api/v3/publications/python/pypi/ \
repository_version=$REPOVERSION_HREF | jq -r '.task')

# Poll the task (here we use a function defined in docs/_scripts/base.sh)
# When the task is complete, it gives us a new Publication
wait_for_pulp $TASK_URL
export PUBLICATION_HREF=${CREATED_RESOURCE[0]}


# Lets inspect our newly created Publication.
http $BASE_ADDR$PUBLICATION_HREF
19 changes: 19 additions & 0 deletions docs/_scripts/quickstart.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# This script will execute the component scripts and ensure that the documented examples
# work as expected.

@asmacdo asmacdo Apr 29, 2019

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Idea for another day:

It might be worth running quickstart.sh when building the docs, to test it out. As a possible side benefit, we will populate the db, which might allow us to include examples in the REST API docs.


# THIS SCRIPT CURRENTLY MUST BE RUN IN A PULPLIFT DEVELOPMENT ENVIRONMENT
# TODO: remove the usage of pulp-devel bash functions so they can be directly modified
# for user environments.

# From the _scripts directory, run with `source quickstart.sh` (source to preserve the environment
# variables)
source clean.sh
source base.sh

source repo.sh
source remote.sh
source sync.sh

source publication.sh
source distribution.sh
source pip.sh
11 changes: 11 additions & 0 deletions docs/_scripts/remote.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Create a remote that syncs some versions of django into your repository.
http POST $BASE_ADDR/pulp/api/v3/remotes/python/python/ \
name='bar' \
url='https://pypi.org/' \
includes:='[{"name": "shelf-reader"}]'

# Export an environment variable for the new remote URI.
export REMOTE_HREF=$(http $BASE_ADDR/pulp/api/v3/remotes/python/python/ | jq -r '.results[] | select(.name == "bar") | ._href')

# Lets inspect our newly created RepositoryVersion.
http $BASE_ADDR$REMOTE_HREF
10 changes: 10 additions & 0 deletions docs/_scripts/repo.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Start by creating a new repository named "foo":
http POST $BASE_ADDR/pulp/api/v3/repositories/ name=foo

# If you want to copy/paste your way through the guide,
# create an environment variable for the repository URI.
export REPO_HREF=$(http $BASE_ADDR/pulp/api/v3/repositories/ | \
jq -r '.results[] | select(.name == "foo") | ._href')

# Lets inspect our newly created repository.
http $BASE_ADDR$REPO_HREF
11 changes: 11 additions & 0 deletions docs/_scripts/sync.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Using the Remote we just created, we kick off a sync task
export TASK_URL=$(http POST $BASE_ADDR$REMOTE_HREF'sync/' repository=$REPO_HREF mirror=False \
| jq -r '.task')

# Poll the task (here we use a function defined in docs/_scripts/base.sh)
# When the task is complete, it gives us a new repository version
wait_for_pulp $TASK_URL
export REPOVERSION_HREF=${CREATED_RESOURCE[0]}

# Lets inspect our newly created RepositoryVersion
http $BASE_ADDR$REPOVERSION_HREF
2 changes: 1 addition & 1 deletion docs/_static/api.json
View file
Open in desktop

Large diffs are not rendered by default.

10 changes: 3 additions & 7 deletions docs/workflows/index.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ If you have not yet installed the Python plugins on your Pulp installation, plea
:doc:`../installation`. These documents will assume you have the environment installed and
ready to go.

httpie
------

**httpie**

The REST API examples here use `httpie <https://httpie.org/doc>`_ to perform the requests.
The ``httpie`` commands below assume that the user executing the commands has a ``.netrc`` file
Expand All @@ -32,8 +32,7 @@ set is the hostname and port::

$ export BASE_ADDR=http://<hostname>:24817

jq
--
**jq**

This documentation makes use of the `jq library <https://stedolan.github.io/jq/>`_
to parse the json received from requests, in order to get the unique urls generated
Expand All @@ -42,9 +41,6 @@ library with:

``$ sudo dnf install jq``

Python Workflows
----------------

.. toctree::
:maxdepth: 2

Expand Down
88 changes: 38 additions & 50 deletions docs/workflows/publish-host.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,66 +4,54 @@ Publish and Host
This section assumes that you have a repository with content in it. To do this, see the
:doc:`sync` or :doc:`upload` documentation.

Create a Publisher
------------------
Create a new Publication
------------------------

Publishers contain extra settings for how to publish. You can use a Python publisher on any
repository that contains Python content::
Kick off a publish task by creating a new Publication. The publish task will generate all the
metadata that ``pip`` needs to install packages (newly created Publications are not consumable
yet).

$ http POST $BASE_ADDR/pulp/api/v3/publishers/python/python/ name=bar
.. literalinclude:: ../_scripts/publication.sh
:language: bash

Response::
Publication GET Response::

{
"_href": "/pulp/api/v3/repositories/foo/publishers/python/python/1/",
...
"_created": "2019-04-29T15:58:04.939836Z",
"_distributions": [],
"_href": "/pulp/api/v3/publications/python/pypi/4cc1ddbb-64ff-4795-894a-09d5ca372774/",
"_type": "python.python",
"publisher": null,
"repository": "http://localhost:24817/pulp/api/v3/repositories/%3CRepository:%20foo%3E/",
"repository_version": "/pulp/api/v3/repositories/1b2b0af1-5588-4b4b-b2f6-cdd3a3e1cd36/versions/1/"
}

Create a variable for convenience.::
Reference: `Python Publication Usage <../restapi.html#tag/publications>`_

$ export PUBLISHER_HREF=$(http $BASE_ADDR/pulp/api/v3/publishers/python/python/ | jq -r '.results[] | select(.name == "bar") | ._href')

Reference: `Python Publisher API Usage <../restapi.html#tag/publishers>`_


Publish a repository with a publisher
-------------------------------------

Use the remote object to kick off a publish task by specifying the repository version to publish.
Alternatively, you can specify repository, which will publish the latest version.

The result of a publish is a publication, which contains all the information needed for ``pip`` to
use. Publications are not consumable until they are hosted by a distribution::

$ http POST $BASE_ADDR$PUBLISHER_HREF'publish/' repository=$REPO_HREF

Response::

{
"task": "/pulp/api/v3/tasks/fd4cbecd-6c6a-4197-9cbe-4e45b0516309/"
}

Create a variable for convenience.::

$ export PUBLICATION_HREF=$(http $BASE_ADDR/pulp/api/v3/publications/ | jq -r --arg PUBLISHER_HREF "$PUBLISHER_HREF" '.results[] | select(.publisher==$PUBLISHER_HREF) | ._href')

Reference: `Python Publish Task Usage <../restapi.html#operation/publishers_python_python_publish>`_

Host a Publication (Create a Distribution)
--------------------------------------------
Create a Distribution (Host a Publication)
------------------------------------------

To host a publication, (which makes it consumable by ``pip``), users create a distribution which
will serve the associated publication at ``/pulp/content/<distribution.base_path>`` as demonstrated
in :ref:`using distributions<using-distributions>`::

$ http POST $BASE_ADDR/pulp/api/v3/distributions/ name='baz' base_path='foo' publication=$PUBLICATION_HREF

Response::

{
"_href": "/pulp/api/v3/distributions/1/",
...
}
will serve the associated publication at ``$CONTENT_HOST/pulp/content/<distribution.base_path>`` as
demonstrated in :ref:`using distributions<using-distributions>`.

.. literalinclude:: ../_scripts/distribution.sh
:language: bash

Distribution GET Response::

{
"_created": "2019-04-29T15:58:07.586250Z",
"_href": "/pulp/api/v3/distributions/63d532cc-94ea-402c-bb6d-2340242c6df8/",
"base_path": "foo",
"base_url": "localhost:24816/pulp/content/foo",
"content_guard": null,
"name": "baz",
"publication": "/pulp/api/v3/publications/python/pypi/4cc1ddbb-64ff-4795-894a-09d5ca372774/",
"publisher": null,
"remote": null,
"repository": null
}

Reference (pulpcore): `Distribution API Usage
<https://docs.pulpproject.org/en/3.0/nightly/restapi.html#tag/distributions>`_
Expand Down
Loading